The Spurs defender, who scored against the Claret and Blue at White Hart Lane last season, would cost around £5m.
Harry Redknapp was confident of securing his signature but it appears Villa have stolen a march on the Fratton Park club.
If Kaboul does sign for the Midlands side it will be the second time in as many days that they have beaten Pompey to a player after a deal for Nicky Shorey was completed on Thursday.