Referee Neil Swarbrick sent off Villa striker Christian Benteke following a 66th-minute scuffle which involved several players from both sides.
Spurs eventually won the game as goals from Nacer Chadli and Harry Kane in the last 10 minutes earned them a 2-1 victory over their 10-man hosts.
Benteke clashed with Spurs midfielders Erik Lamela and Ryan Mason in front of the dugouts before shoving the latter in the face, which sparked a melee between the two teams.
It has prompted a reaction from the FA, with a statement reading: "Both Aston Villa and Tottenham Hotspur have been charged by the FA following their game on Sunday 2 November 2014.
"It is alleged that in or around the 66th minute of the game, the clubs failed to ensure their players conducted themselves in an orderly fashion.
"Both clubs have until 6pm on 10 November 2014 to respond to the charge."
Source : PA
Source: PA