Darren Bent, starting a game for Villa for the first time since the final day of the 2012/13 season, was chief culprit as he missed in each half with only the goalkeeper to beat.
Joe Cole's perfectly-weighted pass behind the Orient defence sent Bent through down the right-hand side of the penalty area but his shot across Gary Woods flicked off the goalkeeper's legs and wide of the far post.
Then moments before Romain Vincelot's 87th-minute winner, the striker fired wide from a similar position after latching on to a superb 35-yard pass from centre-half Philippe Senderos.
''We had enough chances but never took them,'' said Villa boss Lambert
''When you don't take chances, at any level of football, then you get punished
You don't have a divine right to beat any team.
''Darren is deadly in those situations and on another night they would have gone in
But like I said, if you don't take chances things like that happen.''
Villa had a warning 20 minutes before Vincelot's goal when Mathieu Baudry's downward header looked destined for the top corner but Shay Given produced a superb save to turn the ball on to the crossbar.
''We got away with that one,'' said Lambert
''It was a great save from Shay.
''It's disappointing because in the first two Premier League games we've been absolutely solid - we haven't conceded a goal
Shay's save was a warning and we never took that warning, but it should never should got that far we should have finished it off.''
Vincelot's header saw Orient join Bradford in humiliating Villa in this competition, with the Bantams famously defeating the Premier League team over a two-legged semi-final in 2013.
And earlier this year at Villa Park, Sheffield United knocked Lambert's men out of the FA Cup.
Asked if the disappointment resembled those defeats, the Villa boss replied: ''You don't want to lose any football game.
''When you lose any football game, it's not a great feeling.
''I'm really disappointed because I though the team I picked was enough to win the game.
''I knew this would be a hard game and I picked a team I thought could win it, and we should have done as we had enough chances.''
Orient manager Russell Slade felt his game plan worked well, although he could not take the credit for what looked like an inspired match-winning substitution.
Vincelot headed home Bradley Pritchard's left-wing cross at the far post just four minutes after coming on as a replacement for Marvin Bartley.
Slade admitted the switch was necessity rather than a tactical masterstroke.
He said: ''It was a fortunate move
We wanted to keep Marvin on for the full 90 minutes really as he needs minutes on the pitch, but he had to come off.
''If we were going to make a tactical change at that stage I would have probably brought Lloyd James off, but there you go.
''But Romain has that ability in the air, he has proven that already this season
He seems to be getting forward more this season, which is good.
''My over-riding feeling is I'm pleased
I'm pleased we stuck to the game plan
It was important to stay in the game, especially early on
It allowed us time to settle and then we got on the ball reasonably well.
''We grew into game and I'm delighted we didn't go into extra-time for second time this season, having done so in the previous round.
''Obviously getting the goal when we did makes it difficult for them to respond.''
Orient have been drawn at home to League One rivals Sheffield United in the next round.
Slade had two spells as caretaker manager of the Blades in 1998 and 1999 and he is looking forward to facing his old club.
He said: ''First of all it's pleasing to be at home
Sheffield United are a great club
I used to be there for three and a half years so I will look forward to that, although it will be a tough game.''
Source : PA
Source: PA