Ahead of Monday's Barclays Premier League home clash with Arsenal, Villa are on a poor run of form, having won only one and lost five of their last seven games in all competitions.
The midlands outfit's record at their own ground for the entire campaign to date has been disappointing, with it showing just two victories and six defeats from 10 top-flight fixtures
And last weekend they suffered a 2-1 FA Cup third-round reverse at Villa Park against Sheffield United - a club that have been struggling this term towards the lower end of Sky Bet League One.
Lambert - under whom Villa finished 15th in the Premier League last season, his first in charge of the club - said after a 1-1 home draw with Swansea in December that although he has a good relationship with his chairman Randy Lerner, he always works with "that fear" of losing his job.
Returning to the theme this week, Lambert said: "I never stop worrying. I'm not immune from it.
"The relationship between me and Randy is what it is. You are never immune from anything, and I would never view it that way.
"You just keep going - there is no secret formula to it. You just keep going and don't lose focus on what you are trying to do. I will never waver from that."
Lambert has also been keen to stress how bad a week he has had off the back of the Sheffield United result.
Before the game, he had caused a stir by saying that most fellow top-flight managers would probably prefer to do without the FA Cup.
After the match - for which he fielded a first XI showing only four changes - he claimed that comment had been taken out of context.
And asked ahead of the Arsenal contest how he had been feeling in the days since the cup tie, Lambert said: "The same way I do after every other game (Villa have lost).
"I don't sleep great, I don't eat great. The usual stuff.
"It is just the life of a football manager. It is a great job, but you don't feel great (after defeats).
"And contrary to other people's reports, I never degraded the competition at all.
"I tried to win the game, I put the best team out I possibly could and we got knocked out. Good luck to Sheffield United. We weren't good enough."
There were boos from Villa fans at the final whistle in the Sheffield United game and while Lambert has expressed his understanding for the supporters' frustration, he has also called for them to "stick with it" and do all they can to help the team on Monday night.
"I don't have any problem with the crowd," Lambert said.
"If the team is not performing, then you expect people to vent their frustration - that is normal.
"But we are a young side.
"Last year, we played Newcastle at home and we were getting beaten 2-0 in the first half.
"Then we scored, and for about 40 minutes, it didn't matter what we did, that crowd was right with us. They were brilliant that time.
"Because you are in your second year, it might be a different view, but if they can stick with it and see us through the hard times when we need them... because we certainly need them.
"They have been excellent for us overall, and we expect if we are not performing that certain things will happen.
"But we're hoping on Monday that they will get right behind us."
Villa will assess the condition of defenders Ron Vlaar and Nathan Baker ahead of the Arsenal match.
Both club captain Vlaar and Baker sat out the Sheffield United game with calf and hamstring problems respectively.
Forward Gabriel Agbonlahor, who also missed the cup tie, and goalkeeper Brad Guzan are fit despite having had niggles, but the contest is too soon for midfielder Chris Herd and defender Joe Bennett to make their comebacks after injury.
Source : PA
Source: PA