The Villa boss, under pressure after a run of seven defeats in nine games, saw his side ease through 3-1 despite some nervous moments at Bramall Lane.
Kyle Walker, making his Villa debut against his first club, scored his first career goal before Marc Albrighton's volley make it 2-0 to the visitors at half-time.
The second half was a different affair however and Jamie Ward brought the Blades back into it from the spot and they had several chances after Ashley Young's dismissal before Stiliyan Petrov's late strike wrapped up the result.
Houllier said: "It was a typical cup-tie where at times we could play football, at times where we had to withstand the pressure and defend.
"Probably the game could have been put to bed a bit earlier but the third goal was a good relief at the end.
"All the players were terrific - it was not easy. We haven't had the results we wanted in the league and it's never easy away at Sheffield United."
Source: PA
Source: PA