Malky Mackay's side looked in control as they surged into a two-goal lead through goals from John Eustace and Will Buckley but they crumbled after the break.
Sven-Goran Eriksson has galvanised City since he took charge and Paul Gallagher hauled them level with a brace.
But Graham still found time to find an 85th-minute winner and seal a remarkable game.
Centre-half Martin Taylor scuffed a volley wide from an early corner as Watford started well but it was from a set-piece that the home side did go ahead in the 15th minute.
Jordon Mutch delivered the cross from the right and Eustace got across the face of his marker to steer a crisp volley into the far corner.
Mutch and Leicester midfielder Andy King then traded long-range efforts which fizzed wide before Buckley doubled the home side's lead in comical fashion in the 22nd minute.
Defender Jack Hobbs was dispossessed on the halfway line but there still looked little danger as the ball was punted forward.
However Foxes goalkeeper Chris Weale made a complete hash of cutting out the pass, misjudging the flight of the ball to present Buckley with the simple opportunity of tapping into an open goal.
Eriksson's side had endured a dreadful spell but rallied just before the break.
First Richie Wellens surged into the area to drill in a low shot which Scott Loach did well to palm clear, before Roman Bednar shot wide at the end of a strong run.
Leicester were now on top and deservedly pulled a goal back five minutes after the restart.
Darius Vassell ran into the area before playing in Kyle Naughton on the overlap. His cross struck Andrew Taylor and referee Kevin Wright had no hesitation in awarding the penalty.
Gallagher took the kick and smashed it into the roof of the net.
T
he former Blackburn man was denied an equaliser minutes later when Loach pulled off a reflex save with his feet but was rewarded with his second in the 69th minute.
Gallagher was fouled on the edge of the area by Lloyd Doyley and made no mistake with the free-kick, curling a fine effort over the wall and past Loach to find the bottom corner.
The game looked certain to end all square but Graham struck in the last five minutes to claim the win.
Mutch - whose set-piece delivery had been immaculate all afternoon - floated in another great cross and Graham flicked a header across Weale and into the far corner.
Source: DSG
Source: DSG