Liverpool will make their first FA Cup final appearance since 2012 on May 14 when they face Chelsea or Crystal Palace, who meet in Sunday's semi-final.
The Reds, into the FA Cup final for the 15th time, haven't won the competition since 2006, while Klopp has yet to lead them to a domestic knockout trophy.
Trailing Premier League leaders City by just one point with seven games left, Liverpool have already won the League Cup and have also reached the Champions League semi-finals, where they will play Villarreal.
Despite Klopp's protestations that the quadruple is an impossible dream, this was another step towards football immortality.
For City, fatigued physically and mentally, it was a game too far as Liverpool handed them a first defeat in 11 games.
Both league meetings this season had ended in 2-2 draws, including a pulsating clash at the Etihad Stadium last weekend.
But, battered and bruised after surviving a gruelling Champions League quarter-final at Atletico Madrid on Wednesday, City were unable to match Liverpool's intensity as they surrendered possession with uncharacteristic mistakes.
Pep Guardiola claimed City were "exhausted", giving him "no choice" but to make changes, with Kevin De Bruyne on the bench alongside Ederson, Ruben Dias and Rodri, while Kyle Walker was sidelined with an injury.
In contrast, Klopp recalled Mane, Mohamed Salah, Virgil van Dijk and Trent Alexander-Arnold after resting his key players for Wednesday's Champions League quarter-final second leg against Benfica.
Guardiola will believe his selection was the right move if City can pip Liverpool in the title race and reach a second successive Champions League final.
It was a bad day from start to finish for City, who issued an apology after their fans forced referee Michael Oliver to cut short a pre-match tribute to the 97 Liverpool fans who died at Hillsborough 33 years ago by chanting through the minute's silence.
Liverpool exacted revenge on the pitch, tearing into City and taking the lead in the ninth minute.
Konate had scored with headers in both legs of the Champions League quarter-final against Benfica, but City surprisingly opted not to give the French defender special attention.
Andrew Robertson's outswinging corner looped into City's six-yard box, where Konate towered above Nathan Ake to thump his header past Steffen.
Liverpool were tormenting City with pace and movement, leaving their rivals so rattled that they gift-wrapped the second goal in the 17th minute.
Ake's back-pass should have been easy for Steffen, but the American took too long to decide whether to pass or boot it clear, allowing Mane to slide in and deflect the ball into the net from virtually on the goal-line.
Liverpool, pressing relentlessly, scored again on the stroke of half-time.
Thiago Alcantara clipped a deft pass out to Mane and he volleyed past the slow-to-react Steffen from the edge of the area.
City pulled one back in the 47th minute when Gabriel Jesus twisted away from Fabinho and squared for Grealish to fire home from 10 yards.
Suddenly threatening a stunning fightback, City were denied by Alisson's two superb saves from Jesus.
Silva converted Riyad Mahrez's cross in stoppage-time, but the final whistle triggered a wall of sound from Liverpool fans dreaming of quadruple glory.