Mahrez curled in a brilliant free-kick from the edge of the box to break the deadlock after a tight first 45 minutes.
Alvarez then made the most of another chance to start ahead of Erling Haaland by tapping in his third goal in as many games at the back post after Mahrez had been denied.
"Julian is an exceptional player," said Guardiola.
"Of course he has to fight against maybe the best striker in the world, Erling, but Julian is showing the many things he can do."
Arsenal are keeping City off the top of the Premier League, but will not be adding their first League Cup since 1993 after Brighton continued their impressive turnaround under Roberto De Zerbi.
Eddie Nketiah's fine finish put the hosts ahead at the Emirates but a slip from goalkeeper Karl Hein on his Arsenal debut gave Danny Welbeck the chance to level from the penalty spot against his old club.
Kaoru Mitoma's powerful finish turned the tie around before Tariq Lamptey slotted home 19 minutes from time.
"We are out, it is already done, and now we have to concentrate on the Premier League, which is the priority," said Arsenal boss Mikel Arteta.
England manager Gareth Southgate may have been dismayed by the sight of Kane starting at the City Ground the night before he names his World Cup squad.
The England captain did finally get a rest for the final half-hour, but only as Antonio Conte seemingly threw in the towel with his side 2-0 down.
Renan Lodi cut inside on to his weaker right foot to fire in a brilliant opener early in the second half before Jesse Lingard headed in his first goal for Forest.
Kelleher Liverpool's hero
Caoimhin Kelleher was Liverpool's hero as he was for much of last season's competition as the Reds' stand-in goalkeeper saved three Derby penalties.
Jurgen Klopp made 11 changes and it showed as a youthful side struggled to make their possession count.
Roberto Firmino and Darwin Nunez failed to find a winner after being introduced with 25 minutes to go and Firmino rounded off a bad week after being left out of the Brazil World Cup squad by blasting his penalty over the bar in the shootout.
But Kelleher, who scored the winning penalty in last year's final, made sure Liverpool progressed.
Newcastle also needed penalties to get past Crystal Palace after a 0-0 draw at St James' Park.
Nick Pope will be in Southgate's squad for Qatar and showed he can be called on if needed in a shootout by saving three of Palace's five spot-kicks.
Neither goalkeeper managed to save any of the 20 penalties at the London Stadium as Blackburn dumped out West Ham.
The tie was level at 2-2 after 90 minutes as Ben Brereton Diaz equalised two minutes from time for Rovers.
Blackburn converted their 10 penalties and were through when West Ham captain for the night Angelo Ogbonna hit the bar.
Boubacar Traore's strike five minutes from time gave Wolves a 1-0 win over Leeds.
Meanwhile, Southampton edged through on penalties over Sheffield Wednesday after a 1-1 draw in front of Nathan Jones, who is expected to be named the Saints' new manager on Thursday.