"The better team won. We didn't play good. There are many games you can start not good and overcome but we didn't do it," he said.
"When you are not prepared to play this game you arrive one inch late and don't score a goal. When you are prepared you score the goal.
"Today was a bad night, the opponent was better so we have to congratulate them. We didn't deserve to win."
City's surprise exit continued a worrying trend of lacklustre performances this season in games Guardiola would expect them to win, a failing that has left the champions trailing five points behind leaders Arsenal in the Premier League.
Guardiola will demand an immediate response in Saturday's crucial Premier League derby against Manchester United.
He made four changes from Sunday's 4-0 FA Cup third round rout of Chelsea, with Kalvin Phillips handed his first City start since his £42 million ($51 million) move from Leeds last year.
Phillips was labelled overweight by Guardiola after he returned from England duty at the World Cup and the midfielder hardly made the Spaniard eat his words with a largely anonymous display that ended with his substitution after 63 minutes.
Phillips' flop was in keeping with City's unusually limp display, with their lack of poise and purpose visible right from the start.
Southampton went ahead in the 23rd minute when Lyanco cut out a loose pass from Sergio Gomez and delivered a low cross that France Under-21 international Mara swept home for his first goal for the club.
Five minutes later, Mali international Djenepo was allowed to advance unchecked before curling a superb 30-yard strike over Stefan Ortega, who was caught too far off his line.
Guardiola sent on Kevin De Bruyne at the interval and Erling Haaland after 56 minutes, but his lethargic team finished without a shot on target.
Southampton have never won the League Cup, losing the 1979 and 2017 finals, and they will face Newcastle in a two-legged semi-final later in January.
Saints boss Nathan Jones will hope this surprise success can be the springboard to silverware and, more importantly, Premier League survival.
"We've gone through a lot recently, people questioning a lot of things. That goes a little way to justifying why we're here and what team we're trying to create," Jones said.
Nottingham Forest earned a last four tie against Manchester United after beating Wolves 4-3 on penalties following a 1-1 draw at the City Ground.
After last weekend's embarrassing FA Cup loss against second tier Blackpool, Forest are back on the Wembley trail as they reached the League Cup semi-finals for the first time since 1992.
They took the lead through a close-range finish from former Wolves centre-back Willy Boly in the 18th minute.
Raul Jimenez grabbed Wolves' 64th minute equaliser, but Dean Henderson was Forest's shoot-out star as he saved penalties from Joe Hodge and Ruben Neves.