"There was a fantastic feeling in the dressing room. We have been through a tough period and this competition means a lot for us," said Potter.
"We wanted to progress and get into the last eight and it sets us up for the next few weeks."
Dortmund arrived in England on the back of a 10-game winning run in all competitions.
But the visitors posed barely any attacking threat in the first half bar a Marco Reus free-kick that Kepa Arrizabalaga did brilliantly to palm to safety.
"We have to say, over the course of two games, Chelsea deserve to go through," said Dortmund boss Edin Terzic.
"Both legs were very tight games. In this type of games inches and moments decide if you are going to the next round or not."
Chances came and went frequently at the other end. Havertz smashed a volley off the inside of the post and then finally found the net via the underside of the bar only for the goal to be ruled out for an offside against Sterling earlier in the move.
The worst miss was to come as Kalidou Koulibaly miscued with an open goal from Ben Chilwell's free-kick and Felix's follow-up effort was blocked on the line.
Potter was uncharacteristically animated on the touchline as he constantly whipped the crowd up to keep pushing his side forward.
"There is a lot of emotion in the end," added Potter. "It was tense in the end but the boys played fantastic.
"I am delighted for them to get the win and great for everyone here."
The home fans finally had something to celebrate three minutes before half-time.
Sterling comes good
Sterling's first attempt was another one to forget as he swiped and missed Chilwell's low cross, but the England international composed himself to dribble past Reus and fire high into the net.
"I had a little mis-kick with the first touch but it landed perfectly and allowed us to build on that for our performance," said Sterling.
That was just Chelsea's third goal in the last eight games.
Havertz's penalty early in the second half meant Potter's men scored more than once for the first time since December 27, but the German international needed two spot-kicks to make his mark.
Marius Wolf was penalised for handling Chilwell's cross after a VAR review.
Havertz's first penalty came back off the post, but VAR came to Chelsea's rescue once more as Dortmund were further punished for encroaching.
There was no second reprieve as Havertz coolly slotted the ball the same way, this time finding the corner of the net.
Chelsea's spending spree may have ruled them out of the running for Jude Bellingham with the Dortmund midfielder expected to have his choice of Europe's top clubs this summer.
But on his return to England, Bellingham missed Dortmund's best chance to take the tie to extra-time when he side-footed wide from close range.