But their England record seventh-wicket stand of 241 was the cornerstone of an eventual total of 360 -- a first innings lead of 31.
New Zealand opener Will Young then fell cheaply before a second-wicket stand of 97 steadied the tourists.
Captain Kane Williamson cover-drove opposing skipper Ben Stokes's first delivery this Test for four.
Tom Latham forced the all-rounder away for his sixth boundary before a single took him to a 70-ball fifty in a series where his previous best score had been a meagre 26.
Latham, dropped by Joe Root in the slips on 72, fell to the first ball after tea.
The towering Overton, in for injured England great James Anderson, had him caught behind for 76 by stand-in wicketkeeper Bairstow, with first-choice gloveman Ben Foakes sidelined by a stiff back on Saturday.
Devon Conway turned Root's first ball after a rain break off his pads, only for Ollie Pope to hold a superb one-handed catch at short leg.
Conway's exit sparked a mini collapse that saw three wickets lost for just nine runs, with New Zealand's 152-2 quickly transformed into 161-5.
Williamson casually flicked at Potts outside off stump to give Bairstow a simple catch -- the third time in four innings this series he had fallen to the Durham seamer.
The star batter threw his head back in anguish following the end of a well-made 48 featuring eight boundaries.
"I wouldn't say Williamson is sitting in my pocket!," the 23-year-old Potts told Sky Sports.
"It's great to get him out, it's a great Test scalp and to have him three times is really enjoyable."
Left-arm spinner Jack Leach, fresh from his 5-100 in the first innings, then deceived Henry Nicholls into offering a low return catch.
Daryl Mitchell, who has already hit three hundreds this series, was four not out when further rain ended the day's play, with Tom Blundell unbeaten on five.
Overton had resumed on 89 not out at the start of play on Saturday, eyeing a debut century.
But the 28-year-old never regained the fluency he had on Friday and it was no shock when Trent Boult, who had removed all of England's top three with a superb display of new-ball bowling, had Overton caught in the slips for 97.
England's new leadership duo of Stokes and coach Brendon McCullum had urged a team that had won just one out of 17 Tests prior to this series to play with freedom.
Stuart Broad embraced the new approach by hitting Boult for 14 runs in three balls courtesy of two fours and a six during a quickfire 42.
Bairstow, one ball after Broad's exit, holed out off spinner Michael Bracewell to end a brilliant 157-ball innings including 23 fours.