Mustafizur gave away seven extras among 15 runs conceded but New Zealand ended on 137 for five.
Bangladesh spinners again dominated on the slow wicket with Shakib Al Hasan and Mahedi Hasan finishing on two for 12 and two for 29 respectively.
Shakib ended a 43 run second wicket stand between Latham and Will Young to get within one wicket of joining Sri Lanka's Lasith Malinga as the leading wicket taker in Twenty20 internationals.
The Bangladesh all-rounder now has 106 wickets in the shortest format.
Latham, who survived a stumping on 54, put Bangladesh under pressure with his solid batting.
New Zealand, hammered by seven wickets in the opening game, needed six off the last ball for victory but Latham managed only a single.
"It was a great game to take it down to the last over," said the New Zealand captain.
"I thought we learned from the last game and to take it down to the last over with a chance of winning was outstanding."
Mahmudullah said he had confidence in Mustafizur to handle the final over.
"Mustafizur held his nerve well and we went really close," he said. "It was unusual, but I had quite a bit of confidence in Mustafizur."
Naim and Liton Das made 59 runs in Bangladesh's opening stand after Mahmudullah won the toss and elected to bat.
Liton, who was dropped before he had scored by Colin de Grandhomme off Cole McConchie, went on the punish New Zealand by making 33 off 29 balls.
Rachin Ravindra (3-22) dismissed Liton in the 10th over and then had Mushfiqur Rahim out for a duck in the next ball.
Mahmudullah's 37 from 32 balls guided his side to a challenging score.
The third match in the series is on Sunday.