After play resumed, Labuschagne fell for 24. Marcus Stoinis scored 44 off 31 balls. Smith made 53 before he was clean bowled by Dunith Wellalage.
Alex Carey scored a quick 21 off 22 balls and Pat Cummins, playing his first ODI after 18 months with an injury, fell for a duck.
But Maxwell saw his team home, ending the match with two consecutive sixes.
"I tried to be there as long as I could. I thought it was a good opportunity to get the run rate down against the young spinner," Maxwell said.
Sri Lankan skipper Dusan Shanaka bemoaned too many short balls and full tosses.
"300 is a winning total on this ground, but our bowling cost us the game," Shanaka said.
Earlier in the day, the hosts began their innings with solid batting performances by openers Danushka Gunathilaka and Pathum Nissanka, who scored half centuries in their opening stand of 115.
Both men hammered an Australian fast-bowling attack plagued by injuries before the visitors caught Gunathilaka short of his crease with a direct hit by Stoinis.
Nissanka top-edged a reverse sweep to Ashton Agar to hand Finch a simple catch two overs later.
Mendis anchored the innings with an unbeaten 86 off 87 balls to take the hosts to 300.
Shanaka, who stunned the visitors in the final Twenty20 international on Saturday, could only score six.
A quick-fire 37 off 19 balls by Hasaranga helped Sri Lanka post a challenging total.
Spinners Labuschagne and Agar took two wickets each.
Fast bowler Cummins was wicketless and conceded 48 runs in eight overs.
Australia have suffered setbacks with lead bowlers rested from the tour including Kane Richardson, who returned home on Sunday after injuring a hamstring in the last T20 international.
The second ODI on Thursday is at the same venue.