Declaring himself "somewhat agitated", Kelly said Djokovic had provided evidence from "a professor and an eminently qualified physician" about a medical exemption.
"What more could this man have done?" the judge demanded.
Proceedings were repeatedly delayed as the court's online system crashed under a surge of worldwide interest.
Anti-vaccine activists shared a link to the livestream and defied a court order by broadcasting proceedings live on YouTube.
Eventually, the judge pressed ahead without a public livestream, and Djokovic's team of top-flight lawyers made his case.
During overnight questioning at the airport, they said Djokovic was "utterly confused" about the situation.
The Australian Open gets under way in just seven days, and the nine-time defending champion's participation now entirely depends on whether Judge Kelly believes the government acted improperly in revoking his visa.
Most foreigners are still banned from travel to Australia, and those granted entry must be fully vaccinated or have an exemption like "acute" illness.
Another tennis player -- Czech doubles specialist Renata Voracova -- has also had her visa cancelled after obtaining a medical exemption.
She flew out of Australia on Saturday after being held in the same Melbourne centre as Djokovic.
Government lawyers reject Djokovic's case, and are expected to argue later Monday that he fails to meet the medical criteria as his recent infection was not "acute".
They will seek to have his appeal dismissed with costs, according to a 13-page court submission, paving the way for his deportation as soon as Monday evening.
Despite Djokovic's claim of a positive test on December 16, he attended a gathering that day for the Serbian national postal service launching a stamp series in his honour.
Pictures shared by the Belgrade tennis federation also showed him at a young players' event in the city on 17 December.
It reported that he had handed over cups and prizes to players. No one was wearing a mask.
Djokovic is in detention at the former Park Hotel, a five-storey facility that holds about 32 migrants trapped in Australia's hardline immigration system -- some for years on end.
For days, demonstrators and counter-demonstrators have gathered outside the facility. Nobody is usually allowed in or out except staff.
But Djokovic won permission from the court to watch Monday's proceedings from another, undisclosed location before returning to the detention centre when hearings conclude.
An early plea to be moved to a facility where he can train for the Australian Open has fallen on deaf ears, his lawyers said.
The detention centre gained notoriety last year when a fire forced migrants to be evacuated, and maggots were allegedly found in the food.
Hours before the hearing, a pro-refugee banner was unfurled from the roof and police removed a small number of protestors from the scene.
Meanwhile at a rally in Belgrade, Djokovic's mother Dijana claimed her son was staying "in not human conditions".
"They detained him and even don't give him breakfast, he has only lunch and dinner," she said, quoted by local media.
"He does not have a normal window, he stares at a wall."
Prime minister Ana Brnabic said this weekend that Serbia was fully behind the player and that she had held "constructive talks" with her Australian counterpart.
"We managed that he gets gluten-free food, exercise equipment, a laptop," she told Serbia's Pink television.
As other players now enter the final intense phase of preparations for the tournament, Djokovic faces huge pressure to be ready in time.
Djokovic's lawyers have told the court that Tennis Australia needs an answer by Tuesday. The draw for the event is scheduled for Thursday.
But Judge Kelly has warned justice will move at its own pace through all necessary appeals.
"The tail won't be wagging the dog here," he said.
Tennis Australia chief Craig Tiley on Monday defended his organisation from criticism that it failed to warn players that a previous infection did not qualify them for entry without a Covid-19 vaccination.
Tiley said he had asked the government to review medical exemptions before the players arrived, but "they declined".
"We asked if they could please assess our decisions. We said we're going to need some help to make sure we're doing the right thing. We'd be in a different situation today," he told The Sydney Morning Herald.
Meanwhile, much of Australia is tightening restrictions to battle an Omicron-fuelled wave of infections.
The country is now approaching 100,000 cases a day, having been virus-free for much of the pandemic.