Officials and supporters of the bid say that era was when the mines were the world's most productive and mining was done by hand.
But it also stops short of the period when conscripted Koreans toiled in conditions that even some backers of the UNESCO bid call "extremely harsh".
The World Heritage effort has been years in the making, inspired in part by the successful recognition of a silver mine in western Japan's Shimane region.
Ryo Usami, from Sado city's World Heritage Promotion section, said locals hoped the recognition would highlight the mine's contributions to the unique culture and history of the island.
"Many people migrated to Sado to mine gold and silver... They came from all over Japan and brought their local cultures," Usami told AFP.
"Sado's history is basically the history of these gold mines, and its culture formed in part thanks to the mining operations. That's what Sado city wants to preserve."
'Discrimination did exist'
Production at the sites petered out by the 1960s, when mine operator Mitsubishi Materials began accepting tourists.
In the 1970s, animatronic robots were installed in some mining tunnels to give a sense of what life there was like.
The eerie, rundown figures remain, their heads swivelling side-to-side and arms mechanically swinging pickaxes up and down in a forlorn fashion.
Groups of domestic tourists file through the frigid tunnels and read panels that explain the history of Sado's mining industry.
The panels note that Edo-era miners were often homeless or unregistered people who were captured and forced to work, and that child labour was used at times.
But there is little to testify that an estimated 1,500 Koreans worked at the sites during World War II.
Their status is disputed, with some arguing around two-thirds signed contracts voluntarily, while the remainder were conscripted during war-time mobilisation.
"The working conditions were extremely harsh, nevertheless the pay was very high, that's why lots of people, including many Japanese, applied," said Koichiro Matsuura, a former UNESCO director-general who is backing Sado's bid.
Others argue the recruitment conditions effectively amounted to forced labour, and that Korean workers faced significantly harsher conditions than their Japanese counterparts.
"Discrimination did exist," said Toyomi Asano, a professor of history of Japanese politics at Tokyo's Waseda University.
"Their working conditions were very bad and dangerous. The most dangerous jobs were allocated to them."
'A part of our history'
Wartime issues like forced labour have soured ties between Japan and South Korea, and Seoul has formed a task force to push back against the UNESCO bid.
After the bid was announced, the government summoned Tokyo's ambassador and issued a statement saying it "strongly regrets" the nomination and "sternly urges Japan to stop its attempt".
The issue of forced labour affects other Japanese heritage sites, including the "Sites of the Meiji Industrial Revolution" inscribed in 2015.
UNESCO last year demanded an information centre for the sites properly explain that a "large number of Koreans and others (were) brought against their will and forced to work under harsh conditions".
Matsuura believes Japan must "avoid making the same mistake" at Sado.
"We must say in a more concrete and more honest manner how the Korean workers lived and worked at the Sado gold mines."
It's a view shared by some visitors, including 79-year-old Hideji Yamagami.
"Of course they should (explain), I didn't know about it at all," he told AFP after a trip through the Aikawa site.
"I thought it was Japanese who had done all the hard work."
Asano hopes UNESCO will insist the full history of Sado's mines is on display if the site gets World Heritage status, and believes Japan "should not fear" recognising a part of its history.
"Every nation has its dark history, those nations who are completely exempt do not exist."