The government has spent about $350 million building shelters on the 53 square kilometre (20 square mile) island which was formed by tidal silt deposits about 20 years ago.
On top of the inhospitable weather, the island is 60 kilometres (37 miles) from the Bangladesh mainland and some Rohingya groups say people were forced to go there.
About 850,000 Rohingya are packed into camps along the Bangladesh-Myanmar border. Most fled a Myanmar military clampdown in 2017 that the UN says could be genocide.
Bangladesh has been praised for taking in the refugees who poured across the border but has had little success finding them permanent homes.
UN confirms deal
While a Bangladesh security intelligence agency is responsible for relocating the refugees, the government denies that any coercion has been used.
Still, several hundred Rohingya have fled Bhashan Char only to be detained on other islands or in coastal villages. Dozens died after a fishing boat packed with Rohingya trying to flee the island sank in August.
The UN had expressed doubts about the relocations. But Bangladesh and UN officials said a deal has been agreed to give the UN a role in providing humanitarian assistance and monitoring conditions on the island.
"We can confirm that the UN will sign a memorandum of understanding with the government to protect Rohingya refugees in Bhashan Char on Saturday," a spokesperson for UNHCR, the UN's refugee agency, told AFP.
Refugee commissioner Hayat said the UN would play a bigger role on the island than they do now in the mainland refugee camps, where they already have extensive operations.
"Bhashan Char residents will receive similar treatment, humanitarian response-wise, to what is now provided to people in the Cox's Bazar camps," he said.
He added that there would also be greater opportunities for Rohingya to work on the island than there are now in the rigidly policed camps.
But Amnesty International raised new concerns over the relocations and highlighted the attempts made by refugees on the island to get away again.
"Some refugees have drowned at sea and many are either being arrested, detained, or forcibly returned to the island," Saad Hammadi, Amnesty's South Asia campaigner, told AFP.
He said Bangladesh, the UN and donor countries should "develop a rights-respecting policy and ensure the participation of Rohingya refugees in decisions that affect their lives."