Bangladesh has been praised for taking in the refugees but rights groups have criticised restrictions placed on the refugees, most recently a drive to knock down makeshift shops that serve their communities.
The country's deputy refugee commissioner Shamsud Douza told AFP that "more than 3,000 illegal shops" had been demolished.
"The number of Rohingya is increasing. And they need shelters. We are already building sheds on the premises," he said, adding that relief groups were ensuring the refugees were still getting daily necessities.
But Khin Maung, a Rohingya community leader and rights activist, told AFP the demolitions had already hurt tens of thousands of refugees in the camps.
"Rohingya families are large and the amount of food ration given to them is decreasing. Many families used to rely on the income from the shops," he said.
Salim Ullah, whose grocery shop was torn down, said he would now struggle to feed his family of eight.
"That shop was my last hope. How do I run a family now? There is no way out except to die. I am helpless," he told AFP.
Saad Hammadi of Amnesty International said the move would leave refugees vulnerable to exploitation and worsen conditions in the camps.
"Demolition of shops and closure of community-led schools... aggravate tension and frustration," he said.
Hammadi urged authorities to "protect the rights and dignity of the Rohingya refugees by involving them in the decisions including their right to earn a living".