The bus had set out from the city of Cali with 25 passengers, and traveled 270 kilometers (170 miles) before being hit by the landslide as it passed through the Andes mountain region, civil defense officials said.
Colombian media reported that a child had survived and was pulled from the arms of its mother, who did not make it.
One survivor said the bus driver had at first managed to dodge the worst of the landslide.
"Part of it was coming down and the bus was a little bit back from that. The bus driver was backing up when it all came crashing down," Andres Ibarguen told radio station Lloro Stereo.
The rainy season that began in August is Colombia's worst in 40 years, according to the government, causing accidents that have left more than 270 people dead.
The country has declared a national disaster over the rains linked to the exceptionally long La Nina weather phenomenon, which cools surface temperatures and is currently causing drought and flooding around the globe.
Today, the landslide "puts this town in mourning, tomorrow it could be in another area, because we really have many unstable areas in the country, and the rainy season has not ended," said Javier Pava of the UNGRD.
The UN's World Meteorological Organization said last week the La Nina conditions could last until February or March 2023.
In Colombia, the phenomenon has also caused crop damage, compromising food supplies and leading to soaring prices.
In July, three children were killed in northwestern Colombia when a landslide buried a rural school. In February, 14 people died in a mudslide triggered by heavy rains in central-western Risaralda province.