At least three people drowned and nearly 20 others were missing off the Bangladesh coast after a boat carrying Rohingya refugees sank in rough weather on Tuesday, authorities said.

Dire conditions in Bangladesh's refugee camps prompt hundreds each year to pay smugglers for passage out of the country.

The fishing trawler left at dawn and was bound for Malaysia before it ran into trouble in the Bay of Bengal, with two coast guard search boats scrambling to rescue survivors.

Police said at least three bodies had washed ashore at the coastal town of Halbunia.

"We recovered the bodies. They are three young Rohingya women aged between 18 and 20," police inspector Nur Mohammad told Agence France-Presse (AFP).

Coast guard station commander Ashiq Ahmed said the vessel was believed to be carrying around 65 people after stopping at several coastal towns before embarking.

"We have rescued 45 people including 41 Rohingya refugees and four Bangladeshis," he told AFP.

"Nearly 20 people are still missing. The rescue operation is still going," he added.

The boat sank off the coast of Cox's Bazar district, near the site of sprawling camps that are home to around a million Rohingya refugees.

Most arrived in Bangladesh five years ago after a military crackdown in neighboring Myanmar that is now the subject of a genocide investigation at the U.N.'s top court.