Bangladesh has agreed to complete the process of returning Rohingya Muslims to Myanmar within two years after repatriation begins, the south Asian nation said Tuesday.

The agreement, finalized in Myanmar's capital this week, says that the process would be "completed preferably within 2 (two) years from the commencement of repatriation," a statement from the Bangladesh foreign ministry said.

The ministry did not say, however, when the process would begin. But it added the return effort envisages "considering the family as a unit," with Myanmar to provide temporary shelter for those returning before rebuilding houses for them.

According to the deal, Bangladesh will "establish five transit camps from which returnees would be received initially in two reception centers on Myanmar side."

Myanmar will prioritize resettling people living in close proximity to the "zero line."

The international community and rights groups have stressed that the Myanmar government must guarantee the "safe and voluntary" return of the refugees, and urged that international organizations be allowed to participate. However, the U.N. refugee agency said yesterday that it has not been invited to take part or given full access to the areas where refugees are to return.

The U.N. called yesterday for Rohingya refugees who have fled to Bangladesh to be informed about conditions in Myanmar's northern Rakhine state before they are allowed to return voluntarily and in safety.

"Major challenges have to be overcome," Andrej Mahecic, spokesman of the U.N. High Commissioner for Refugees (UNHCR), told a Geneva news briefing. "These include ensuring they are told about the situation in their areas of origin [...] and are consulted on their wishes, that their safety is ensured," as reported by Reuters.

Left out of the talks were the fears and concerns of the refugees themselves, "as if they are an inert mass of people who will go where and when they are told," Phil Robertson, deputy Asia director at Human Rights Watch, told Reuters in an email.

"Where are considerations for protection of the Rohingya from Myanmar security forces who months ago were raping and killing them? How come the discussions ignore the deprivation of rights of people held in indefinite detention, which is what these so-called 'temporary' accommodations may become?" Robertson asked.

Myanmar agreed to repatriate Rohingya refugees from Bangladesh in November, but the U.N. and rights group fear for their safety in the country, where anti-Muslim sentiment is surging.

More than 650,000 Rohingyas were forced to flee the Buddhist country since Aug. 25 due to "clearance operations" by the army.

The U.N. the military's actions as "ethnic cleansing," and rights groups have accused security forces of atrocities, including mass rape, arson and killings. The U.N. rights chief said Myanmar army could commit acts of genocide.

Described by the U.N. as the most persecuted minority, Rohingyas have been the target of communal violence and vicious anti-Muslim sentiment in mainly Buddhist Myanmar for years. They have also been systematically oppressed by the government, which stripped the minority of citizenship and severely restricts their movement, as well as their access to basic services. Tensions erupted into bouts of bloodshed in 2012 that pushed more than 100,000 Rohingya into grim displacement camps. Despite the squalid conditions in the overcrowded camps in Bangladesh, many of the refugees say they are reluctant to return to Myanmar unless they are granted full citizenship.