The Taliban said it would on Sunday release its first prisoners in a delayed exchange deal with the Afghan government, a potential breakthrough after the insurgents walked out of talks with Kabul last week.

The announcement comes as fears rose that an avenue for fragile peace talks between the two sides was being undercut by mounting disagreements over the prisoner swap, considered key to paving the way for negotiations.

"Today, 20 prisoners of the Kabul administration will be released," Taliban spokesman Suhail Shaheen said on Twitter, adding that the group would be handed over to representatives of the Red Cross in the southern city of Kandahar. The deal calls for the government to release 5,000 Taliban prisoners in exchange for 1,000 government officials held by the Taliban insurgents.

The exchanges come after the Taliban met with the head of U.S. forces in Afghanistan to call for an end to what they say is an increase in American attacks since a peace deal was signed in February, allegations the U.S. military denied.

On Saturday, a U.S. military spokesman called on the Taliban to stop attacking Afghan security forces, whom he said American troops would continue to aid in accordance with the agreement. He spoke on the condition of anonymity in line with regulations.

The peace deal is aimed at paving the way for the U.S. to extricate itself from the 19-year war, America’s longest.

The spokesman confirmed that Gen. Scott Miller met with the Taliban "as part of the military channel established in the agreement" to discuss ways to reduce the violence. According to the Taliban's Shaheen, the meeting was held late Friday in the Gulf nation of Qatar, where the insurgent group maintains a political office. Shaheen tweeted that the two sides held "serious" discussions. He said the Taliban called for a halt to attacks against civilians. The U.S. military says it does not target noncombatants.