ISLAMABAD, Pakistan — Even as hundreds of thousands of people stream back to the Swat Valley after months of fighting, one important group is conspicuously absent: the wealthy landowners who fled the Taliban in fear and are the economic pillar of the rural society.
The reluctance of the landowners to return is a significant blow to the Pakistani military’s campaign to restore Swat as a stable, prosperous part of Pakistan, and it presents a continuing opportunity for the Taliban to reshape the valley to their advantage.
About four dozen landlords were singled out over the past two years by the militants in a strategy intended to foment a class struggle. In some areas, the Taliban rewarded the landless peasants with profits of the crops of the landlords. Some resentful peasants even signed up as the Taliban’s shock troops.
How many of those peasants stayed with the militants during the army offensive of the last several months, and how many moved to the refugee camps, was difficult to assess, Pakistani analysts said.
But reports emerging from Swat show that the Taliban still have the strength to terrorize important areas. The army continues to fight the Taliban in their strongholds, particularly in the Matta and Kabal regions of Swat, not far from the main city, Mingora, where many refugees have reclaimed their homes.
In those regions, the Taliban have razed houses, killed a civilian working for the police in Matta and kidnapped another, worrying counterinsurgency experts, who fear that the refugees may have been encouraged by the Pakistani authorities to go back too soon.
The rebuilding of Swat, a fertile area of orchards and forests, is a critical test for the government and the military as they face Taliban insurgencies across the tribal belt, particularly in Waziristan on the Afghanistan border.
In a sign of the lack of confidence that Mingora was secure, the Pakistani military declined a request by the Obama administration’s special envoy to Pakistan, Richard C. Holbrooke, to visit the town last week.
There was nervousness, an American counterinsurgency expert said, that the plans by the Pakistani authorities to build new community police forces in Swat would not materialize quickly enough to protect the returning civilians, who are also starved of basic services like banks and sufficient medical care.
“There is no apparatus in place to replace the army,” said an American counterinsurgency official. “The army will be the backstop.”
About two million people have fled Swat and surrounding areas since the military opened its campaign to push back the Taliban at the end of April. The United Nations said Monday that 478,000 people had returned to Swat so far, but it cautioned that it was unable to verify the figure, which was provided by the government.
Assessment trips by United Nations workers to Swat scheduled for Monday and Tuesday were canceled for security reasons, and the United Nations office in Peshawar that serves as the base for Swat operations was closed Monday because of a high threat of kidnapping, a spokesman said.
The landlords, many of whom raised sizable militias to fight the Taliban themselves last year, say the army is again failing to provide enough protection if they return.
Another deterrent to returning, they say, is that the top Taliban leadership, responsible for taking aim at the landlords and spreading the spoils among the landless, remains unscathed.
If it continues, the landlords’ absence will have lasting ramifications not only for Swat, but also for Pakistan’s most populated province, Punjab, where the landholdings are vast, and the militants are gaining power, said Vali Nasr, a senior adviser to Mr. Holbrooke, the American envoy.
“If the large landowners are kept out by the Taliban, the result will in effect be property redistribution,” Mr. Nasr said. “That will create a vested community of support for the Taliban that will see benefit in the absence of landlords.”
At two major meetings with the landlords, the Pakistani military and civilian authorities requested that they return in the vanguard of the refugees. None have agreed to do so, according to several of the landowners and a senior army officer.
“We have sacrificed so much; what has the government and the military done for us?” asked Sher Shah Khan, a landholder in the Kuz Bandai area of Swat. He is now living with 50 family members in a rented house about 60 miles from Swat. Four family members and eight servants were killed trying to fight off the Taliban, he said.
At one of the meetings, Mr. Khan said he had asked the army commanders to provide weapons so the landlords could protect themselves, as the landowners had in the past.
The military refused the request, he said, saying it would fight the Taliban. Yet Pakistani soldiers had failed to protect his lands, he said. Twenty of his houses were blown up by the Taliban after the army ordered him and his family to leave their lands on two hours’ notice last September, he said.
A letter he sent last month to Gen. Ashfaq Parvez Kayani, the head of the Pakistani military, asking for compensation has gone unanswered, he said. In the meantime, one of his tenants called asking if he could plant crops on Mr. Khan’s property. He refused but had little idea what was happening back home, Mr. Khan said.
Other landlords are equally frustrated. The mayor of Swat, Jamal Nasir, fled after his father, Shujaat Ali Khan, regarded as the biggest landlord in Swat, narrowly avoided being killed by the Taliban. Mr. Nasir, a major landowner himself, now stays in his house in Islamabad.
The top guns of the Taliban are still in Swat, or perhaps in neighboring Dir, Mr. Nasir said. “These people should be arrested,” he said. “If they are not arrested, they are going to come back.”
Another landlord, Sher Mohammad, said he was still bitter that the army refused to help as he, his brother and his nephew fought off the Taliban last year for 13 hours, even though soldiers were stationed less than a mile away. Mr. Mohammad was hit in the groin by a bullet and lost a finger in the fight.
At one of the meetings with the military in Peshawar, Mr. Mohammad, a prominent politician with the Pakistan Peoples Party, said he told the officers that he was not impressed with their performance.
“They said, ‘We will protect you,’ ” he recalled. “I said, ‘We don’t trust you.’ ”
