BRUSSELS—In a troubled part of West Africa, Europe is fielding a military force intended to show it can handle its own security and reduce its reliance on the U.S.

The coalition of European countries, led by France, is assembling based on a calculation that helping each other overseas will cement security and defense back in Europe.

But the effort in the Sahel, the semiarid belt running along the southern edge of the Sahara, faces an uphill battle. European forces are scattered across various missions and have differing appetites for lethal combat. Military and civilian deaths are mounting while the number of violent attacks by Islamist extremist groups has risen sharply. Local allies have been accused of human-rights abuses, and one of them, the president of Mali, was ousted in a coup in August.

France and Estonia this summer deployed elite troops as part of a new task force that will train local militaries and take part in counterterrorism operations with them. Commandos from countries including the Czech Republic, Sweden, Belgium and Italy will join over the next year.

The Takuba task force, named after a local word for a saber, is a signature initiative of an effort by French President Emmanuel Macron—who questions U.S. commitment to Europe—to bolster Europe’s military capabilities.

As the Trump administration has cut troop numbers in Africa and announced plans for reductions in Europe, Mr. Macron has led calls for what he has dubbed a European army. Mr. Trump hit out at the suggestion, tweeting that it was “very insulting” and that Europe should “pay its fair share of NATO.”

To be sure, no one sees the efforts as a replacement for the U.S.-led North Atlantic Treaty Organization. The European effort in Africa is relatively modest and is more of a coalition of forces than a joint army.

But the combination of U.S. reticence and French ambition is pushing the project forward and bringing in European allies.

“When France gets involved, Europe gets involved,” Mr. Macron said at a meeting in Mauritania with the leaders of five Sahel countries in June.

“Macron sees the vacuum in international politics and wants to fill it but doesn’t have the means,” says Yvan Guichaoua, an expert on the Sahel at the Brussels School of International Studies, part of the U.K.’s University of Kent.

“France wants to share the burden and the political responsibility and has persuaded others that the Sahel is a European problem,” said Mr. Guichaoua.

France, once the main colonial power in the Sahel, has some 5,100 troops in the region as part of efforts, launched in 2013, to counter a militant insurgency. They are spread among three permanent bases in Mali, Niger and Chad and several other temporary advanced bases in those countries. Some allies, including Germany, are focused on the United Nations’ peacekeeping mission in Mali. Others are part of the European Union’s training mission for the Malian armed forces.

Takuba is aimed at bolstering the performance of local forces in the fight against Islamist militants by training and mentoring troops from the region and accompanying them into battle.

Estonia, a tiny Baltic nation with no evident link to Africa, has assigned a small team of special-forces troops to Takuba.

“Estonia wants to find a solution to a wider European problem,” Estonian Defense Minister Jüri Luik said in an interview. “If we are ready to help others, others are ready to help us.”

French soldiers have rotated through Estonia as part of a battalion aimed at deterring Russia.

“It’s not a bargain, but it’s a spirit of solidarity,” Mr. Luik said.

French President Emmanuel Macron spoke to the media in Mauritania in June.

French President Emmanuel Macron spoke to the media in Mauritania in June.

Photo: Ludovic Marin/Associated Press

“The idea that Europe needs to take more responsibility for its own security interests is right,” said Danish Foreign Minister Jeppe Kofod in an interview. Danish service members are part of both the French counterterrorism mission and the U.N. mission. “There’s no conflict between U.S. interest and European interest. But Europe has been too reluctant [for] too long to invest in security in our neighborhood.”

Some analysts say European efforts in the Sahel are too small and tasks too diffuse. But boosters say each country’s different approach adds up to a more nuanced coalition.

“Europeans won’t all become France look-alikes,” said Olivier-Rémy Bel, a visiting fellow at the Atlantic Council in Washington and a former French Defense Ministry official. “But by working together they are bringing their strategic cultures together.”

The European efforts also rely on U.S. intelligence and logistical support. Europe doesn’t have enough drones to conduct surveillance of the Sahel, and France needs U.S. aircraft to deploy forces in many of its missions, according to French officials. Amid rumors that the U.S. would pull back some troops last year, France’s defense minister quickly flew to Washington to lobby against it.

“It’s good that France wants to do more,” said a senior U.S. official. “But I think they have realized that they can’t do it alone.”

Write to James Marson at james.marson@wsj.com