U.S. commanders increasingly are turning to a relatively obscure battlefield detective to find buried bombs, a major killer of Americans in Afghanistan.
But getting permission to buy the detection system has not been easy, a congressman says.
The detective is not a sniffer dog, surveillance gear or metal detector. It is a software platform called Palantir that allows intelligence analysts to quickly mine streams of war-fighting data and calculate likely spots where the Taliban are hiding improvised explosive devices (IEDs) to attack allied convoys.
The Washington Times has obtained internal military documents that show commanders praising Palantir as a roadside-bomb buster. At the same time, they had to urge higher-ups to let them buy the system because the software platform is not standard issue or a separate budgeted item.
“The problem with the military is they have big programs for billions and billions of dollars, and Palantir is a semi-off-the-shelf, awesome product — software that’s written by geniuses not working at the Defense Department,” said Rep. Duncan Hunter, California Republican and member of the House Armed Services Committee.
“This is a long-term Pentagon issue in the way that the Pentagon and the war-fighters kind of fight over what they should be using because they don’t see things eye to eye,” he said.
“Palantir is being used, but not without a fight. Ground combat commanders are having to fight to get stuff to use that the Pentagon doesn’t want them to use.”
The Army told The Times that it has invested in a broad-based intelligence collection system — the Distributed Common Ground System-Army, which it can update as needed. The Army said soldiers are using the ground system and Palantir concurrently.
The Common Ground System “is the Army’s program of record, and has been built deliberately to support current and future requirements of the intelligence community and government agency framework so the Army can save time and money leveraging existing tools and resources,” said Col. Charles Wells, the system’s project manager. “We combine government and commercial software to integrate best-of-breed applications and solutions to our soldiers.”
Documented effectiveness
“Program of record” designation means it is included as a separate item in the Army’s annual budget submission to Congress.
According to Mr. Hunter’s office, the Pentagon has spent about $222 million in the past two years on the Common Ground System and $20 million on the Palantir system, which comprises computer servers, laptops and other support equipment.
Palantir is best known inside government circles as software that can track terrorist financing and uncover fraud. At some point during the war, Palantir Technologies Inc., the company that developed the software, adapted it to do detective work on IEDs as it was fed volumes of information — open-sourced and classified — on battlefield trends.
Internal military documents show commanders pressing their superiors to make Palantir more available.
One such document was written Feb. 13 by Maj. Gen. John Toolan, then the top Marine Corps commander in southwestern Afghanistan, to the Pentagon’s Combating Terrorism Technical Support Office. The office had provided Gen. Toolan with funds to buy three Palantir servers and other components for a yearlong deployment.
“I am pleased to report that Palantir has performed outstandingly,” Gen. Toolan said. “Intelligence analysts find it straightforward and intuitive. Palantir reduced the time required for countless analytical functions and streamlined other, once cumbersome, processes.”
In a sentence that shows Palantir was still not a fixture in the Pentagon’s procurement cycle, Gen. Toolan said: “I hope the Marine Corps will further its relationship with [the combating terrorism office] providing this capability to USMC forces engaged in the current fight and that the Marine Corps will eventually integrate Palantir into its program of record.”
After having taken charge of southwestern Afghanistan’s regional command, Gen. Toolan in June 2011 told Pentagon reporters that his intelligence team was locating where the roadside bombs were being assembled.
’Where the bad guys are’
“One of our primary objectives is to identify the network of those IED builders, find out who they are, and then take them off of the network,” he said. “I think we are getting much better at that process. We have had success almost on a continuous basis since we’ve been here of identifying the IED makers and then targeting them appropriately so we can locate them and either capture them or kill them.”
Another internal document obtained by The Times shows that Army combat units in Afghanistan were seeking Army approval to buy Palantir.
It was written November by an intelligence officer with the 82nd Airborne Division, deployed in Kandahar. The officer sought approval to buy Palantir through the Army’s Rapid Equipping Force. In the memo, the officer compares Palantir with an existing intelligence-sifting system.
“Currently,” the unnamed officer wrote, “there are several databases that IED information is stored on. These databases, although very comprehensive, have to be searched individually for IED reports and currently available tools do not allow for the timely fusion and analysis of the information. Solving very hard analytical problems takes several days when using existing tools against these data sources. In our experience in using the Palantir platform against the same problems, we were able to reduce this time to a few hours. This shortfall translates into operational opportunities missed and unnecessary risk to the force.”
The division ultimately won approval to buy more platforms. In the first three months of 2012, it reported a 12 percent increase in the number of IEDs found and cleared, according to a memo reviewed by The Times.
“Palantir maps out where the bad guys are,” said Mr. Hunter, who fought in Iraq and Afghanistan as a Marine officer. “It maps out all kinds of stuff. Secreted stuff. Open source stuff. It brings it all together for the analysts to say, ’Hey, here’s where they’re going to be putting IEDs.’ It’s pretty amazing for their found-and-cleared rate to go up 12 percent in three months. That is drastic.”
Overall, the NATO command reports a decrease in IED attacks compared with last year. According to the website icasualties.org, of 174 battlefield deaths of Americans this year, 86, or 49 percent, were caused by IEDs.
Palantir Technologies was founded in 2004 by alumni of PayPal, where they developed software to detect fraud.
“Palantir Technologies is working to radically change how groups analyze information,” its website says. “Our products are built for real analysis with a focus on security, scalability, ease of use and collaboration. They are broadly deployed in the intelligence, defense, law enforcement and financial communities, and are spreading rapidly by word of mouth into applications in other industries and realms of impact.”
The company did not respond to a request for comment.
• Rowan Scarborough can be reached at rscarborough@washingtontimes.com.
Please read our comment policy before commenting.