Army’s own data mining system fails test

The Pentagon’s top weapons tester has given a failing grade to the Army’s premier battlefield intelligence processor, which troops in Afghanistan have criticized as being too slow and unreliable in sifting data to find the enemy.

A Nov. 1 memo from the Defense Department’s Operational Test and Evaluation office calls an updated version of the $2.5 billion Defense Common Ground System “not operationally effective, not operationally suitable and not operationally survivable against cyber threats,” according to a copy obtained by The Washington Times.

The across-the-board indictment is a blow to the Army’s most cherished data-collection and analysis system just as appropriators in Congress are taking a closer look at Pentagon expenditures.

What’s more, the common ground system faces more competition from non-Army commercial products, such as one called Palantir, that can outperform it in mining data to identify targets, a process called “link analysis.”

The Times has reported extensively on the Palantir and common ground systems, which military intelligence analysts use to determine the probable location of roadside bombs, the No. 1 killer of U.S. troops in Afghanistan.

The Nov. 1 internal memo, sent to Frank Kendall, undersecretary of defense for acquisition, said the Army is proposing a solution to address the common ground system’s operational failure: Eliminate the software “enclave” that handles sensitive classified data.

The enclave is apparently why the system, in part, performed so poorly in recent tests. Without it, the common ground system should “significantly improve the work flow,” the memo states.

According to the memo, the Army is asking Mr. Kendall to approve a scaled-back version for deployment to troops.

Limiting capabilities

Two members of Congress are telling Mr. Kendall that permitting a scaled-back version of the Defense Common Ground System (DCGS) to be deployed is “deeply troubling.”

In a Nov. 28 letter, Reps. Duncan Hunter, California Republican and a member of the House Armed Services Committee, and James P. Moran, Virginia Democrat and a member of the House Appropriations subcommittee on defense, said a restricted system “represents yet another setback for an already troubled program.”

“The very people it was designed for, the warfighters, are requesting commercial off-the-shelf software in lieu of DCGS to accomplish their mission,” the congressmen wrote. “Specifically, by our count, at least 13 deploying Army units have requested commercial off-the-shelf solutions in place of DCGS, indicating that DCGS has so far failed to meet our warfighters’ in-theater needs.”

The Army referred questions about the test to the Pentagon, where Air Force Lt. Col. Melinda Morgan, a Defense Department spokeswoman, provided a statement explaining the proposed modification.

The Pentagon’s operational test found that classified data within the common ground system could not be passed down from division level to the brigade.

“The sheer effort of moving data across those boundaries led to substantial delays and other problems,” the statement says.

“Clearly [the new version] is less capable,” the statement says. Fielding a scaled-back processor “will provide users capabilities at least as good as those provided by the currently fielded versions.”

Mr. Hunter, and critics in the field, say that justification rings hollow because it is the current version that troops generally pan.

Mr. Hunter and Mr. Moran are not alone.

The Senate Armed Services Committee report for fiscal 2013 defense spending criticizes the Army for refusing to incorporate commercially available software into the common ground system. The top intelligence officer in Afghanistan in 2010 made such a request, the report said.

“The Army indicated that only 115 analysts in Afghanistan are using the Army’s DCGS cloud analyst tools, despite years of development and considerable costs,” the senators said, adding that they “lacked confidence” that the Army was ever going to field a “fully capable” system.

Requests denied

The experience of an Army commando unit in Afghanistan in 2011 is an example of soldiers requesting Palantir, an off-the-shelf commercial data analysis system that costs far less than the Defense Common Ground System.

Special operations Task Force 10, as it was then called, had become fed up with the common ground system, which was failing in its link analysis to identify insurgents.

“The current system does not allow them to efficiently request and receive critical real-time intelligence,” said one of several memos that were sent by U.S. commandos to Army headquarters and obtained by The Times.

The task force asked Army headquarters in Washington for permission to buy Palantir, which was gaining fame as a conduit for finding buried improvised explosive devices (IEDs).

The Army said “no” to Palantir and, instead, noted that it was sending more common ground system servers — the same ones that special warriors said did not work.

The Army did not respond last week to questions submitted by The Times.

“This is just another instance of ground forces asking for a specific technology to support their mission and getting their request denied because there’s competing technology that’s significantly more expensive and far-less effective,” said Mr. Hunter, a who served in Iraq and Afghanistan as a Marine Corps officer.

Internal memos obtained by The Times in recent months show that Army soldiers are not the only ones requesting Palantir.

A Marine general wrote that Palantir should be included in the annual defense budget. It can be bought now only as a special request item. U.S. Special Operations Command is trying to make Palantir standard issue to its units fighting terrorists around the world.

’Without explanation’

In spring 2011, Task Force 10, a combined unit of U.S. and allied troops, was training specialized Afghan units in counterterrorism, then going out on missions to kill the enemy.

The paper trail rings of desperation.

“TF-10’s current battlefield visibility and information platform system limits their ability to see and efficiently share battlefield information in real-time,” said a March 2011 message from the unit’s chief of staff. “The Palantir system will significantly and efficiently increase TF-10’s situational awareness and intelligence sharing capabilities.”

A month later, the chief of staff for all U.S. forces in Afghanistan added his endorsement: “The current system does not allow [Task Force 10] to efficiently request and receive critical real time intelligence,” he wrote.

At the Pentagon, Army procurement officials were not impressed.

That May, the Army wrote back to Task Force 10 that it “validates the requirement for secure information sharing across organizational boundaries and in real-time.” But the message added that the Army “returns the request for Palantir without action.”

It said the common ground system program manager was sending new software to Afghanistan.

Mr. Hunter told The Times: “This is even one of the more egregious cases, with the Army officially acknowledging the need for the capability, but then turning down the request without explanation and instead sending servers that just sat in storage and collected dust.”

Other units in Afghanistan over the past three years have bypassed the Army hierarchy and appealed directly to Pentagon offices that can approve buying Palantir.

In a powerful endorsement of Palantir, the director of intelligence for U.S. Special Operations Command sent a memo in August to a special Pentagon procurement office seeking permission to buy Lighthouse, a mobile data collector on the battlefield, and link it with Palantir.

Using the parlance of intelligence professionals, the memo said: “Lighthouse and Palantir users are equipped to exploit structured data using link analysis, data mining” and other techniques.

It said Special Operations Command planned to deploy the Lighthouse-Palantir hookup in ongoing operations in Afghanistan, the Philippines, the Horn of Africa, trans-Saharan Africa, and Central and South America.