July 1,
2008: The U.S. Congress has refused to
give the FBI $11 million to expand the use of data mining in counter-terrorism
efforts. American politicians are generally hostile to government use of data
mining, a technique widely used, for decades, in business (marketing), law
enforcement (catching criminals) and the military (finding the enemy). This
last use has become much more sophisticated since the U.S. Department of
Defense began pouring billions of dollars a year into finding ways to defeat
IEDs (improvised explosive devices, usually roadside bombs). The effort to
lower IED casualties has opened up all sorts of opportunities for technological
innovation. No one harasses researchers for using data mining, or makes fun of
building supercomputers with graphics processors (often the same ones found in
video game consoles, making super-fast computers cheap enough to be used in a
combat zone to make life saving predictions), when it saves troops from getting
killed.
The data
mining was initially used to figure out who the bomb making crews were, and
where they operated from. Then, using math techniques first developed during
World War II, the intel geeks began creating predictions about where IEDs were
most likely to show up next. These predictive models get better as the quality
of the information going into them improves. As more terrorists are captured
and interrogated, and their computers and data is translated, the predictions
become more accurate.
Using more
primitive computers, Germany employed data mining successfully in the 1970s, to
find leftist, middle class terrorists who were operating with assistance from
the East German secret police. The terrorists thought they were well concealed,
but data mining can do wonders with the slightest pieces of information.
The FBI
has been unable to make this point to Congress, mainly because some key
legislators are ideologically opposed to data mining, and refuse to acknowledge
the widespread success of the technique in civilians and military sectors.
