From seven years ago, Bruce Schneier explains why collecting massive amounts of data won’t allow us to find terrorist patterns hidden within:
Many believe that data mining is the crystal ball that will enable us to uncover future terrorist plots. But we’re not going to find terrorist plots through systems like this. We’re not trading privacy for security; we’re giving up privacy and getting no security in return.
Data mining works best when there’s a well-defined profile you’re searching for, a reasonable number of attacks per year, and a low cost of false alarms. Credit card fraud is one of data mining’s success stories: all credit card companies data mine their transaction databases, looking for spending patterns that indicate a stolen card. Many credit card thieves share a pattern — purchase expensive luxury goods, purchase things that can be easily fenced, etc. — and data mining systems can minimize the losses in many cases by shutting down the card. In addition, the cost of false alarms is only a phone call to the cardholder asking him to verify a couple of purchases.
Terrorist plots are different. There is no well-defined profile, and attacks are very rare. Taken together, these facts mean that data mining systems won’t uncover any terrorist plots until they are very accurate, and that even very accurate systems will be so flooded with false alarms that they will be useless.
There are trillions of connections between people and events — things that the data mining system will have to “look at” — and very few plots. This rarity makes even accurate identification systems useless. Terrorist attacks are rare, so any “test” is going to result in an endless stream of false alarms.