June 7, 2005: The U.S. government is investing a lot of money to develop a search engine that will out-Google Google. The new system detects concepts, rather than just words and phrases, in online (or off-line) material (documents, email, chat room text). Actually, this is an old concept, but it requires lots of computer power to make it work. Someone noticed that the U.S. government now has access to lots and lots of computer power. Someone thought, "we-can-do-this." And it was done. Actually, many outside observers believed that it had been done years ago. Maybe it was, and this is just a new, or simply public, version. No matter.
The new concept search engine (CSE) was developed for counter-terrorism work. The CSE looks for connections between two people that it would normally take a human investigator to figure out. Given the huge mass of data on the Internet (including billions of words of material captured from instant messaging, chat rooms and emails), there's no way humans could comb through it all for anything other than key words and phrases. The CSE makes sense of material, stores those interpretations and constantly compares all that it has found for connections between two or more authors, or members of the same organization (or way of thinking). CSE is expected to be a powerful tool in searching for terrorist activity on the Internet. |