robots.txt is your friend

by Prometheus 6
February 9, 2004 - 6:12am.
on News

Online Search Engines Help Lift Cover of Privacy
By Yuki Noguchi
Washington Post Staff Writer
Monday, February 9, 2004; Page A01

Sitting at his laptop, Chris O'Ferrell types a few words into the Google search engine and up pops a link to what appears to be a military document listing suspected Taliban and al Qaeda members, date of birth, place of birth, passport numbers and national identification numbers.

Another search yields a spreadsheet of names and credit card numbers.

"All search engines will get you this," O'Ferrell said, pointing to files of spoils he has found on the Internet: Medical records, bank account numbers, students' grades, and the docking locations of 804 U.S. Navy ships, submarines and destroyers.

And it is all legal, using the world's most powerful Internet search engine.

Cybersecurity experts say an increasing number of private or putatively secret documents are online in out-of-the-way corners of computers all over the globe, leaving the government, individuals, and companies vulnerable to security breaches. At some Web sites and various message groups, techno-hobbyists are even offering instructions on how to find sensitive documents using a relatively simple search. Though it does not technically trespass, the practice is sometimes called "Google hacking."

"There's a whole subculture that's doing this," said O'Ferrell, a long-time hacking expert and chief technology officer of Herndon-based security consultancy Netsec Inc.