ever wonder if it’s even possible to search every corner of the great world wide web?
well this is the goal of several orgs now, one of them is the deeppeep by the university of utah, they hope to crawl and index every database on the web.
another is by kosmix, which has developed a software that matches searches with the databases most likely to yield relevant information.
for the IT PINOYs – how this works is that to extract these info from the web, these search engines have to have an efficient way to analyze user-entered query strings and to immediately broker these to a specific databases for faster results.
this would be a daunting task, google which just reached it’s trillionth address added to list of known websites believes that they are far from fully exploring the deep web!
read more @ [nytimes.com]
set it good 