Search Engine History

Just another WordPress.com site

W3Catalog and Wanderer (1993)

leave a comment »

In the early 1990s. Initially, the only widely available browsers were purely textual. Mosaic was the first browser to display images in line with text instead of displaying images in a separate window. While often described as the first graphical web browser. W3Catalog was one of the first search engines that attempted to provide a general searchable catalog for WWW resources.

Unlike later search engines, like Aliweb, which attempt to index the web by crawling over the accessible content of web sites, W3Catalog exploited the fact that many high-quality, manually maintained lists of web resources were already available. W3 Catalog simply mirrored these pages, reformatted the contents into individual entries, and provided a Perl-based front-end to enable dynamic querying.

In 1993, Matthew Gray, then at MIT, produced what was probably the first web robot, the Perl-based World Wide Web Wanderer, and used it to generate an index called “Wandex”. The World Wide Web Wanderer, also referred to as just the Wanderer, was a Perl-based web crawler that was first deployed in June 1993 to measure the size of the World Wide Web.

Wanderer was developed at the Massachusetts Institute of Technology by Matthew Gray, who now works for Google. It was used to generate an index called the Wandex later in 1993. While the Wanderer was probably the first web robot, and, with its index, clearly had the potential to become a general-purpose WWW search engine.

Go back to Search Engine History Index

Advertisements

Written by Bobby

May 23, 2010 at 4:51 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: