Related Posts

Share This

History of Search Engine Episode II

This article is the continuation from my previous on in here.

After the era of Archie and friends on the web, came along a new search engine technology called the World Wide Web Wander by Matthew Gray. This was the first web robot with its aim to track the web’s growth by counting web servers. Then it began to count and index url into database as well.

Development of World Wide Web Wanderer inspired a lot of developers out there, to innovate and try out new form of searching. One of the most prominent ones was ALIWEB, a web indexing software that worked like Archie by Martinjn Koster – In fact, ALIWEB stands for “Archie Like of the Web”.

ALIWEB works like known we have today as Online Directory of websites (the first Online Directory of websites) and it’s not a robot like Wanderer. It allows webmasters to post and submit their links and websites.

Then in end of 1993, three new search engines were launched with advance robot inspired by the first Wanderer. They were:-

  • JumpStation – indexes as far as reading out header and title of the website. Then introduce ordering.
  • World Wide Web Worm – indexes URL and the corresponding web title but lack of ordering
  • Repository Based Software Engineering Spider (RBSE) – got around by doing the same and also by introducing the term we know now as relevancy and ranking.
Later the following year after all 3 of the search engine started operating. Came another one called Architext which also try to do the same thing but, lack of intelligence. They got around eventually by using a product called Elnet Galaxy . Achitext was the search software that became the Excite Engine. Its websites links were organized in a hierarchical structure and divided into subcategories much like what Yahoo! search engine is today.
The next significant stage was the creation of Yahoo! directory in April 1994 – which began as a couple of students list of favorite web pages and grew into the worldwide phenomenon. Yahoo! guaranteed the quality of the websites it listed because, they were accepted and rejected by human editors manually. The advantage of the directories plus their guaranteed quality, was their detailed description that comes along with the title of the page.
The first advance robot created by University of Washington was called WebCrawler which features indexing of full text of documents, allowing users to search these in web. WebCrawler was then adopted by American Online (AOL) who purchased the system. WebCrawler was the first search engine that could index full text of a page of HTML.
Well, that’s it for part II of the amazing history of search engine. Hope you enjoy reading it as I do. Next episode I’ll cover a few more emerging companies and take overs which eventually lead to the Rise of Google.