How does a Search Engine work?
How does Google Search work?
To a lot of people, the internet = Google. In reality however, the internet is slightly bigger than Google. Actually, it’s a lot bigger. And every time we look up “Justin Beiber” or “Game of Thrones new episode” or even “blue sweater”, the search results that magically appear actually have a lot going in behind the scenes. Let’s find out what the behind-the-scene action is and how search engines (even the ones other than Google) work.
Search Engines work using three basic stages:-
The stage where content is discovered.
In the crawling stage, what basically happens is, bots (called spiders), visit (or crawl) websites at incredibly fast speeds, and analyse all the content on the website, including but not limited to its layout, ads, the words, numbers, images, keywords and links. Any new links found are also added to a list to links to be crawled. Crawling is a continuous process. Spiders crawl websites all the time to constantly update the database.
The stage where content is analysed and stored in databases.
In the indexing stage, all of the data collected by the bots is sorted and filed in huge databases. The database is constantly updated by re-crawling websites.
3. Ranking and Retrieval
The stage where data is fetched in response to a query.
This is the most relevant part of the equation for you and I. Whenever a query is made, the search engine displays results that are most relevant to us. This is done by using various algorithms to “rank” the data retrieved. This is also where search results may vary from engine to engine. Different algorithms rank data differently. Some may rank those results higher that contain greater instances of the words searched for. Some may also consider the proximity of the different words in case of multiple words. Still others may consider the page ranks of the source websites to rank the search results.
A search engine does all that to ensure that the results it displays is the one most relevant to you.
Who would’ve thought that so much work goes in behind a simple Google Search? Amazing isn’t it?