This is how Search Engines Operate
From llnl.gov 2008 Search engines accept a abbreviate account of analytical operations that allows them to accommodate accordant web after-effects back searchers use their arrangement to acquisition information.
Crawling the Web
Chase engines run automatic programs, alleged “bots” or “spiders”, that use the hyperlink anatomy of the web to “crawl” the pages and abstracts that accomplish up the World Wide Web. Estimates are that of the about 20 billion absolute pages, chase engines accept crawled amid 8 and 10 billion.
Indexing Documents
Once a folio has been crawled, its capacity can be “indexed” - stored in a behemothic database of abstracts that makes up a chase engine’s “index”. This basis needs to be deeply managed so that requests which charge chase and array billions of abstracts can be completed in fractions of a second. source : space.swri.edu 2008
