Search Engines Explained

What a Search Engine Actually Is

A search engine is a system that reads the entire web, stores a copy of it, and instantly serves the best matches when someone asks a question. Google isn't searching the live internet when you type a query — that would take days. It's searching its own pre-built library of the web, assembled in advance.

Understanding this changes how you think about SEO. Your job isn't to "show up when someone searches." Your job happens long before the search: getting your page into Google's library, making sure it's stored correctly, and earning the signals that make Google pull it to the top when the moment comes.

That entire process runs on three core stages, and every page on the web moves through them in order — with one important sub-step, rendering, hiding between the first two.

Crawling — How Google Finds Pages

Googlebot is an automated program that constantly travels the web by following links. It starts with pages it already knows, finds the links on them, follows those to new pages, finds more links, and repeats — endlessly. This is why links matter so much: they're the roads Googlebot drives on.

A page with no links pointing to it is like a house with no roads leading to it. Googlebot may never find it. This is also why submitting a sitemap (a list of all your pages) helps — it hands Google a map instead of making it find every room by wandering.

Some pages get crawled many times a day; others once a month. Google allocates more crawling attention to sites it considers important and frequently updated. A brand-new site with no reputation gets crawled slowly at first — patience matters.

Indexing — How Google Stores Pages

Once Googlebot crawls a page, Google processes it: it reads the text, analyzes images, evaluates the layout, and works out what the page is about. If the page is worth keeping, Google adds it to its index — the enormous database it consults for every search.

Here's the catch that trips up most beginners: crawled does not mean indexed. Google chooses what to index. If a page is thin, duplicate, low-quality, or blocked by a technical setting, Google may crawl it and then decide not to store it. No index entry means no possibility of ranking — ever.

You can check exactly how many of your pages are indexed in Google Search Console (which you set up in Module 0). The "Pages" report shows indexed vs. not-indexed, and tells you why pages were excluded. This is one of the first things to check when traffic is low.

Rendering — The Step Most People Miss

There's a stage hiding between crawling and indexing that trips up modern sites: rendering. When Googlebot crawls a page, it first sees the raw HTML the server sends. But many sites today build their content with JavaScript — the page arrives nearly empty, then code runs in the browser to fill it in. If your content only appears after JavaScript runs, Google has to render the page (actually run that code) before it can see what's there.

Google does render JavaScript — but rendering is expensive, so it often happens in a second pass, sometimes days after the initial crawl. And if the JavaScript fails, is slow, or is blocked, Google may index a blank or half-built version of your page. Many "why isn't my content showing up?" mysteries come down to this.

Crawl Budget — Why Big Sites Care

Google doesn't crawl every page of every site equally often. It assigns each site a rough crawl budget — how much attention Googlebot spends on you, based on your site's size, speed, health, and importance. For a small site of 20 or 30 pages, this never matters; Google crawls everything easily. It becomes real once a site has thousands of pages.

When budget is limited, you don't want Google wasting it on junk: broken pages, endless filter combinations, duplicate URLs, or thin pages that shouldn't be indexed anyway. Every wasted crawl is attention not spent on your pages that matter. This is why large e-commerce sites obsess over crawl efficiency — and why a clean structure (Module 5) and good technical health (Module 10) pay off at scale.

Ranking — How Google Chooses the Order

This is the stage everyone obsesses over, but it only matters once crawling and indexing are handled. When someone searches, Google pulls every relevant indexed page and orders them using hundreds of signals. Those signals fall into three buckets.

Indexed does not mean ranked. Being in the library is the entry ticket, not a guarantee of a good shelf position. Two pages can both be indexed, yet one sits at position 1 and the other at position 47 — because one matched intent better, was more useful, and carried more authority.

This is also why old content can outrank fresh content. A five-year-old guide that's comprehensive, accurate, and heavily linked-to will usually beat a brand-new post on the same topic. Freshness is only one signal among hundreds, and it matters more for some searches (news, prices) than others (how-to guides, definitions).

The Three Stages, Applied

How Google Understands a Page

Google doesn't "read" like a human, but it's far past matching keywords. It interprets meaning: synonyms, related concepts, the questions a page answers, and how the topic connects to others. A page about "java" is understood as coffee or as code depending on the surrounding words.

This is why keyword stuffing fails and why genuinely covering a topic wins. When you thoroughly answer what a searcher wants — including the related sub-questions — Google recognizes the page as a strong, relevant match. You're not feeding it keywords; you're demonstrating genuine coverage of a topic.