Skip to content
Learn

What SEO is and how search works

In this lesson

Section titled “In this lesson”
  • Tell the difference between SEO, GEO and AEO, and between organic, paid and direct traffic.
  • Follow one page through the stages of a search engine: find, crawl, index and serve.
  • Name the crawlers of Google and Bing, and say how to check that a visit is really from them.
  • Classify a query by its search intent.
  • Explain lexical, vector and hybrid search, and the difference between training data and grounding.

This is the first day of the SEO topic. It gives you the map. The next eight days fill in each part of it.

SEO, GEO and AEO

Section titled “SEO, GEO and AEO”

SEO (search engine optimization) is the work that helps people find your site through a search engine. Google’s own definition is short: “SEO … is about helping search engines understand your content, and helping users find your site.” SEO does not buy a position. Google says that it “doesn’t accept payment to rank pages higher, and ranking is done programmatically.”

Today, people also get answers from AI systems, for example Google AI Overviews, ChatGPT and Perplexity. Two newer terms describe work for these systems:

Term Full name The goal
SEO Search engine optimization Your page is found and ranks in the list of results.
GEO Generative engine optimization An AI answer uses and cites your page.
AEO Answer engine optimization A system can take a direct answer from your page, for example for a featured snippet, a voice assistant or a chat answer.

GEO comes from a 2023 research paper by Aggarwal and others. It defines generative engines as systems that “use generative models to gather and summarize information to answer user queries.” In its tests, some changes to content made a page up to 40% more visible in generated answers. These are results from one study, not a rule for every AI system.

The terms are not standard. Different writers use GEO and AEO in different ways. Google says: “There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.” In this topic, treat SEO, GEO and AEO as three layers of one job: be found, be cited and be quoted. The base layer is the same for all three. A system must find and store your page before it can rank, cite or quote it.

Where visitors come from

Section titled “Where visitors come from”

A visit to a site has a source. Analytics tools put each visit into a group, called a channel. Three channels are important for SEO:

Channel What it is Google Analytics 4 rule (short form)
Organic A click on an unpaid search result. The source is a known search site, or the medium is organic.
Paid A click on a search ad. The source is a search site, and the medium shows a paid click, for example cpc or ppc.
Direct The visitor typed the URL or used a bookmark. Analytics also uses this group when it does not know the source. The source is (direct), and the medium is (none) or (not set).

SEO works on the organic channel. Ads and organic results use different systems. Google matches ads to the advertiser’s keywords and orders them by “a combination of cost-per-click (CPC) bid and relevance.” Organic results are not part of Google’s advertising programs. On the results page, Google puts text ads under a “Sponsored results” label. Since October 2025, a “Hide sponsored results” control lets the user collapse the text ads.

The search results page

Section titled “The search results page”

The SERP (search engine results page) is the page that you see after you search. It is not only ten blue links. Google calls its parts visual elements and groups them like this:

  • Text result: a title link, a snippet (a short extract), and the site name and URL.
  • Rich result: a result with extra graphics or data. It usually needs structured data on the page (day 05).
  • Image result and video result: these are more likely when the query asks for images or videos.
  • Exploration features: for example, related searches and related questions.

Ads and AI Overviews can also show on the page. The layout changes with the device, country, language and query, so two people can see different pages. Day 04 teaches SERP features in more detail.

How a search engine works

Section titled “How a search engine works”

Every large search engine does the same three jobs. Google calls them crawling, indexing and serving. Learn these three stages first. Every later day of this topic depends on them.

1. Crawling. A crawler is a program that downloads web pages. Google’s crawler is Googlebot. First, Google must know that a URL exists. Google finds most URLs through links from pages that it already knows. It can also get URLs from a sitemap, a file that lists the URLs of a site. To crawl a page is to download it. During crawling, Google also renders the page: it runs the JavaScript “using a recent version of Chrome.” Then it sees the same content as a browser user.

Crawling is not certain. Google says that Googlebot “doesn’t crawl all the pages it discovered.” The site owner can block a page with the robots.txt file. Googlebot also cannot read a page that needs a login.

2. Indexing. The index is the large database of pages that the search engine can show. During indexing, Google reads the text, the <title> element, alt attributes, images and videos. It also checks if the page is a copy of another page. From a group of copies, Google picks one canonical page. The canonical is the version that can show in results. Indexing is not certain either: “not every page that Google processes will be indexed.”

3. Serving and ranking. When a user searches, the engine takes matching pages from the index and puts them in order. This order is the ranking. Google says that it uses “automated ranking systems that look at many factors and signals about hundreds of billions of web pages.” Relevance can depend on “the user’s location, language, and device.”

Facts about the page

Turn the facts about the page on and off. Notice that a page can fail at each stage, and that a failure at an early stage stops all later stages.

The order of the stages gives the most important rule of technical SEO. A page that is not crawled cannot be indexed, and a page that is not indexed cannot rank. When a page gets no search traffic, check the stages in order. Day 03 teaches the tools for each stage: robots.txt, sitemaps, canonical tags and status codes.

Googlebot and Bingbot

Section titled “Googlebot and Bingbot”
Googlebot Bingbot
Owner Google Microsoft Bing
Types Googlebot Smartphone and Googlebot Desktop. Google indexes mainly the mobile version of a page, so most crawls use the smartphone crawler. Both types obey the same robots.txt token, Googlebot. A desktop and a mobile user agent. Both contain bingbot/2.0.
Rendering A recent version of Chrome. The newest stable version of Microsoft Edge. Bing calls this “evergreen”.
Limits and speed It reads the first 2 MB of a supported file and the first 64 MB of a PDF. On most sites, it visits “no more than once every few seconds on average.” Bing reads your sitemap, and it accepts change notices through IndexNow (see below).

Anyone can copy a crawler’s user agent string, so do not trust it alone. Google tells you to do a reverse DNS lookup on the IP address, or to compare the IP address with the published Googlebot IP ranges. Bing Webmaster Tools has a “Verify Bingbot” tool for the same check.

Bingbot is more important than Bing’s market share suggests. Other services use the Bing index. DuckDuckGo says that it “largely source[s]” its traditional links from Bing. Microsoft also sells “Grounding with Bing Search” to AI agents. So a page that Bingbot cannot read can be missing from more than one product.

IndexNow is a simple ping. With it, a site tells search engines that a URL is “added, updated, or deleted.” In October 2026, the IndexNow site lists Microsoft Bing, Naver, Seznam.cz, Yandex and Yep as members. Google is not on the list.

Search intent

Section titled “Search intent”

Search intent is the goal behind a query. Intent often matters more than the exact words. Google’s guidelines for its human quality raters use four intent types:

Intent The user wants to … Example query
Know find information or learn about a topic. A Know Simple query needs a short fact. how tall is barack obama
Do do a task, for example buy, download or watch. pay parking ticket
Website go to one site or page. reddit login
Visit-in-person find a place near them. coffee shop

Many SEO tools use a different set of four: informational, navigational, commercial and transactional. The two sets overlap. Informational is close to Know, and navigational is close to Website. The guidelines also say that “many queries do not fit neatly into one and only one of these categories.”

A query can also have more than one meaning. For apple, the company is the dominant interpretation: what most users mean. The fruit is a common interpretation. The rater guidelines also say: “No single rating can directly impact how a particular webpage … appears in Google Search.” Google uses the ratings to measure and improve its systems, not to rank single pages.

Retrieval and ranking

Section titled “Retrieval and ranking”

The serving stage has two steps. Retrieval finds a set of candidate pages in the index that can match the query. Ranking puts those candidates in order. Retrieval must be fast and must not miss good pages. Ranking can use more signals, because it works on fewer pages. This split is important for SEO and GEO. A page that is not retrieved never gets a chance to rank. Google describes its neural matching system as “a critical part of how we retrieve relevant documents.” It also says that BERT helps both to rank pages and to retrieve them.

Section titled “Lexical, vector and hybrid search”

There are two main ways to retrieve text:

  • Lexical search matches the words of the query with the words in the page. A common scoring method is BM25. Lexical search is precise and easy to explain. It finds exact terms such as product names and error codes. It misses a page that uses other words for the same idea.
  • Vector search changes the query and each page into an embedding: a list of numbers that represents the meaning. Then it finds the pages whose embeddings are nearest to the query embedding. It finds “cheap flights” for the query “affordable plane tickets”. It can miss rare exact terms.

Semantic search is search by meaning, not by exact words. Vector search is the usual way to do it. Semantic search can also use context, for example the user’s location.

Hybrid search runs both methods and joins the two result lists into one. A common way to join them is Reciprocal Rank Fusion (RRF). For each page, RRF adds 1 ÷ (60 + rank) from each list. The authors fixed the number 60 in a pilot test, and they found that the exact value was “not critical.” A page that is high in both lists gets the highest score.

Example data. Pick a query, then change the search method. Notice that the lexical method misses 'Cheap flights to Paris', the vector method puts the exact error page in third place, and the hybrid method puts pages that both methods found at the top.

For SEO, the lesson is simple. Use the exact words that people search for, so that lexical retrieval finds you. Also explain the topic clearly and completely, so that meaning-based retrieval finds you. Day 07 shows how AI search uses embeddings and chunks of text.

Personalization and localization

Section titled “Personalization and localization”

The results are not one fixed list for everyone. Two kinds of change are possible:

  • Personalization uses information about the user. For a signed-in user, Google can use things like the Search history in the Google Account. The user can turn this off.
  • Localization uses the place and language of the search. Google says that “bicycle repair shops” gives different results in Paris and in Hong Kong. Google estimates the place from the device location, the IP address, saved home and work addresses, and past searches.

Even with personalization off, Google still uses the location, the language, the type of device and the current searches. Because of this, a rank check from your own browser shows only one view. Day 08 teaches rank tracking and other ways to measure. Day 09 teaches local and international SEO.

Training data and grounding

Section titled “Training data and grounding”

An AI model learns from training data. That knowledge stops at a date, called the knowledge cutoff. Microsoft’s documentation says it directly: models “can’t access new information beyond a fixed point in time.”

Grounding gives the model current information when it answers. The system runs one or more searches, reads the results and writes an answer that is based on them. Google says that grounding with Google Search can “reduce model hallucinations” and lets an application “provide citations.” Google AI Overviews and AI Mode use a related method, query fan-out: they issue “multiple related searches across subtopics and data sources.”

Training data Grounding
When the model gets it Before release At the time of the question
Can it change after release? No Yes, with each search
Can it link to your page? Usually not Yes, it can cite the source
How you influence it Hard to influence The normal SEO stages: crawl, index, rank

This is why the stages in this lesson are also the base of GEO. For AI Overviews and AI Mode, Google says that a page must be “indexed and eligible to be shown in Google Search with a snippet.” Grounded answers come from a search index. If your page is not in the index, a grounded answer cannot cite it.

What matters most

Section titled “What matters most”

This lesson has many terms, but four ideas matter most:

  1. Crawl, index, serve. Every result, organic or AI, starts with a page that the system found, crawled and stored.
  2. Retrieval comes before ranking. A page that is not retrieved does not rank, and AI systems cannot cite it.
  3. Intent beats exact words. Match the goal of the query, and use the words that people use.
  4. There is no single SERP. Device, place, language and history change what each person sees.

Day 02 adds the rules: what search engines punish, and how they change their systems.

Key terms

Section titled “Key terms”
  • SEO: the work that helps people find a site through the organic results of a search engine.
  • GEO: the work that helps AI systems use and cite your content in generated answers.
  • AEO: the work that makes it easy for a system to take a direct answer from your content.
  • Organic traffic: visits from unpaid search results.
  • Direct traffic: visits from a typed URL or a bookmark, or visits with an unknown source.
  • SERP: the search engine results page.
  • Crawler: a program that finds and downloads web pages, for example Googlebot or Bingbot.
  • Crawl: to download and read a page.
  • Index: the database of pages that a search engine can show in results.
  • Canonical page: the one version, from a group of copies, that can show in results.
  • Ranking: the order of the results for a query.
  • Retrieval: the step that finds the candidate pages for a query before ranking.
  • Search intent: the goal behind a query.
  • Embedding: a list of numbers that represents the meaning of text.
  • Lexical search: retrieval that matches the words of the query.
  • Vector search: retrieval that matches embeddings, so it matches meaning.
  • Hybrid search: retrieval that joins lexical and vector results into one list.
  • Grounding: giving an AI model current search results when it answers.
  • Knowledge cutoff: the date after which a model has no training data.
  • IndexNow: a ping that tells some search engines that a URL changed.

Exercise

Section titled “Exercise”

Exercise · about 15 min

Read the SERP for three intents

Use a private browser window, so that your Search history does not change the results.

  1. Write three queries about one subject that you know. Make one Know query, one Do query and one Website query. Example: how does a heat pump work, buy heat pump filter and the name of a heat pump brand.
  2. Search each query on Google. For each SERP, write down which elements you see: sponsored results, an AI Overview, text results, image or video results, related questions or a map.
  3. Do the same three searches on Bing. Write down one difference for each query.
  4. Add a city name to the Do query, for example heat pump installer Melbourne. Write down what changes.
  5. For one organic text result, write which intent the page serves. Does it match the intent of your query?
Show a possible solution

Your results depend on the date, the place and the device, so there is no single correct list. These patterns are common:

  • The Know query often gets an AI Overview or a short answer, related questions and text results from guides.
  • The Do query often gets sponsored results and shopping or product results. The text results are often shop pages.
  • The Website query usually puts the brand’s own site first, sometimes with sitelinks under it.
  • The city name makes the results more local. You often see a map with local businesses (day 09).
  • Bing and Google use different indexes and ranking systems, so the order and the elements are different.

If the top result does not match your intent, the query probably has more than one interpretation. The engine shows a mix of results for the different interpretations.

Check yourself

Section titled “Check yourself”
A new page has no links to it from any other page, and no sitemap lists it. What is the most likely result?
Googlebot crawled a page, but the page is not in the index. Which reason fits what Google says about indexing?
A user searches for `XR-200 battery`. The best page uses the exact product code but few other words. Which retrieval method is most likely to miss it?
Why does SEO work still matter for answers from Google AI Overviews?
You check your rank for `plumber` from your own laptop and see your site in first place. Why is this not a reliable measure?

Sources

Section titled “Sources”

Next

Section titled “Next”

Day 02 teaches the rules of search: spam policies, penalties, and the updates that change how Google ranks pages.