Jason Barnard Hosts Fabrice Canel and Dawn Anderson for Indexing and Crawling: What You Should Know - Webinar September 2021

ByBernadeth Brusola

Webinar: Indexing and Crawling: What You Should Know with Fabrice Canel, Dawn Anderson, and Jason Barnard

In this foundational discussion, Jason Barnard brought together leading technical SEO experts to clarify how crawling and indexing actually work in search engines and why being indexed is only the first step toward organic visibility.

September 23, 2021 - Jason Barnard hosted a webinar featuring Fabrice Canel, Principal Product Manager at Microsoft Bing, and Dawn Anderson, internationally recognised technical SEO consultant and lecturer. Published by Duda, the session explored how search engines discover, crawl, render, and index web pages, and why indexing alone does not guarantee rankings or traffic.

The discussion focused on practical insights into search engine behaviour, helping website owners understand what is required for content to become truly discoverable and competitive in organic search.

Indexing Is Not the Same as Ranking

A core message throughout the webinar was the clear distinction between indexing and ranking.

Fabrice Canel explained that while indexing is a necessary step for visibility in search engines, it does not guarantee that a page will perform well in rankings or attract organic traffic.

The speakers reinforced a simple but critical principle:

If a page is not indexed, it cannot rank. But being indexed does not automatically mean it will rank.

This distinction helped frame the importance of ensuring content is not only published but also properly understood and processed by search systems.

You Must Be in the Index to Be in the Game

The discussion emphasised a foundational SEO principle often summarised as:

You have to be in it to win it.

Jason Barnard highlighted that the first and most essential step in SEO is ensuring that pages are discoverable, crawlable, and indexable. Without these conditions, even the highest quality content will remain invisible to search engines.

This led into a broader explanation of how search engines process the web before ranking content.

How Search Engines Discover and Process Content

Dawn Anderson provided a technical breakdown of how search engines operate, focusing on the journey from discovery to indexing.

The process typically includes:

  • Discovery of URLs through links and sitemaps
  • Crawling of page content
  • Processing and rendering of page elements
  • Evaluation of content relevance and structure
  • Inclusion in the search index

She emphasised that each stage introduces potential barriers that can prevent content from progressing to the next step.

The Role of Crawling in SEO Performance

A significant portion of the webinar focused on crawling efficiency and how it affects indexation outcomes.

The speakers discussed how search engines allocate crawl resources based on site structure, authority, and perceived value. Pages that are difficult to access or poorly linked internally may be crawled less frequently or missed entirely.

Key factors influencing crawling include:

  • Internal linking structure
  • Server performance and response times
  • URL clarity and consistency
  • Duplicate or thin content issues

Improving crawlability ensures that search engines can consistently access and evaluate important pages.

Indexing Challenges and Common Misconceptions

The webinar also addressed common misunderstandings about indexing.

Fabrice Canel clarified that indexing is not a simple binary outcome. Instead, search engines may partially index content, prioritise certain sections, or choose not to index pages that do not meet quality or relevance thresholds.

The discussion highlighted that:

  • Not all crawled pages are indexed
  • Not all indexed pages are equally valued
  • Indexing decisions are influenced by quality and relevance signals

This helped shift the conversation from a technical checklist mindset to a more holistic understanding of search engine evaluation.

Why Crawl Budget Matters for Larger Websites

The speakers also explored the concept of crawl budget, particularly for large websites with thousands or millions of URLs.

Crawl budget refers to the number of pages a search engine is willing and able to crawl on a site within a given timeframe.

If a site wastes crawl budget on low-value pages, important content may be discovered and indexed more slowly.

Optimising crawl efficiency involves:

  • Removing or consolidating low-value pages
  • Improving internal linking to priority content
  • Ensuring clean URL structures
  • Reducing unnecessary redirects and duplication

The Relationship Between Indexing and Organic Traffic

Jason Barnard brought the discussion back to its strategic implication for businesses and marketers.

While indexing is a technical requirement, it is only the starting point. Real organic performance depends on whether indexed pages are relevant, authoritative, and aligned with user intent.

The webinar reinforced that SEO success requires both:

  • Technical accessibility (crawlability and indexing)
  • Content quality and relevance (ranking potential)

Final Takeaway: Visibility Requires Full Accessibility

The closing message of the webinar was clear and practical.

To achieve organic visibility, websites must ensure that their content is not only published but fully accessible to search engines at every stage: discovery, crawling, rendering, and indexing.

Without this foundation, even strong content strategies will fail to generate meaningful search traffic.

Watch the Full Webinar

Indexing and Crawling: What You Should Know Webinar

Published by: Duda. Guests: Fabrice Canel, Dawn Anderson. Host: Jason Barnard. September 23, 2021.

Similar Posts