Skip to content

Indexed, though blocked by robots.txt

Google indexed the URL without crawling it, usually because other pages link to it. If you want it hidden, remove the robots.txt block and add noindex. If you want it in search, remove the block.

Why blocking was not enough

Robots.txt stops crawling, not indexing. A blocked URL can still be indexed from links pointing at it.

Diagnose it

Symptoms, checks and fixes

Common symptoms

  • A warning in the Page indexing report for "Indexed, though blocked by robots.txt".
  • Results show the URL with little or no description.

Checks to run

Diagnostic checks
CheckHow to tellFix
Should the page be in Google?Decide whether this URL is meant for search visitors.If yes, remove the robots.txt rule so Google can read the page.[1]
If not, can Google see a noindex?Check whether robots.txt blocks the URL. A blocked page hides its noindex from Google.Remove the robots.txt block and add noindex, so Google crawls the page and drops it.[2][1]
FAQ

Frequently asked questions

Why does the result show no description?

Because Google could not crawl the page, so it indexed the address without reading its content.[3]

How we work together

Two ways to get started

Website Subscription

Your website, fully managed — no big upfront cost.

We design, build, host, maintain and keep improving your site for one simple monthly fee.

  • Custom-designed & built for your brand
  • Hosting, security & maintenance included
  • Ongoing edits & content updates
  • Continuous SEO & performance improvements
  • Scale up or down as you grow

Fixed Project

A one-off custom build you own outright.

A defined scope and a single fixed quote — delivered once, and yours to keep.

  • Bespoke design & development
  • Defined scope & timeline
  • You own the website & code
  • Optional care plan after launch
  • Custom-quoted to your needs

Ready to get started?

Get a free, no-obligation quote for your project. We respond within 24 hours.