Why blocking was not enough
Robots.txt stops crawling, not indexing. A blocked URL can still be indexed from links pointing at it.
Google indexed the URL without crawling it, usually because other pages link to it. If you want it hidden, remove the robots.txt block and add noindex. If you want it in search, remove the block.
Robots.txt stops crawling, not indexing. A blocked URL can still be indexed from links pointing at it.
| Check | How to tell | Fix |
|---|---|---|
| Should the page be in Google? | Decide whether this URL is meant for search visitors. | If yes, remove the robots.txt rule so Google can read the page.[1] |
| If not, can Google see a noindex? | Check whether robots.txt blocks the URL. A blocked page hides its noindex from Google. | Remove the robots.txt block and add noindex, so Google crawls the page and drops it.[2][1] |
Because Google could not crawl the page, so it indexed the address without reading its content.[3]
Your website, fully managed — no big upfront cost.
We design, build, host, maintain and keep improving your site for one simple monthly fee.
A one-off custom build you own outright.
A defined scope and a single fixed quote — delivered once, and yours to keep.
Get a free, no-obligation quote for your project. We respond within 24 hours.