When crawl budget matters
Google's crawl budget guide is written for very large sites and for sites with many URLs in this status. Smaller sites usually see it clear as Google gets to the pages.
Google found the URL but has not crawled it yet. It postponed the crawl to avoid overloading your site, so the last crawl date is empty. Server health, duplicate URLs and internal links all affect when it comes back.
Google's crawl budget guide is written for very large sites and for sites with many URLs in this status. Smaller sites usually see it clear as Google gets to the pages.
| Check | How to tell | Fix |
|---|---|---|
| Is the server slow or erroring? | Open Crawl Stats and look for server errors, 429 responses or slow response times. | Fix hosting capacity, because Google crawls less when a site returns 5xx or 429.[1][2] |
| Are there many duplicate or low-value URLs? | Look for parameter, sorting, filter or tag URLs that repeat the same content. | Consolidate duplicates with canonicals and stop generating URLs you do not want crawled.[2][3] |
| Is the page well linked? | Check whether the page is reachable from relevant pages, not just the sitemap. | Add internal links from related pages.[4] |
Because Google has not crawled the URL yet. It rescheduled the crawl to avoid overloading the site.[5]
Your website, fully managed — no big upfront cost.
We design, build, host, maintain and keep improving your site for one simple monthly fee.
A one-off custom build you own outright.
A defined scope and a single fixed quote — delivered once, and yours to keep.
Get a free, no-obligation quote for your project. We respond within 24 hours.