Indexing Problems: Why Pages Stay Out
Pages stay unindexed for a handful of reasons: low quality, a blocked crawl path, a wrong canonical, or Google simply choosing not to bother yet.
Pages stay unindexed for one of four reasons: a crawler cannot reach them, a directive tells search engines not to index them, Google judges the content too thin or too duplicate to bother with, or the page is simply still waiting in a crawl queue behind others. Diagnosing which one it is decides the fix.
Indexing sits near the top of the technical SEO checklist, because nothing else on that list matters for a page search engines have not stored yet.
On this page
Where do I find out why a specific page isn't indexed?
Start with the URL Inspection tool in Search Console, not a guess. Paste the URL and it reports whether Google has it indexed, and if not, which specific status applies [2]. That status is the whole diagnosis; everything after this section is what to do for each one.
| Status | What it usually means | Where to look first |
|---|---|---|
| Discovered - currently not indexed | Known but not yet crawled; a queue problem | Internal links, sitemap priority, crawl budget |
| Crawled - currently not indexed | Crawled and judged not worth indexing | Content quality, duplication, thin pages |
| Excluded by 'noindex' tag | A directive is telling Google not to index it | Check the tag was intentional |
| Blocked by robots.txt | Cannot be crawled at all | robots.txt rules |
| Duplicate without user-selected canonical | Google picked a different page as canonical | Canonical tags and near-duplicate content |
Why does Google discover a page but never crawl it?
Usually crawl priority, not a block. Google allocates more frequent, deeper crawling to sections it judges more valuable, and a page sitting several clicks deep with few internal links pointing at it competes for that attention against pages that are easier to reach and more often updated.
Check how many clicks deep the page sits
A page four or five clicks from the homepage with no other links in is easy for a crawler to deprioritize.
Add internal links from pages that already get crawled often
A link from a frequently updated hub page is a stronger discovery signal than a sitemap entry alone.
Check the sitemap priority and lastmod fields
These are hints, not commands, but a sitemap with no real lastmod values gives Google less reason to revisit it.
Look for a crawl budget problem on a large site
Thousands of low-value URLs, like faceted filters, can eat the budget that would otherwise reach the pages you actually want indexed.
Why would Google crawl a page and still not index it?
Because it read the page and judged it not worth storing. This status means the technical path worked; the content did not clear the bar. Common causes are a page that duplicates another almost word for word, a page too thin to answer anything on its own, or a page assembled from boilerplate with little unique content once the template is stripped away.
There is no technical fix for this one. The page needs to become more useful, more specific, or it needs to be merged into a page that already covers the same ground well, with a redirect from the weaker version. Chasing this status with a "request indexing" click repeats the same judgment and gets the same answer.
Programmatic pages are where this shows up most often. A site that generates a page for every combination of two filters, or every city and service pair with no unique content behind it, produces hundreds of pages that read as the same page to Google with a few words swapped. The fix is rarely to improve every page individually; it is usually to noindex the combinations that add no real value and invest the effort in the smaller set that can actually stand on its own with real detail, real examples and content a competitor's page does not already say in the same order.
A second common cause is content copied or lightly rewritten from elsewhere, including your own other pages. If a service page and a blog post both explain the same concept in the same structure, Google may treat one as redundant with the other. Differentiate them by purpose: the service page sells the work, the blog post teaches the concept, and each one should read like it exists for a different reason, not like two drafts of the same page.
Could a technical block be the real cause?
Check this before assuming a quality problem. A robots.txt disallow, an accidental noindex left over from staging, a canonical pointing at a different page, or a server error on the specific crawl attempt can all produce a status that looks like a content judgment but is actually a switch left in the wrong position.
This is especially common right after a site migration, when a staging robots.txt or a template-wide noindex sometimes ships to production by accident. If your indexing numbers dropped sharply right after a deploy, check the deploy before you check the content.
Rendering is worth ruling out too. If a page's main content only exists after a script runs, Google may crawl an empty shell on the first pass and form a low-quality judgment from that, even though the fully rendered page looks fine to a visitor. Our guide to JavaScript SEO covers how to check what the first response actually contains.
How do I fix duplicate or near-duplicate content?
Pick one version and make every signal agree with it. If two pages cover the same topic, choose the stronger one, set its canonical to itself, redirect or noindex the weaker one, and update internal links across the site to point at the survivor rather than the page you are retiring.
Google's own note on this status is direct: when it finds several similar pages, it picks what it considers the best representative, and that choice may not match the canonical you set if the pages are similar enough [1]. The fix is to make the pages different enough that the choice is not ambiguous, or to merge them so there is only one to choose.
What should I actually audit, in what order?
Work from the cheapest checks to the most expensive ones. Confirm nothing is blocked before you investigate content quality, because a five-minute robots.txt check can save days spent rewriting a page that was never the real problem.
Pull the full list from the Page indexing report
Export every status, not just the total count.
Separate technical statuses from quality statuses
Blocked and noindexed pages go in one pile; crawled-not-indexed goes in another.
Fix every technical block first
These are usually quick and unambiguous once found.
Triage the quality pile by traffic potential
Rewrite or merge the pages worth saving; retire the rest with a redirect.
Re-check the report after two to three weeks
Indexing status updates on Google's own schedule, not instantly after a fix.
If the list is long enough that triage itself feels like the project, that is exactly the kind of prioritized work our technical SEO audit produces: a ranked list, not just a longer spreadsheet of the same problem.
Should I use "request indexing" while I wait?
Use it to confirm a fix, not to force a decision that has not changed. Once you have actually corrected the cause — removed a block, merged a duplicate, rewritten a thin page — requesting indexing on that specific URL is a reasonable way to see the new judgment sooner than the next scheduled crawl would bring it.
Running it across dozens of URLs a day without having changed anything about them is a different action entirely. It does not speed up a decision Google has already made; it just spends a limited daily allowance asking the same question again. Save it for URLs where something has genuinely changed since the last crawl.
What does a healthy indexing rate actually look like?
There is no single target percentage, because it depends heavily on how many low-value URLs a site generates on its own, such as filtered listings or tag pages nobody intended to compete for rankings. A smaller, well-curated site can reasonably expect most of its real content to be indexed; a large site with heavy faceted navigation will always show a meaningful share of "excluded" URLs, and that is not automatically a problem if those URLs were never meant to rank.
The number worth watching over time is not the raw percentage but the trend among the pages you actually care about: your core service pages, your best content, your highest-intent pages. If those specific URLs are consistently indexed and stay indexed after each deploy, the site's overall exclusion rate from low-value URLs is a much smaller concern than it looks on a dashboard.
Questions people also ask
What does 'Discovered - currently not indexed' actually mean?
Google knows the URL exists, usually from a sitemap or a link, but has not crawled it yet. This is most often a crawl-priority problem: the page is queued behind others Google judges more worth visiting first, not blocked outright.
What does 'Crawled - currently not indexed' mean, and is it different?
Yes. Google crawled the page and chose not to index it, which is almost always a quality or duplication judgment rather than a technical block. Fixing this means improving the page, not fixing an access problem.
Does requesting indexing in Search Console actually help?
It can nudge a single URL into the queue sooner, but it does not override a quality decision, and using it on many URLs a day is rate-limited. It is a tool for confirming a fix on one page, not a substitute for fixing the underlying cause across many.
How long should I wait before worrying about an unindexed page?
A few days for a small site with strong existing authority, up to several weeks for a new site or a low-authority section. If a page is still unindexed after a month and you have ruled out the technical causes, treat it as a quality problem.
Can a robots.txt disallow and a noindex tag both be on the same page?
They can, but it defeats the noindex tag. If robots.txt blocks the page, Google cannot crawl it to see the noindex instruction, so the URL can still appear in results with no snippet, which is the opposite of what a noindex tag is meant to do.
Sources
- Page indexing report — Search Console Help (checked 22 September 2026)
- URL Inspection tool — Search Console Help (checked 22 September 2026)
- Ask Google to recrawl your URLs — Google Search Central (checked 22 September 2026)
Got a stack of unindexed URLs and no time to triage them?
Book a free 30-minute consultation with Ibrahem and Jeremy. We look at your site while we talk, and you leave with a prioritised plan.
Who wrote this
Ibrahem HelalCo-founder
Leads strategy and delivery.
Works on: SEO, GEO
Reviewed before publishing by Jeremy Zava, Co-founder.
- Technical SEO: the complete checklist for 2026Technical SEO makes sure search engines and AI crawlers can reach, render and index your pages. This checklist covers every layer, in the order to fix it.
- Hreflang Errors: How to Find and Fix ThemHreflang errors mostly come from missing return tags or wrong language codes. Here is how to find every broken pair and fix them without breaking more.
- Site Migration SEO: The Full ChecklistA site migration loses traffic when redirects, canonicals or content go missing. This checklist covers the mapping, the launch and the four weeks after.
- SEO Audit Template: What to Check and HowA working SEO audit template covering access, indexing, content, links and speed, with what to check in each layer and how to tell pass from fail.