Pages stuck in Crawled - currently not indexed. Discovered - currently not indexed creeping up. URLs that used to rank quietly sliding out of the index. If that sounds familiar, you likely have a few template-level signals crossed. Use this no-nonsense checklist to pinpoint the cause, fix it once at the source, and get Google to revisit your pages fast.
Spot the problem quickly
Start in Google Search Console. Open Indexing → Pages and switch the view to “Why pages aren’t indexed.” Look for clusters, not single URLs. If 200 product pages under /collections/shoes/ show the same status, you have a template or rule to fix, not 200 one-offs.
- Crawled - currently not indexed: Google saw the URL but chose not to index it. Usual causes are thin content, duplicates, or soft 404 templates.
- Discovered - currently not indexed: Google knows the URL exists but has not crawled it. Crawl budget is usually burned on parameters, infinite filters, or weak internal links.
- Alternate page with proper canonical: Multiple URLs show the same content and signals are inconsistent. Canonical, redirects, internal links, and sitemaps disagree.
- Blocked by robots.txt: A rule is preventing crawling of pages that should be indexable.
- Excluded by ‘noindex’: A meta robots or x-robots-tag is set where it should not be.
Export a sample from each status. Note path patterns, parameters, and templates. Fix the pattern that impacts the most pages first.
The fast technical SEO checklist
-
Verify robots.txt, sitemaps, and canonicals. These three set your ground truth. If they disagree, indexing stalls.
- Robots.txt: Confirm /robots.txt serves 200. A safe default looks like:
User-agent: *
Disallow: - Watch for accidental broad rules. Disallow: /, Disallow: /*?*, or Disallow: /search/ can wipe out large sections if your nav or filters live there. If you must block junk parameters, be surgical, for example Disallow: /*?sessionid=
- XML Sitemaps: Serve a sitemap index at /sitemap.xml or /sitemap_index.xml. Each child sitemap should return 200, reference only canonical 200 URLs, and include a recent lastmod. Remove any 3xx, 4xx, 5xx, noindex, paginated pages past page 1, and URLs with tracking parameters.
- Canonical tags: Use absolute, self-referencing canonicals on canonical pages, for example: <link rel="canonical" href="https://www.example.com/widgets/blue/">. Make sure the canonical URL also appears in your sitemap and matches your internal links. Avoid cross-domain canonicals unless consolidation is intentional and permanent.
- One primary version: Enforce https, pick www or non-www, and choose a trailing-slash style. Redirect all variants to the primary with a single 301 hop. Fix mixed internal links that still point to the wrong protocol or host.
- Robots.txt: Confirm /robots.txt serves 200. A safe default looks like:
-
Audit crawl traps and messy parameters. Endless variations waste crawl budget and delay the good stuff.
- Find traps: Common culprits are sort and filter facets (?color=, ?size=, ?sort=), calendar pages with infinite next links, session IDs, replytocom, and search results pages. Crawl a large category and compare unique URLs found vs actual items. If a 100-product category yields 10,000 unique URLs, you have a trap.
- Contain variants: Default to clean, descriptive paths for canonical URLs. Keep only one URL per content item. For unavoidable parameters, point all variants to the canonical with a rel="canonical", and exclude variants from sitemaps. Link internally only to the canonical form.
- Robots rules with care: It is fine to block obviously useless parameters like ?sessionid= or ?replytocom=. Do not block parameter pages that should consolidate via canonical or 301, or Google may never see the canonical hint.
- Navigation hygiene: Strip tracking parameters from all internal links, menus, breadcrumbs, and pagination. Make sure category filters have a crawlable path to at least one indexable version of each item.
-
Find and fix soft 404s. A page can return 200 but still look like a 404 to Google if it is empty or off-topic.
- Identify them: In Search Console, open the Soft 404 report. Check templates for out-of-stock products, empty categories, thin tag pages, search results, and expired events. Render the page and ask: would a user get value here?
- Return the right code: Use 404 or 410 for content that is permanently gone and has no close replacement. If an item is temporarily unavailable, keep 200 and add clear alternatives and internal links to related items.
- Strengthen thin pages: For empty category pages, add unique copy that explains the category, featured subcategories, and 6 to 12 handpicked internal links. Do not index filter permutations that have near-zero value.
-
De-duplicate content and consolidate signals. Tell Google exactly which URL is the keeper.
- Protocol and host: Force https and your chosen host with 301s. Remove any http links in templates, sitemaps, hreflang, and canonicals.
- Trailing slash and case: Pick a standard and redirect the other form. Avoid uppercase or mixed-case paths if your server treats case as unique.
- UTMs and tracking: Never include tracking parameters in internal links or sitemaps. If external links include UTMs, keep the page accessible but canonical to the clean URL.
- Legacy templates: Retire print, AMP, and staging duplicates. If a legacy version must exist for users, add noindex and a canonical to the main URL.
-
Repair orphan pages with internal links. If you do not link to a page, Google will not prioritize it.
- Find orphans: Combine a fresh crawl with your sitemap and Search Console export. Any URL in sitemap or GSC that has zero internal inlinks in your crawl is an orphan candidate.
- Add links where it counts: From relevant category hubs, high-traffic guides, and related evergreen posts, add 2 to 5 contextual links using descriptive anchors. Place at least one link high on the page. Add the page to a logical nav path or breadcrumbs.
- Keep it maintained: When you publish new pages, add them to hubs immediately. Review clusters quarterly to keep the internal link graph current.
-
Fix status codes and conflicting signals. One page, one message.
- Serve accurate codes: Do not return 200 for gone content. Use 404 or 410. If there is a successor, 301 to it. Avoid 302s for permanent moves.
- Align all hints: The canonical tag, internal links, sitemap, hreflang, and redirects should all point to the same URL. Remove noindex from any page you expect to rank. Check for x-robots-tag noindex in HTTP headers on files like PDFs.
- Rendering: If key content is injected via JavaScript, make sure the rendered HTML contains the text, links, and canonical. Pre-render or server render important templates so Googlebot gets the same content users see.
- Pagination: Keep only page 1 in sitemaps. Make each paginated URL canonical to itself unless it is a true duplicate of page 1. Provide clear next and previous links in the UI and link to a view-all page if performance allows.
-
Revalidate in Search Console. Nudge Google to see the fixes.
- Inspect a sample: Use URL Inspection on representative URLs from each fixed template. View the rendered HTML to confirm content and canonical are correct. Request indexing for high-priority pages.
- Resubmit sitemaps: Remove obsolete files. Submit the correct sitemap index. Watch for processing errors and last read timestamps.
- Check Crawl stats: In Settings → Crawl stats, verify Googlebot is hitting clean, canonical paths more often and that 5xx errors are not spiking.
Revalidate and monitor results
Indexing recovery is a process. Start with the fix that affects the largest set of URLs, then work down. In the Pages report, you want to see Crawled - currently not indexed and Discovered - currently not indexed decline while Indexed rises. Alternate page with proper canonical should flatten once your signals align.
Set a quick baseline: count affected URLs per status, note sitemap URL counts and lastmod freshness, record 5xx and 4xx rates, and capture a one-day slice of server logs to see how much crawl is going to parameters vs clean URLs. Recheck 7, 14, and 21 days after changes. For small sites, key pages often update within 24 to 72 hours. Large sites and big template shifts can take 1 to 3 weeks to settle.
Keep an eye on templates users actually see. If the rendered HTML in URL Inspection is missing main content, fix rendering. If Google is spending a large share of crawl on junk parameters, tighten robots rules and internal linking. If soft 404s persist, improve the template or return the right code.
Close the loop with automation
If this checklist keeps landing on your calendar, hand the recurring work to a tool that does it every week and fixes what it can. RankGoat runs scheduled indexation checks, validates sitemaps, robots, canonicals, and redirects, and flags or fixes template-level issues so pages get indexed. Its technical SEO cleanup covers orphan-page repair, soft 404 audits, duplicate content triage, and crawl, parameter, and pagination handling.
Beyond indexing, RankGoat is a done-for-you blog writing service. It plans topics around your money pages, writes and auto-publishes posts with clean on-page SEO and schema, and adds a contextual dofollow backlink from a vetted partner site for each post. Daily link monitoring replaces dropped or nofollowed placements. Its AI search optimization tracks AI answer visibility and applies Generative Engine Optimization with small on-page tweaks to improve inclusion in AI answers across engines.
Thinking about cost and time saved. If you are weighing RankGoat pricing, put a number on the hours you spend chasing indexing issues and cleaning templates. For many teams, automating the weekly checks and cleanup pays back fast, and the steady publishing and link support helps new pages earn their place once they are in the index.
Key takeaways
- Fix template-level signals first. Align robots, sitemaps, canonicals, and redirects.
- Contain crawl traps and parameters. Only the clean, canonical version should be indexable.
- Resolve soft 404s and duplicates. Return the right status codes and consolidate to one URL.
- Give every important page internal links from relevant hubs. Remove or noindex low-value variants.
- Revalidate fixes in Search Console and watch coverage trends for 1 to 3 weeks.
- Automate recurring checks with RankGoat so indexing issues do not pile up again.
Work the checklist from the top, validate each fix, and watch your coverage stabilize. Clean signals plus consistent internal links get good pages indexed. Keeping that clean makes growth predictable.
FAQ
How long does it take for indexing fixes to show results?
Most template-level fixes start to show in Google Search Console within 1 to 3 weeks. Request indexing for priority URLs to speed up recrawls.
What is a soft 404 and how do I fix it?
A soft 404 is a page that returns 200 but looks empty or irrelevant. Serve a proper 404 or 410 for gone pages, or improve thin content so it clearly satisfies the intent.
Should I block parameter URLs with robots.txt?
Block useless parameters like session IDs, but prefer canonical tags and clean internal links for variants you want to consolidate. Do not block pages that need to pass signals.
Why are my pages discovered but not indexed?
Common reasons include weak or duplicate content, conflicting canonical or noindex tags, crawl traps wasting budget, or orphan pages with no internal links. Fix signals and add links.
Can RankGoat help with recurring indexing issues?
Yes. RankGoat runs weekly indexation checks, validates sitemaps, robots, and canonicals, and handles technical SEO cleanup like soft 404s, duplicates, and orphan pages.