XML Sitemaps Done Right: What to Include (and Leave Out)

XML Sitemaps Done Right: What to Include (and Leave Out)

ByAdam


A clean XML sitemap lists only canonical, indexable URLs. Learn what to include, what to exclude, and how to submit and monitor it in Search Console.

An XML sitemap is a list of URLs you want Google to find and consider for indexing. That definition sounds simple, but most automatically generated sitemaps include URLs that shouldn’t be there — noindexed pages, parameter variants, redirecting URLs, paginated pages, and thin content — which dilutes the signal the sitemap is supposed to send and can slow down how efficiently Google processes the pages that matter.

A well-maintained sitemap lists only the canonical, indexable URLs you actually want in the index. Nothing else.

The Short Version

  • Only canonical, indexable, 200-status URLs belong in the sitemap — no noindexed pages, redirects, or parameter URLs.
  • A single sitemap can hold up to 50,000 URLs / 50MB uncompressed; larger sites need a sitemap index with separate child sitemaps per content type.
  • Submitting a sitemap doesn’t guarantee indexing — it’s a discovery signal, not a quality override.
  • Plugin and CMS updates are the most common cause of a previously clean sitemap silently filling with junk URLs — recheck it after major updates.

What to include in an XML sitemap

  • Published, indexable pages: Pages with no noindex tag, no canonical pointing elsewhere, and a 200 status code. These are the only URLs that should be in the sitemap.
  • The canonical version of each URL: If a page exists at both www and non-www, or with and without a trailing slash, include only the canonical version — the one the canonical tag points to.
  • Your most important pages first: While sitemap order doesn’t directly determine crawl priority, keeping the sitemap clean and well-organised (pillars, service pages, and high-value content near the top) reflects your own understanding of site structure.

What to leave out

This is where most auto-generated sitemaps go wrong:

URL typeWhy it doesn’t belong
Noindexed pagesCreates a contradiction — asking Google to visit a page while telling it not to index it; wastes crawl budget
Redirecting URLsOnly the final destination should be listed; Google should only be directed to live, canonical URLs
Parameter and filter URLsFaceted navigation, sort parameters, session IDs, and tracking parameters aren’t canonical pages
Paginated pages (most cases)Unless a page has meaningful standalone content, keep pagination crawlable via internal links but out of the sitemap
Tag and archive pagesWordPress tag, date, and author archives are often thin-content pages that shouldn’t be indexed at all
Staging or development URLsA sign the sitemap was generated on the wrong environment and imported without checking

Sitemap structure: when to use multiple sitemaps

A single XML sitemap can contain up to 50,000 URLs (and should be under 50MB uncompressed). For most sites, one sitemap is sufficient. For large sites, a sitemap index file that references multiple child sitemaps is the right structure — one sitemap for posts, one for pages, one for products, one for categories. This makes it easier to monitor crawl status for each content type in Search Console separately.

Useful sitemap additions: <lastmod> — the date the page was last meaningfully updated. Google uses this to prioritise crawling recently updated content. Only include it if you’re actually updating it correctly; a static lastmod date from years ago that never changes is worse than no lastmod at all.

Sitemap limits and what they mean in practice

Sitemap reference numbers

50,000
Max URLs per sitemap file
Beyond this, split into a sitemap index referencing multiple child sitemaps.
50MB
Max uncompressed file size
Most sites never approach this — the 50,000 URL cap is usually the binding constraint first.
Days–1wk
Typical re-fetch cadence
How often Google re-reads a submitted sitemap on active sites, varying with authority and crawl rate.

Submitting and monitoring in Search Console

Submit your sitemap in Search Console → Indexing → Sitemaps → Add a new sitemap. The URL is typically yourdomain.com/sitemap.xml or yourdomain.com/sitemap_index.xml. After submission, Search Console shows the number of URLs submitted vs the number discovered (which may differ if you’re using a sitemap index). The “Couldn’t fetch” error means Google can’t access the sitemap URL — check it’s publicly accessible and returns a 200 status.

Monitor the sitemap periodically: if the submitted count grows significantly, parameter or archive URLs may have been added by a plugin update. Keep it clean — a sitemap that’s a reliable list of your best content is more useful to Google than one that’s a complete dump of every URL on the site.

For the broader technical SEO audit framework, see the SEO audit checklist.

A worked example: a plugin update that broke a sitemap

A WordPress site’s submitted sitemap count jumped from 340 URLs to 2,100 URLs overnight, with no corresponding content launch. Search Console’s Sitemaps report showed the increase but not the cause. Opening the sitemap XML directly in a browser showed the new entries were all /tag/ archive URLs — a plugin update had silently re-enabled tag archive indexing and added them back into the sitemap, undoing a noindex configuration set up a year earlier.

The fix took fifteen minutes: re-applying the noindex rule to tag archives in the SEO plugin’s settings and excluding the /tag/ path from the sitemap generator. Within a week, the submitted count returned to 340, and the “Discovered, currently not indexed” count in Search Console — which had also spiked — dropped back down as Google stopped finding new thin archive pages to flag. The broader lesson: any major plugin or CMS update is worth a quick sitemap-count sanity check before assuming nothing changed.

Frequently asked questions

No — a sitemap is a request, not a command. Google uses the sitemap to discover URLs and prioritise crawling, but indexation decisions are made based on Google’s assessment of each page’s quality, relevance, and uniqueness. A page submitted in the sitemap that Google considers thin, duplicate, or low-quality will be listed as “Crawled, currently not indexed” in Search Console — Google visited it but chose not to index it. Fixing the content quality or uniqueness issue is what gets it indexed, not resubmitting the sitemap.

Technically, Google can discover pages through crawling internal links without a sitemap. For small sites (under 100 pages) with strong internal linking, a sitemap provides minimal additional benefit. However, it’s still useful for: signalling which version of a URL is canonical when you have www/non-www or trailing slash variations; getting new content indexed faster (Google checks submitted sitemaps regularly); and monitoring indexation coverage in Search Console. The effort to set one up is low, so it’s worth doing even on small sites — just make sure it stays clean.

Google re-fetches submitted sitemaps regularly — typically every few days to a week for active sites, though this varies based on site authority and crawl rate. When you publish new content, you don’t need to manually resubmit the sitemap; Google will pick it up on its regular schedule. If you need faster indexation of a specific new page, use the URL Inspection tool in Search Console → Request Indexing — this queues the individual URL for crawling independently of the sitemap schedule.

Image and video sitemaps are supplementary — they help Google discover media files and associated metadata (captions, titles, licences) that might not be obvious from crawling the page HTML alone. For most content-focused sites, they’re not necessary. They become more useful when: you have large libraries of original images you want indexed in Google Images; you’re hosting videos directly on your site (not on YouTube) and want them to appear in video search results; or your media is loaded dynamically and may not be discovered through normal page crawling. If you’re running a standard blog or service site, prioritise getting your page sitemap right before worrying about media sitemaps.

Open the sitemap URL directly in a browser — most CMS-generated sitemaps render as readable XML or an HTML table. For a full audit, download it and run it through a crawler like Screaming Frog’s list mode, which will check each URL’s status code, indexability, and canonical tag in bulk and flag anything that shouldn’t be there (noindexed pages, redirects, non-canonical URLs). This is worth doing quarterly, not just after something looks wrong in Search Console.

A sitemap lists individual page URLs directly. A sitemap index is a sitemap of sitemaps — it lists the locations of multiple child sitemap files rather than pages themselves, used when a site exceeds the 50,000-URL or 50MB single-file limit, or simply to keep content types organized separately (posts, pages, products). You submit the sitemap index URL to Search Console, and Google follows it to find and crawl each child sitemap automatically.

A sitemap is only as useful as it is accurate

An auto-generated sitemap that includes every URL on the site — noindexed pages, redirects, parameter variations — is noisier than no sitemap at all. A curated sitemap that lists only the canonical, indexable URLs you want Google to find is a useful crawl signal. The difference is whether you’ve thought about what belongs there.

If you’d like your sitemap and indexation coverage reviewed as part of a broader technical audit, get in touch.

Similar Posts