XML Sitemaps: Everything You Need to Know

Are you new to SEO or need a refresher on XML sitemaps? Well, you’ve come to the right place. In my decade of experience, I’ve seen XML Sitemaps ignored, shirked off, or straight-up disrespected. Time and time again, I find that XML sitemaps are often underutilized (or honestly, completely ignored) as part of a strong technical SEO strategy. If they’re even touched, they are often a one-time, set-it-and-forget-it type of task that flies under the radar.

That’s not how it should be. Your XML sitemap is one of your direct lines of communication to all search engines, giving them important information about where your URLs are, which ones are the most important, when they were last updated, and how frequently you are updating them. It’s important to have a strong understanding of what they are and how to maximize their use. That’s what I’m here for! Let’s learn the RIGHT way to do XML sitemaps. 

In this article, I dive into everything you need to know including:

• What they are.
• Why they’re important.
• How to create one.
• Best practices.
• Submitting it to search engines.
• Common mistakes to avoid.
• Wrap up & summary.

At its core, an XML sitemap is a simple yet powerful tool for search engines. It’s an XML file that lists all the important pages on your website, making it easier for search engines like Google to find and index them. Think of it as a detailed blueprint that tells search engines, “Hey, these are the pages that matter on my website and here’s when they were last modified! Come crawl the crap out of them…respectfully!”

While search engines are smart, they still need a little help navigating your site, especially if it’s large or has deep, complex structures or architecture. That’s where an XML sitemap comes in. It ensures that no page gets left behind (no pun intended) and that Google, and other search engines, can crawl your site more effectively and efficiently. My most important lesson learned is getting bots to your live pages as fast as possible, and XML sitemaps help that! 

To break it down:

• It’s a map: Like a traditional map guides you from one place to another, an XML sitemap guides search engines through your website.
• It’s in XML format: XML (eXtensible Markup Language) is a coding language that’s easy for search engines to read but still flexible enough for different types of websites.
• It highlights your priorities: You can tell search engines which pages are most important, helping them focus their crawling efforts where it matters most (although, because SEOs are dicks and abuse everything, priority is far less important these days). 

The best part? It’s not just for huge websites with hundreds/thousands of pages. Whether you’re running a small blog or a growing eCommerce store, an XML sitemap helps search engines find your pages, crawl them, and then (hopefully) index your content more quickly and accurately, helping to increase your chances of ranking higher. These small files are powerful and when used right, can have a large impact on your SEO performance. DON’T leave them out of your technical SEO strategy.

Now that you know what an XML sitemap is, let’s talk about why it’s a crucial component for your website’s SEO. While having an XML sitemap might seem like a small technical task, it’s a key part of ensuring your content gets the visibility it deserves in search engines. Here’s why it matters:

1. It Ensures Full Site Crawlability

Search engines like Google are constantly crawling the web, looking for new content to index and rank. However, they don’t have x-ray vision, especially when it comes to complex or very large websites. Your XML sitemap, when optimized and updated correctly, acts like a guide, helping search engines find all the important pages on your site, ensuring they aren’t overlooked or missed. Without it, search engines may struggle to crawl deeper pages, especially if you have orphan or island pages, creating a potential for valuable content to be left out.

2. It Speeds Up Indexing

Indexing is the process of adding pages to Google’s database, making them eligible to appear in search results. By submitting an XML sitemap, you’re telling search engines which pages to prioritize and crawl first. This speeds up the process of getting your content into the index and increases the chances of your pages being ranked more quickly.

3. It Highlights Your Priorities

Your website may have hundreds, or even thousands, of pages, but not all of them need to be crawled or ranked equally. With an XML sitemap, you can highlight the most important pages and help search engines focus their crawl budget on those pages that matter most. You can set priority values and define how often pages are updated, guiding search engines to put more resources into critical areas of your site.

4. It Helps Search Engines Understand Website Structure

A well-organized sitemap not only helps search engines crawl your pages but also gives them insight into your site’s structure. When your website’s architecture is easy to understand, search engines can better interpret the relationships between different pages and deliver more accurate search results. This can ultimately boost your site’s relevancy and ranking.

5. It Supports SEO for Large or Complex Sites

For websites with extensive content, like blogs, eCommerce sites, or large corporate websites, having an XML sitemap is even more essential. These sites often have complicated structures with many categories, subcategories, and internal links. An XML sitemap acts as a map that makes it easier for search engines to follow the path, ensuring they don’t miss anything important as they crawl.

6. It Helps with Dynamic Content

If your website is constantly updating with new pages or products, an XML sitemap helps search engines stay on top of these changes. When new content is added, updated, or removed, an XML sitemap lets search engines know what to re-crawl and when, ensuring your site’s index is always current.

Now that we’ve covered why XML sitemaps are important, let’s dive into how to create one. The process is relatively straightforward, but ensuring your sitemap is structured correctly and optimized for SEO is where many websites get it wrong. Below, I’ll break down the steps to generate an XML sitemap and best practices to follow.

1. Using a CMS (WordPress, Shopify, Wix, etc.)

Most modern content management systems (CMS) automatically generate an XML sitemap for you. Here’s how it works on popular platforms:

WordPress: If you’re using WordPress, an XML sitemap is automatically created (found at yourwebsite.com/wp-sitemap.xml). However, for better control and optimization, I recommend using an SEO plugin like Yoast SEO or All-in-One SEO, which allows you to manage what gets included in the sitemap and will give you better customization options.

See It In Action

You can check out mine here: https://loganmosby.com/sitemap_index.xml. This sitemap is an index, or home page, for your sitemaps that can be easily categorized by page types. Be sure to check out the settings of each plugin to see what to include and what not to include. As mentioned, I recommend having ONLY live, status 200 URLs included that are the highest priority for the site.

Shopify: Shopify automatically generates an XML sitemap (yourstore.com/sitemap.xml), covering your product pages, collections, and blogs. This sitemap will operate similarly to the /sitemap_index.xml one, providing each of the individual product, category, and page type sitemaps. While it’s hands-free, Shopify doesn’t provide a way to customize it, so you may need third-party apps if you want more control.

Wix & Squarespace: Both platforms generate sitemaps automatically (yourwebsite.com/sitemap.xml) and submit them to search engines. Customization is limited, but for most users, their default sitemap setup works fine.

2. Using Online Sitemap Generators (Manual Creation)

If your website isn’t running on a CMS that generates an XML sitemap automatically, you can use free online tools to help your generate an optimized one and implement it on your site  like:

XML-sitemaps.com: This tool will crawl your website to find as many URLs as possible and then create you an XML sitemap seamlessly from that crawl.

Screaming Frog SEO Spider: Screaming frog is one of my favorite SEO tools and a key benefit + feature is that it will also help you build out an XML sitemap from a crawl of your site. Check out that tutorial linked in the name to see how to do it!

MySitemapGenerator: This is another good tool that will do the same as these others and generate you an XML sitemap, ready to go, based on a crawl of your website.

These tools crawl your site and create an XML sitemap file that you can then upload to your website. This is useful for static sites or custom-built websites that don’t have an automatic sitemap generator.

3. Creating an XML Sitemap Manually

For developers, more advanced SEOs, or those managing custom-built websites, manually creating an XML sitemap is also an option. This could be needed if you are unable to customize a CMS-generated sitemap and need to get something cleaner and more comprehensive than what those provide. You can of course use the above tools to create a manual sitemap or set up a spreadsheet with some formulas to build it for you.

The format follows a standard structure like this:

Key elements in any XML sitemap:

loc – The URL of the page.
lastmod – The last time the page was updated.
changefreq – How often the page changes (e.g., daily, weekly, monthly).
priority – A value (0.1 to 1.0) indicating the importance of the page.

Once you’ve created the sitemap, save it as sitemap.xml and upload it to the root directory of your website (e.g., yourwebsite.com/sitemap.xml). You can access your root directory by going through hosting and using the file manager or by accessing through FTP via FileZilla or another file transfer program.

While these are key elements in an XML sitemap, I have also found that just including the URL and the last modified date works perfectly as well! Google has also mentioned that they often ignore these other elements as SEOs can’t have nice things and abuse things like the priority

Now that you know what an XML sitemap is and hot to create one, it’s time to make sure it’s optimized for search engines and is easily discoverable. A poorly structured XML sitemap can lead to massive inefficiencies in crawling and indexing, so following these best practices will ensure your sitemap helps rather than hinders your SEO efforts. I’ve encountered countless XML sitemap issues and may have forgotten a best practice or 2 here, but I’ll make sure to add as I think of them.

1. Include Only Indexable, Status 200 URLs

This is hands down the most important piece of an XML sitemap. Your XML sitemap should ONLY contain URLs that are live (status 200), canonical, and intended to rank in search engines. Avoid including:

• 404 error pages
• Redirects (301/302)
• Non-canonical URLs
• Noindexed pages

A clean sitemap ensures that Google is focusing on the right pages, not wasting crawl budget on URLs that shouldn’t be indexed.

2. Keep It Up-To-Date

A stale XML sitemap with outdated URLs can confuse search engines. If you frequently update content, add new pages, or remove old ones, make sure your sitemap reflects those changes. Most SEO plugins and tools will automatically update your sitemap for you when these changes happen, but if you’re managing one manually, update it regularly. I recommend at a minimum once per month and will build it into my workflow.

3. Keep It Under 50,000 URLs or 50MB (Or Use Index Sitemaps)

Google’s official limit for a single XML sitemap is 50,000 URLs or 50MB in file size. If your site exceeds this, you should break it into multiple sitemaps and use a sitemap index file (e.g., sitemap_index.xml) to organize them. This is especially useful for large eCommerce or media sites with thousands of pages.

4. Submit Your Sitemap to Google Search Console

Simply having a sitemap isn’t enough. You also need to make sure Google knows about it. Submitting it to Google Search Console ensures search engines are crawling it efficiently. Here’s how:

1. Log into your Google Search Console account
2. On the left-hand menu, go to Sitemaps
3. Enter your sitemap URL (e.g., yourwebsite.com/sitemap.xml)
4. Click Submit

This allows you to monitor crawl errors, indexing issues, and sitemap health. If you have an index and then separate sitemaps inside of that, I always recommend adding them all individually to ensure maximum coverage.

5. Don’t Rely Solely on Sitemaps for Crawling

An XML sitemap is helpful, but it shouldn’t be your only method for making content discoverable. Internal linking remains crucial for SEO. If a page isn’t linked anywhere on your site, search engines may struggle to determine its importance, even if it’s in the sitemap.

6. Exclude Low-Value Pages

Not every page needs to be in your sitemap. Pages like:

• Thank you pages
• Admin or login pages
• Duplicate category/tag pages
• Internal search results

Keeping your sitemap lean ensures Google prioritizes high-value content that actually drives traffic.

7. Use Dynamic Sitemaps for Large or Constantly Changing Sites

If your website frequently adds or removes pages (e.g., news sites, eCommerce stores, job boards), using a dynamically generated sitemap is a must. Many CMS platforms and SEO plugins automatically update sitemaps when new content is published, keeping search engines up to date.

So we now know what an XML sitemap is, why they are important, how to build them, and the best practices. The final step is making sure search engines can find and use it effectively. While search engines can often discover your sitemap automatically, manually submitting it ensures that they crawl it efficiently and stay updated with any changes.

1. Submitting to Google Search Console

As mentioned above, Google Search Console is the most important place to submit your XML sitemap. Here’s how:

1. Log in to Google Search Console (https://search.google.com/search-console/about)
2. Select your property (website)
3. In the left menu, click “Sitemaps”
4. In the sitemap window, a field will be at the top of the screen for you to add your sitemap (e.g., https://yourwebsite.com/sitemap.xml)
   
5. Click“Submit”

After submitting, you can monitor your sitemap status, see which pages are indexed, and check for errors.

2. Submitting to Bing Webmaster Tools

Bing also allows sitemap submissions through Bing Webmaster Tools:

1. Log in to Bing Webmaster Tools (https://www.bing.com/webmasters/about)
2. Select your website
3. Click on “Sitemaps” in the left menu
4. Enter your sitemap URL and submit

3. Using the Robots.txt File

Another way to help search engines discover your sitemap is by adding it to your robots.txt file. This file is located in the root directory of your website (yourwebsite.com/robots.txt). Simply add this line at the bottom:

Sitemap: https://yourwebsite.com/sitemap.xml

Search engines regularly crawl the robots.txt file, so this helps reinforce the location of your sitemap. You can check out my robots.txt file for an example by going here: https://loganmosby.com/robots.txt

4. Link In Your Footer

One final thing that I do, to ensure that the crawlers can find it is link the sitemap in the footer of the website along with your HTMl sitemap. This gets it linked from any page and minimizes it being missed. Scroll to the bottom of the article and check out my footer.

XML sitemaps are incredibly powerful, but if you aren’t paying attention or frequently checking their status, they can cause a lot of problems. Luckily, I’ve seen almost every XML sitemap mistake, so here are some of the most common mistakes to avoid. I’ve learned the lessons, so you don’t have to.

1. Including Non-Indexable or Broken URLs

Your XML sitemap should only contain live (status 200) URLs that you want indexed. Think of it as your direct line to get these bots to your most important, priority pages. Including 404 pages, redirects, or pages blocked by robots.txt confuses search engines and wastes crawl budget when it navigates the sitemap. Remember, the goal is to get search engines to your content as quickly and efficiently as possible. Having these types of URLs in them sends them on a wild goose chase.

Before submitting your sitemap, run a site audit to ensure all URLs are valid. If I have a current sitemap or am working on building a new one, I will export all URLs and run through a tool like Screaming Frog to get the status codes and indexability, then remove any that shouldn’t be there.

2. Overloading the Sitemap with Low-Value Pages

Not every page on your site deserves to be in the XML sitemap. Thin, duplicate, or low-priority pages dilute the importance of your key pages. Focus on high-value pages like service pages, categories, product pages, and valuable blog content that should rank.

3. Failing to Update the Sitemap Regularly

An outdated sitemap can mislead search engines, causing them to miss fresh content or waste time crawling deleted pages. If you frequently update your website, ensure your CMS or sitemap generator is automatically refreshing the lastmod field.

4. Not Submitting the Sitemap to Search Engines

While search engines may eventually discover your sitemap, relying on that alone isn’t ideal. Submitting it manually to Google Search Console and Bing Webmaster Tools ensures it’s recognized and monitored for errors.

5. Using Incorrect Formatting

A single formatting error can break your entire XML sitemap. Make sure your sitemap follows proper XML structure, uses UTF-8 encoding, and doesn’t contain typos or extra characters. You can validate your sitemap using Google’s Search Console or an XML validator tool.

6. Exceeding the Sitemap Size Limit

Google limits XML sitemaps to 50,000 URLs or 50MB uncompressed. If your site is massive, use a sitemap index file to split large sitemaps into multiple sections, making it easier for search engines to crawl efficiently.

7. Ignoring Sitemap Errors in Search Console

If Google flags errors in your sitemap, don’t ignore them. Issues like “submitted URL blocked by robots.txt” or “couldn’t fetch” indicate problems that could prevent indexing. Regularly check Search Console and fix any reported errors.

A well-optimized XML sitemap is one of the simplest yet most effective ways to improve your website’s crawlability and indexing. When done right, it helps search engines find your most important pages quickly, ensuring they get the attention they deserve.

Let’s recap the key takeaways:

• An XML sitemap is your website’s blueprint for search engines, helping them prioritize and index the right pages.
• Only include live, high-priority URLs and remove any broken links, redirects, or low-value pages that don’t need to be indexed.
• Keep it clean and up to date. Regularly audit your sitemap to ensure it reflects your current website structure.
• Submit it to Google Search Console & Bing Webmaster Tools. Don’t wait for search engines to find it on their own.
• Avoid common mistakes like incorrect formatting, excessive low-value URLs, and sitemap errors can hurt your SEO.

If you haven’t checked your XML sitemap recently, now’s the time. Run a quick audit, clean up any issues, and make sure search engines have a clear path to your most important content. Need help optimizing your sitemap and overall SEO strategy? Let’s talk.