Your site has a sitemap.xml, but Google crawls the wrong pages and ignores the ones that matter. The problem isn't the sitemap's presence — it's how you built it. Priority and change frequency aren't decorative fields: they're signals that guide crawlers toward what truly matters. We at Meteora Web see it every day in projects that come to us: default sitemaps with all priorities set to 0.5 and made-up frequencies. The result? Crawl budget wasted on service pages and old products, while the pages that drive revenue wait weeks to get indexed.
Why do priority and frequency really matter for crawling?
Google's crawl budget is a limited resource. If your site has 10,000 URLs but only 500 generate contacts or sales, you can't afford the crawler wasting time on the rest. The XML sitemap is your tool to tell Google which URLs matter and how often they change. It's not a direct command — Google can ignore your suggestions — but it's a strong signal that guides crawler decisions.
Think of it as a restaurant menu: if you list everything without distinction, the customer (Google) orders randomly. If you highlight the daily specials, they order those. Priority in the sitemap is the equivalent of the recommended dish: it signals what deserves attention. Frequency tells the crawler how often to return for updates. A product page that changes price daily needs high frequency; an "About Us" page that changes every two years needs low frequency.
The most common mistake? Setting all priorities equal. If every URL has priority 0.5, you're telling Google everything is equally important. And it takes you at your word: it crawls everything with the same attention, wasting budget on pages that don't deserve it. We always start with one question: which page drives the most revenue? That one gets priority 1.0. Everything else scales down.
Sponsored Protocol
How crawl budget works in practice
Google assigns each site a quota of pages to crawl in a given period. This quota depends on domain authority, server speed, and content quality. If your server responds slowly or returns too many errors, the budget shrinks. The sitemap doesn't increase the budget, but it helps you use it better.
A concrete example: an e-commerce client with 50,000 products had a plugin-generated sitemap with all priorities at 0.5 and weekly frequencies. Google crawled thousands of filter and tag URLs, but best-selling products took weeks to get indexed. We rewrote the sitemap: priority 1.0 for top-selling products, 0.3 for tags, 0.1 for filters. Crawling focused on the right pages and sales increased 18% in two months.
How to set priority in your XML sitemap?
Priority is expressed as a value from 0.0 to 1.0. There are no fixed rules, but a sensible hierarchy for most business sites is:
- 1.0 — Homepage and main conversion pages (contact, quote, key product pages)
- 0.8 — Main categories and service pages that generate leads
- 0.6 — Recent blog posts and second-level pages
- 0.4 — Support pages, FAQ, terms and conditions
- 0.2 — Tags, archives, filter pages, duplicate content
- 0.0 — URLs to exclude (better to remove them entirely from the sitemap)
The golden rule: the sum of priorities must not exceed the total importance value of the page. If you have 100 pages and give all of them priority 1.0, the signal cancels out. Priority is relative: it's meant to distinguish, not to equalize.
Dynamic priorities for e-commerce and growing sites
For e-commerce, priority shouldn't be static. A product that sells well today might sell nothing in three months. The solution is generating the sitemap dynamically, reading sales or traffic data. We do this with a PHP script that queries the database and assigns priority based on sales from the last 30 days.
Sponsored Protocol
// Example of dynamic priority generation for e-commerce
$sales = getProductSales($productId, 30); // sales last 30 days
if ($sales > 100) {
$priority = 1.0;
} elseif ($sales > 50) {
$priority = 0.8;
} elseif ($sales > 10) {
$priority = 0.6;
} else {
$priority = 0.3;
}
// Write the URL in the sitemap with the calculated priorityThis approach ensures Google always sees pages that generate real value as priorities. If you don't have a sales system, you can use Google Analytics data: sessions, time on page, conversions.
How to choose the right change frequency?
Change frequency (changefreq) indicates how often the page content changes. Valid values are: always, hourly, daily, weekly, monthly, yearly, never. Again, Google doesn't blindly follow the value, but uses it as a hint for planning visits.
The right choice depends on content type:
- always — Only for pages that change on every visit (forums, boards, real-time prices)
- hourly — News, pages with live data, product availability
- daily — Blogs with frequent posts, e-commerce with dynamic pricing
- weekly — Most business sites, blogs with weekly posts
- monthly — Service pages, product descriptions that rarely change
- yearly — Legal pages, privacy policy, terms of service
- never — Truly immutable content (rare, almost never correct)
The most common mistake is setting daily for all pages, hoping Google will crawl more often. It doesn't work that way: if content doesn't change, Google reduces visit frequency on its own. Better to be honest: a realistic frequency builds trust with the crawler.
Sponsored Protocol
Frequency and last modification: the winning duo
Frequency alone isn't enough. The lastmod field is equally important: it tells Google when the page was last modified. Together, these two fields tell the crawler: "this page changes weekly and was last modified on March 15." If the date is recent, Google crawls it sooner.
Many auto-generated sitemaps have empty or incorrect lastmod. Verify your system fills it correctly. For WordPress, the Yoast SEO plugin does it automatically; for custom sites, you need to ensure your script reads the actual content modification date.
<url>
<loc>https://www.yoursite.com/products/running-shoes</loc>
<lastmod>2026-03-15T10:30:00+01:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.8</priority>
</url>What errors should you avoid in your XML sitemap?
After years of auditing dozens of sites, here are the most common errors we see:
- All priorities equal — Cancels the signal, Google doesn't know what to prioritize
- Non-canonical URLs — Including tracking parameters, www and non-www versions, mixed http and https pages
- 404 or redirect pages — URLs returning errors or redirects waste crawl budget
- Duplicate content — Tags, filters, and archives that shouldn't be indexed
- Single sitemap for huge sites — Better separate sitemaps for products, blog, categories (50,000 URL limit per file)
- Ignoring images and videos — Extended sitemaps for media can improve visibility in vertical searches
A concrete case: a B2B client had 300 filter page URLs in the sitemap, generated by the internal search system. Google indexed them all, diluting the authority of service pages. We removed filters from the sitemap and added noindex to the pages themselves. In three weeks, service pages went from position 12 to position 4 for main keywords.
Sponsored Protocol
How to check your sitemap health
Verification isn't a one-time action. It must be done periodically. Here's our operational checklist:
- Check Google Search Console's "Sitemaps" section for parsing errors or blocked URLs
- Use
curlor an online tool to download the sitemap and verify URLs respond with 200 OK - Check that
lastmodis updated for modified pages - Verify priorities reflect the current commercial value of pages
- Ensure the sitemap is referenced in robots.txt
# Quick check of URLs in the sitemap
curl -s https://www.yoursite.com/sitemap.xml | grep -o '<loc>[^<]*</loc>' | sed 's/<\/?loc>//g' | while read url; do
status=$(curl -o /dev/null -s -w "%{http_code}" "$url")
echo "$status $url"
done | grep -v "^200"This script shows you all URLs that don't respond with 200 OK. If you find 404s or 301s, fix them immediately.
How to structure multiple sitemaps for complex sites?
If your site exceeds 10,000 URLs, a single sitemap isn't enough. The solution is a sitemap index that groups multiple sitemaps. This approach improves organization and lets you update only the relevant sitemap when a site section changes.
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>https://www.yoursite.com/sitemap-products.xml</loc>
<lastmod>2026-03-15</lastmod>
</sitemap>
<sitemap>
<loc>https://www.yoursite.com/sitemap-blog.xml</loc>
<lastmod>2026-03-14</lastmod>
</sitemap>
<sitemap>
<loc>https://www.yoursite.com/sitemap-services.xml</loc>
<lastmod>2026-03-10</lastmod>
</sitemap>
</sitemapindex>Each child sitemap can have different priorities and frequencies, calibrated to content type. The products sitemap will have dynamic priorities based on sales; the blog sitemap lower priorities but high frequency; the services sitemap medium priority and monthly frequency.
Sponsored Protocol
For automatic generation, we use PHP or Laravel scripts that read the database and generate XML files. If you use WordPress, plugins like Yoast SEO or Rank Math handle automatic splitting. For custom sites, the solution is writing a command that regenerates sitemaps on every content change.
What to do now
The XML sitemap isn't a file to create and forget. It's a continuous communication tool with Google. Here are immediate actions to take today:
- Download your current sitemap and check priorities: are they all equal? If so, that's the first problem to solve
- Identify the 10 pages that drive the most revenue (sales, leads, contacts) and assign them priority 1.0 or 0.9
- Remove from the sitemap all URLs that shouldn't be indexed: tags, filters, thank-you pages, duplicate content
- Set the
lastmodfield correctly, reading the actual modification date from the database - Set up periodic verification with the curl script shown above, to catch broken URLs immediately
If you need help optimizing your site's sitemap or understanding why Google isn't indexing the right pages, contact us. We at Meteora Web work on these problems every day: we know how to turn a neglected sitemap into a tool that drives traffic and revenue.
