Dynamic Sitemaps: Auto-Generate Your Sitemap

A static sitemap file works until it doesn't. You add a blog post, and the sitemap is stale. You delete a product, and the sitemap has a dead URL. You restructure your site, and the sitemap points nowhere. Dynamic sitemaps solve this by generating from your actual data every time they're requested (or on a smart schedule). Here's how to implement them across the most common frameworks, with caching strategies that keep things fast.

What Is a Dynamic Sitemap?

A dynamic sitemap is generated programmatically from your content source -- a database, CMS API, file system, or any other data store. Instead of maintaining an XML file by hand, your application builds the sitemap on the fly based on what actually exists right now.

The output is identical: a valid XML sitemap that search engines can parse. The difference is in how that XML gets created.

Benefits of Dynamic Sitemaps

Implementation: Next.js (App Router)

Next.js 13+ with the App Router has built-in sitemap support. Create a sitemap.ts file in your app/ directory and export a default function.

Basic Static Generation

// app/sitemap.ts
import { MetadataRoute } from 'next'

export default function sitemap(): MetadataRoute.Sitemap {
  return [
    {
      url: 'https://example.com',
      lastModified: new Date(),
      changeFrequency: 'weekly',
      priority: 1,
    },
    {
      url: 'https://example.com/about',
      lastModified: new Date(),
      changeFrequency: 'monthly',
      priority: 0.8,
    },
  ]
}

Dynamic Generation from a Database

// app/sitemap.ts
import { MetadataRoute } from 'next'
import { db } from '@/lib/database'

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const posts = await db.posts.findMany({
    select: { slug: true, updatedAt: true },
    where: { published: true },
  })

  const products = await db.products.findMany({
    select: { slug: true, updatedAt: true },
    where: { active: true },
  })

  const postUrls = posts.map((post) => ({
    url: `https://example.com/blog/${post.slug}`,
    lastModified: post.updatedAt,
  }))

  const productUrls = products.map((product) => ({
    url: `https://example.com/products/${product.slug}`,
    lastModified: product.updatedAt,
  }))

  return [
    { url: 'https://example.com', lastModified: new Date() },
    ...postUrls,
    ...productUrls,
  ]
}

This generates your sitemap at build time. For sites with frequently changing content, you can use generateSitemaps() for dynamic route generation and combine it with ISR (Incremental Static Regeneration).

Multiple Sitemaps with generateSitemaps

For large sites, Next.js supports generating multiple sitemaps:

// app/sitemap.ts
import { MetadataRoute } from 'next'
import { db } from '@/lib/database'

const URLS_PER_SITEMAP = 50000

export async function generateSitemaps() {
  const totalProducts = await db.products.count()
  const numSitemaps = Math.ceil(totalProducts / URLS_PER_SITEMAP)

  return Array.from({ length: numSitemaps }, (_, i) => ({ id: i }))
}

export default async function sitemap(
  { id }: { id: number }
): Promise<MetadataRoute.Sitemap> {
  const products = await db.products.findMany({
    skip: id * URLS_PER_SITEMAP,
    take: URLS_PER_SITEMAP,
    select: { slug: true, updatedAt: true },
  })

  return products.map((product) => ({
    url: `https://example.com/products/${product.slug}`,
    lastModified: product.updatedAt,
  }))
}

Implementation: Ruby on Rails

Rails doesn't have built-in sitemap support, but the sitemap_generator gem is the standard solution:

# Gemfile
gem 'sitemap_generator'

# config/sitemap.rb
SitemapGenerator::Sitemap.default_host = "https://example.com"

SitemapGenerator::Sitemap.create do
  # Static pages
  add '/about', changefreq: 'monthly'
  add '/contact', changefreq: 'monthly'

  # Dynamic content
  Product.find_each do |product|
    add product_path(product), lastmod: product.updated_at
  end

  Post.published.find_each do |post|
    add post_path(post), lastmod: post.updated_at
  end
end

Run rake sitemap:refresh to generate sitemaps. The gem handles splitting, gzip compression, sitemap indexes, and pinging search engines automatically. Schedule it with cron for automatic regeneration.

Implementation: Django

Django has a built-in sitemap framework in django.contrib.sitemaps:

# sitemaps.py
from django.contrib.sitemaps import Sitemap
from .models import Product, BlogPost

class ProductSitemap(Sitemap):
    changefreq = "daily"

    def items(self):
        return Product.objects.filter(active=True)

    def lastmod(self, obj):
        return obj.updated_at

    def location(self, obj):
        return f"/products/{obj.slug}"

class BlogSitemap(Sitemap):
    changefreq = "weekly"

    def items(self):
        return BlogPost.objects.filter(published=True)

    def lastmod(self, obj):
        return obj.updated_at

# urls.py
from django.contrib.sitemaps.views import sitemap
from .sitemaps import ProductSitemap, BlogSitemap

sitemaps = {
    'products': ProductSitemap,
    'blog': BlogSitemap,
}

urlpatterns = [
    path('sitemap.xml', sitemap, {'sitemaps': sitemaps},
         name='django.contrib.sitemaps.views.sitemap'),
]

Django generates the sitemap on each request by default. For large sites, enable the GenericSitemap with caching or use django.contrib.sitemaps.views.index for sitemap indexes.

Implementation: Laravel

Laravel's Spatie sitemap package is the most popular option:

// Install: composer require spatie/laravel-sitemap

// routes/console.php (scheduled command)
use Spatie\Sitemap\Sitemap;
use Spatie\Sitemap\Tags\Url;

Schedule::call(function () {
    Sitemap::create()
        ->add(Url::create('/')->setPriority(1.0))
        ->add(
            Product::all()->map(function ($product) {
                return Url::create("/products/{$product->slug}")
                    ->setLastModificationDate($product->updated_at);
            })
        )
        ->writeToFile(public_path('sitemap.xml'));
})->daily();

This generates a static file on a schedule -- a hybrid approach that gives you dynamic content with static file performance.

Implementation: Custom Script

If you're not using a framework, a simple script that queries your database and outputs XML works fine:

// generate-sitemap.js
const { Pool } = require('pg')
const fs = require('fs')

const pool = new Pool({ connectionString: process.env.DATABASE_URL })

async function generateSitemap() {
  const { rows: pages } = await pool.query(
    'SELECT slug, updated_at FROM pages WHERE published = true'
  )

  const urls = pages.map(page => `
  <url>
    <loc>https://example.com/${page.slug}</loc>
    <lastmod>${page.updated_at.toISOString()}</lastmod>
  </url>`).join('')

  const xml = `<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
${urls}
</urlset>`

  fs.writeFileSync('./public/sitemap.xml', xml)
  console.log(`Generated sitemap with ${pages.length} URLs`)
}

generateSitemap()

Run this script in your CI/CD pipeline, as a cron job, or as a webhook triggered by content changes.

Caching Strategies

Dynamic sitemaps can be expensive to generate on every request, especially for large sites with database queries. Here are the most effective caching approaches:

Build-Time Generation

Generate the sitemap during your build/deploy process. The sitemap is a static file that serves instantly. Regenerate on every deploy.

Best for: Sites deployed frequently (multiple times per day) where content changes align with deployments.

Time-Based Caching (TTL)

Generate the sitemap on request but cache it for a fixed duration (e.g., 1 hour, 6 hours, 24 hours). Subsequent requests serve the cached version.

Best for: Sites with moderate update frequency where a slight delay in sitemap updates is acceptable.

Event-Driven Regeneration

Regenerate the sitemap only when content changes -- triggered by a webhook, database event, or CMS callback. Cache indefinitely until the next change.

Best for: Sites where content changes are infrequent but you need the sitemap to reflect changes quickly when they happen.

Performance Considerations

For sites with hundreds of thousands of URLs, sitemap generation can become a bottleneck. Here's how to keep it fast:

• Paginate database queries with LIMIT/OFFSET or cursor-based pagination instead of loading everything into memory
• Use a sitemap index so each child sitemap queries only its segment of data
• Cache aggressively -- most sites don't need real-time sitemap accuracy
• Generate asynchronously in a background job rather than on the request path
• Compress with gzip -- serve .xml.gz files to reduce bandwidth and transfer time
• Index only what matters -- don't include every URL. Include canonical, indexable pages that return 200

When to Regenerate

The right regeneration trigger depends on your content velocity:

The goal is a sitemap that's accurate enough for search engines without burning resources on unnecessary regeneration. For most sites, generating at build time or caching for a few hours hits the sweet spot.

---

Static sitemaps decay. Dynamic sitemaps stay accurate. Pick the approach that matches your content velocity and stop maintaining XML by hand.