Making Your Xperience by Kentico Site Agent-Ready

Blog Development

June 10, 2026

Author

Victor Hugo García

AI agents are no longer a future concept, they are actively crawling, reading, and interacting with websites today. As search evolves from keyword matching to generative discovery, the way we prepare our sites for these new consumers of content must evolve with it. Our updated XperienceCommunity.SEO package brings Xperience by Kentico sites up to speed with the latest standards, including Markdown Content Negotiation, now updated for .NET 10.

The Shift Toward Generative Engine Optimization

Traditional SEO has always been about making your content discoverable by search engine crawlers. But the landscape is shifting rapidly. Generative Engine Optimization (GEO) extends that concept to AI agents: models like ChatGPT, Gemini, Claude, and a growing number of autonomous agents that consume, reason about, and reference your content.

These agents don't parse HTML the same way browsers do. They prefer structured, token-efficient formats. They look for explicit signals about what your content is, how it should be used, and whether they're allowed to access it.

There are now emerging tools and standards designed to evaluate how well a site communicates with AI systems, covering areas like discoverability, content accessibility, bot access control, and protocol discovery. Benchmarking against these criteria can reveal gaps you didn't know existed.

When we evaluated our own Xperience by Kentico projects against these standards, one gap stood out immediately: the need for Markdown Content Negotiation.

What We Already Covered

Last year, we released the XperienceCommunity.SEO package and published a detailed walkthrough in our previous article, Simplify SEO in Xperience by Kentico. That initial release addressed three core infrastructure files:

robots.txt: configurable per environment, so you can block crawlers in staging while allowing full access in production
Sitemap.xml: dynamically generated from your Kentico content types, with cache optimization using Kentico's dependency system
Llms.txt: a structured file that helps AI models understand your site's page hierarchy and purpose

If you haven't read the previous article, we recommend starting there for the full setup context. Everything from that version still applies, the new release builds on top of it.

What's New: Markdown Content Negotiation

The headline feature in this release is Markdown Content Negotiation. This is a mechanism based on standard HTTP content negotiation: when an AI agent sends a request with the header Accept: text/markdown, your site responds with a Markdown rendition of the page instead of the default HTML.

Why does this matter? HTML pages carry a massive amount of structural overhead: navigation, scripts, styles, footers, spinners; that adds no value for an AI system trying to understand your content. Markdown strips all that away and delivers clean, structured text that's dramatically more token-efficient.

Cloudflare has been championing this approach through their Markdown for Agents initiative. Their scanner specifically checks whether your site supports this negotiation pattern. With the updated XperienceCommunity.SEO package, your Xperience by Kentico site can pass that check.

How It Works

Under the hood, the middleware intercepts requests that include Accept: text/markdown, captures the HTML response from your application, and converts it to Markdown using ReverseMarkdown. The conversion preserves headings, links, lists, images, and tables. Relative URLs are rewritten to absolute URLs using the current request's scheme and host.

The response includes two important details:

The Content-Type header is set to text/markdown; charset=utf-8
An x-markdown-tokens response header provides an estimated token count, which agents can use for context window management

Non-HTML responses (XML, JSON, binary) pass through unchanged.

Enabling Markdown Negotiation

Add one line to your middleware pipeline in Program.cs:

var app = builder.Build();

// Enable Markdown content negotiation (place before UseRouting/MapControllers)
app.UseMarkdownContentNegotiation();

app.MapControllers();
app.Run();

Configuration Options

The middleware accepts an optional callback to fine-tune its behavior:

app.UseMarkdownContentNegotiation(opts =>
{
    opts.SkippedPaths  = ["/api/*", "/health"];
    opts.StripElements = ["header", "nav", "footer", "script", "style", "noscript"];
    opts.StripClasses  = ["page-spinner"];
});

SkippedPaths lets you exclude paths that should never receive Markdown negotiation: API endpoints, health checks, and similar routes. Patterns support exact matches (/health) and wildcard matches (/api/*).

StripElements defines which HTML tag names to remove before conversion. The defaults: header, nav, footer, script, style, noscript, strip layout chrome so agents receive only meaningful content. You can override this to an empty array if you want to disable tag-based stripping entirely.

StripClasses removes any element carrying one of the specified CSS classes. This is useful for decorative wrappers, loading spinners, or empty layout containers that produce noise in the Markdown output.

Testing Your Implementation

You can verify the feature works with a simple curl command:

# Default HTML response
curl https://yoursite.com/about-us

# Markdown negotiation
curl -H "Accept: text/markdown" https://yoursite.com/about-us

# Inspect response headers
curl -I -H "Accept: text/markdown" https://yoursite.com/about-us

Updated for .NET 10

This release also updates the package to support .NET 10, aligning with Xperience by Kentico's latest refresh. With Kentico version 31.0.0 introducing full .NET 10 support, the newest Long-Term Support release from Microsoft. Wwe've updated the package dependencies and target framework accordingly. Version 1.1.1 now requires Xperience by Kentico 31.3.0 or higher.

If you're already running on .NET 10 with Xperience by Kentico 31.3+, the upgrade path is straightforward. Simply update the package reference:

<PackageReference Include="XperienceCommunity.SEO" Version="1.1.1" />

Quick Start for New Projects

If you're setting up the package for the first time, the process remains minimal:

dotnet add package XperienceCommunity.SEO --version 1.1.1

using XperienceCommunity.SEO;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddWebsiteDiscoveryProvider(options =>
{
    options.ReusableSchemaName = "PageMetadata";
    options.DefaultLanguage = "en-US";
    options.DescriptionFieldName = "MetaDescription";
    options.TitleFieldName = "MetaTitle";
    options.SitemapShowFieldName = "ShowInSitemap";
    options.ContentTypeDependencies = new[]
    {
        "BlogPost",
        "Article",
        "LandingPage"
    };
});

var app = builder.Build();

// Enable Markdown content negotiation
app.UseMarkdownContentNegotiation();

app.MapControllers();
app.Run();

Then expose the endpoints using controllers, Minimal APIs, or route attributes, whichever fits your project's conventions. Refer to our previous article or the GitHub repository for the full endpoint setup examples.

Why This Matters for Your Kentico Projects

The Cloudflare scanner is just one signal, but it represents a broader industry shift. AI agents are becoming first-class consumers of web content, and the sites that speak their language will have a visibility advantage in AI-generated answers, in agentic workflows, and in the next generation of search.

By investing a few minutes to add the XperienceCommunity.SEO package to your Xperience by Kentico project, you're positioning your site for both traditional search engine performance and AI-driven discovery. It's a small configuration change with outsized impact.

At [A], our team of certified Kentico developers and content engineers help clients go beyond implementation, designing scalable, intelligent content systems that grow with your business.

💡 Want to make your site AI-ready?

Let’s talk about how [A] can help you optimize your Kentico or CMS implementation.