Markdown Content Negotiation for AI Agents

HTML remains the canonical representation of most public websites, but an agent may spend substantial context parsing navigation, scripts, and presentation markup. A negotiated Markdown representation can reduce that overhead without creating a second content publishing system.

Cloudflare Markdown for Agents performs this conversion at the edge for enabled zones. A client requests text/markdown; Cloudflare fetches the HTML origin response and returns converted Markdown when the page qualifies.

This is a Cloudflare product feature, not an HTTP standard for universal HTML-to-Markdown conversion. HTTP content negotiation supplies the request and caching semantics. Cloudflare defines the conversion behavior, account setting, response structure, token-count fields, and limits.

The request and response contract

A client expresses its preference through the Accept request field:

curl -sS -D - https://www.tkoresearch.com/ \
  -H 'Accept: text/markdown' \
  -o /tmp/tkoresearch-home.md

For a successful converted response, Cloudflare documents fields like these:

HTTP/2 200
Content-Type: text/markdown; charset=utf-8
Vary: Accept
x-markdown-tokens: 725
x-original-tokens: 12345
Content-Signal: ai-train=yes, search=yes, ai-input=yes

Content-Type identifies the selected representation. Vary: Accept tells caches that requests with different Accept values can receive different response variants. Without correct variation, a shared cache could serve Markdown to a browser that requested HTML, or HTML to a machine client that requested Markdown.

Clients should still inspect the actual response. An Accept field states preference; it doesn't guarantee that conversion will occur. The origin may return a non-HTML resource, the response may exceed a limit, or the feature may be disabled.

What the token-count fields mean

Cloudflare includes x-markdown-tokens as its estimate for the converted document and x-original-tokens as its estimate for the source HTML. These values help a client decide whether a document fits a context budget or needs chunking.

They aren't billing records or exact counts for every model tokenizer. Tokenization differs among models and versions. Treat the fields as comparative size estimates. A client making a hard capacity decision should leave margin or count locally with its actual tokenizer.

The ratio helps with operations. A reduction from 12,000 estimated tokens to 2,000 shows that conversion removed substantial page chrome. Nearly identical values deserve inspection because the source may already be lean or the converted body may repeat material.

Don't use a low count as the only quality check. A compact document can omit tables, code, labels, or relationships a client needs. Read representative output and compare it with the rendered page.

Conversion behavior and limits

Cloudflare documents a predictable output structure. Supported page metadata can become YAML frontmatter. The main document body is converted to Markdown after non-content elements are removed. JSON-LD may be preserved in a fenced JSON block at the end.

The current feature converts HTML. It doesn't promise conversion for every document type. The origin response cannot exceed 2 MB, or 2,097,152 bytes. Pages that require a browser to execute client-side code before meaningful content appears may also need separate validation, because edge conversion works from the fetched HTML response.

These constraints should shape tests. Include short pages, long articles, tables, code blocks, structured data, redirect responses, errors, and a page near the size limit. Check relative links from directory-style URLs. Cloudflare's July 13, 2026 release corrected relative-link resolution for those URLs, so a current deployment should preserve the proper target.

Origin headers remain authoritative

The origin owns security, caching, and content-use policy. Edge conversion shouldn't replace that policy with a generic default.

On July 13, 2026, Cloudflare announced origin header preservation for Markdown for Agents. Security fields such as HSTS, CSP, X-Frame-Options, cookies, and CORS fields are preserved on converted responses. Cache fields continue to pass through.

The change also makes the origin's Content Signals policy authoritative. When the origin sends a Content-Signal field, Cloudflare preserves it. If the field is absent, Cloudflare adds its default all-yes value.

TKOResearch sets its approved public-content policy at the origin:

Content-Signal: ai-train=yes, search=yes, ai-input=yes

That policy happens to match the current Cloudflare default, but origin ownership still matters. A future policy change should be deployed and reviewed with the site, not inferred from an edge default. Verify the field on both HTML and Markdown variants.

Repository work and account work are different

The TKOResearch application deploys on Vercel and the production domain is served through Cloudflare. Markdown for Agents is enabled at the Cloudflare zone, either in AI Crawl Control or through the zone setting API. A repository commit can't turn on that account-level setting.

The codebase therefore carries the parts it can own:

the origin content-use response field;
accurate public content and canonical links;
a maintained /llms.txt index;
an operations runbook with enablement, verification, and rollback.

The runbook requires a scoped API token with Zone Settings edit permission if the API path is used. This separation makes deployment auditable. Source control defines the origin behavior, while the Cloudflare change remains an explicit infrastructure operation.

Verification before enablement

Capture an HTML baseline first:

curl -sS -D /tmp/html-headers.txt \
  https://www.tkoresearch.com/ \
  -o /tmp/home.html

After enablement, request Markdown and compare status, canonical links, security fields, cache fields, and content-use policy. Confirm Vary: Accept and both token-count fields. Open the saved body and inspect headings, lists, code, links, and JSON-LD.

Repeat the request through any production hostname that clients use. Check caching by alternating HTML and Markdown requests. A passing origin test doesn't establish that the CDN's public behavior is correct.

Monitor conversion after site template changes. Moving important copy into a client-only widget or a navigation landmark that preprocessing removes can alter machine-visible content even though the browser view still appears correct.

Rollback is part of the feature

Rollback means disabling the content_converter zone setting or turning off Markdown for Agents in the Cloudflare dashboard. After propagation, a request with Accept: text/markdown should no longer receive a Cloudflare-converted Markdown response.

The site's other machine-readable controls remain independent. /robots.txt, /llms.txt, the origin Content-Signal field, RSS discovery, and the RFC 9116 security contact still work. That independence reduces rollback risk.

Record who can make the account change, how long propagation took, and what response confirms success. Don't wait for a conversion defect to discover that the API token lacks permission or that the dashboard owner is unavailable.

Adoption decision

Markdown negotiation is appropriate for TKOResearch because it represents existing public content rather than advertising a new agent service. The edge feature can improve parsing efficiency while the HTML page remains canonical for ordinary browsers.

Enable it through Cloudflare operations, verify both variants, and preserve origin policy. If output quality or caching becomes unreliable, disable conversion without removing the site's crawler policy or maintained public indexes.