In this lesson you’ll cover:
- Why agent traffic fails silently — no error, no analytics event — when a page doesn’t fit an agent’s context window
- How agents read in tokens, not pages, and how to estimate token counts (characters ÷ 4) and budget them per page
- The four failure modes when you exceed the budget: truncation, skipping, chunking overhead, and hallucination
- LLMs.txt — an agent-specific sitemap: what it is, what goes in it, and how to build one
- skill.md — declaring your capabilities up front so an agent can match a task before reading your whole site
- Optimizing robots.txt for AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) and the reference list of agent user agents
- Structuring content for sequential agent reading: front-loading value, tables over prose, consistent headings, and keeping specifics next to their claims
- Common implementation mistakes and a prevention checklist