llms.txt

2 posts

• The Quiet Death of robots.txt: How AI Crawlers Actually Decide What to ReadAn honest look at how modern AI crawlers respect (or ignore) robots.txt, the bot taxonomy publishers face in 2026, and a decision tree for setting a defensible AI-bot policy.May 7, 2026 · 9 min read
• How AI Answer Engines Actually Pick SourcesAn honest look at how ChatGPT, Perplexity, Claude, and Gemini select and cite sources, what AEO really means in practice, and what publishers should do this week.May 6, 2026 · 7 min read