# AGENTS.md and llms.txt: make your code and site agent ready

> How to write AGENTS.md for your codebase and llms.txt for your website so AI agents and answer engines can read them. StellarFirm is the worked example.

By StellarFirm team. Published 2026-10-24. 4 min read.

Keywords: agents.md, llms.txt, agent readiness, generative engine optimization, markdown for ai, ai crawlers
Tools covered: StellarFirm, OpenAI Codex, Cursor

## Key takeaways

- AGENTS.md tells coding agents how to build, test, and behave in your repository, and llms.txt gives AI systems a Markdown map of your website.
- The two files are plain Markdown, cheap to write, and most useful when they are short, accurate, and link to deeper pages.
- Serving a Markdown copy of each page and answering Accept: text/markdown helps agents read your content without scraping HTML.
- StellarFirm publishes llms.txt, agents.md, an OpenAPI file, and Markdown pages, and you can copy the pattern.

Making a codebase and a website agent ready means giving AI systems a clear, short, accurate place to start: AGENTS.md for the repository and llms.txt for the website. This guide shows what each file contains, how to serve Markdown copies of your pages, and how the StellarFirm site does it, so you can copy the pattern.

## What is AGENTS.md and what goes in it?

AGENTS.md is a plain Markdown file at the root of a repository that tells coding agents how to work there. It is documented at agents.md and read by tools such as OpenAI Codex, and Cursor documents support for it too. Think of it as a README written for an agent.

Keep it short and verifiable. A reliable layout:

| Section | Content |
| --- | --- |
| Overview | One paragraph: what the project is and its main folders |
| Setup | Exact install and run commands |
| Checks | The commands that must pass before a change is done |
| Conventions | Style, naming, structure, patterns to repeat |
| Boundaries | What not to touch, and when to ask first |
| Pull requests | Size, title format, what to put in the description |

Nested folders can have their own file for local rules, and the closest file to the code usually applies. Link to deeper docs instead of copying them.

```prompt title="Generate a first AGENTS.md"
Read this repository and write an AGENTS.md with sections for overview, setup, checks, conventions, boundaries, and pull requests. Use only commands you can verify in the repository, keep it under 80 lines, and list anything you could not verify at the end.
```

```prompt title="Audit an AGENTS.md"
Audit our AGENTS.md. Find commands that no longer work, rules that contradict each other, and sections that are too long. Propose a shorter version and show me the changes before saving.
```

## What is llms.txt and what goes in it?

llms.txt is a proposed convention, described at llmstxt.org, for a Markdown file at the root of your website that gives language models a concise map of the site. It is not a guarantee of visibility, and support varies by system, but it is cheap and it forces you to decide what matters.

The proposal suggests this shape:

1. A title line with the site or product name.
2. A short summary in a quote block.
3. Sections of links, each with a one line description, pointing to the pages an agent should read.

Make every description a fact, not a slogan. An agent deciding whether to open a page reads that line.

```prompt title="Draft an llms.txt"
Draft an llms.txt for [website]. Start with the product name and a two sentence summary, then add sections for the main pages, documentation, pricing or plans, and contact. Give each link a plain one line description. Use only pages that exist, listed here: [list of URLs].
```

## Why serve Markdown copies of pages?

Agents read Markdown cleanly and HTML poorly. A Markdown copy strips navigation, scripts, and layout, so the answer to a question is a few hundred clean tokens instead of a noisy page.

Two common ways to serve it:

- **A parallel address**, such as the same path with a .md ending.
- **Content negotiation**: when a request sends an Accept header asking for text/markdown, answer with the Markdown version of the same page.

Do the two if you can. Keep the Markdown identical in substance to the page, because answer engines compare what they read with what is shown.

## How does StellarFirm do it?

StellarFirm's public site is made for agents, and every piece below is a stable, public address.

| Piece | What it gives an agent |
| --- | --- |
| `/llms.txt` | A map of the site for AI systems |
| `/agents.md` | Instructions on when to use StellarFirm and how to reach it |
| `/openapi.json` | The public API as an OpenAPI file |
| `/docs` | Documentation pages, also available as Markdown |
| `/mcp` | A public, read only MCP server |

The public API is JSON with one error shape, so a program can tell what went wrong without reading a web page. See the [API docs](/docs/api), and the [MCP server page](/docs/getting-started/mcp-server) for how an agent can search the docs directly. Everything public is for reading and checking: anything that sends, deploys, or charges waits for a person's Approve inside the app.

## How do I write for answer engines too?

Answer engines quote pages that answer a question directly and clearly. A few habits help, and they help human readers as well.

- Put a direct answer in the first two sentences under each heading.
- Use question style headings that match how people ask.
- Prefer tables and short lists for facts and steps.
- State names, versions, and limits exactly, and cite official sources.
- Date your pages and keep them current.

## What is a good rollout order?

Do the cheap, high value steps first.

1. Write AGENTS.md and test it by giving an agent a small task.
2. Write llms.txt from the pages you already want read.
3. Add Markdown copies of your key pages.
4. Add content negotiation for Accept: text/markdown.
5. If you have an API, publish an OpenAPI file and one error shape.

```prompt title="Plan agent readiness"
Look at [website or repository] and give me a prioritized plan to make it agent ready. Cover AGENTS.md, llms.txt, Markdown copies of key pages, and API documentation. For each step say what to write, how to test it, and how long it should take.
```

If you work with a coding agent daily, pair this with [Cursor and Codex tips](/blog/cursor-and-codex-agent-tips) and [Claude Code tips](/blog/claude-code-tips-from-the-docs).

## Frequently asked questions

### What is AGENTS.md?

AGENTS.md is a plain Markdown file in a repository that tells coding agents how to work there: setup, checks, conventions, and boundaries. It is documented at agents.md and read by several tools.

### What is llms.txt?

llms.txt is a proposed convention, described at llmstxt.org, for a Markdown file at the root of a website that gives language models a concise map of the site with links to the key pages.

### Do I need two files?

They solve different problems. AGENTS.md guides agents that work inside your code, and llms.txt guides AI systems that read your public site. A product with a website and a repository benefits from each.

### Does llms.txt guarantee AI search visibility?

No. It is a convention, not a guarantee, and support varies by system. Clear content, good structure, and accurate facts matter most. Treat llms.txt as a helpful extra.

## Sources

- [AGENTS.md](https://agents.md/)
- [The llms.txt proposal](https://llmstxt.org/)
- [OpenAI Codex documentation](https://developers.openai.com/codex)
- [Cursor documentation](https://cursor.com/docs)

---

Source: https://stellarfirm.ai/blog/agents-md-and-llms-txt-guide