An agent that answers questions about a music store's database with tools you designed, inside a step limit, with a trace of every run.
Download the Chinook sample database (Chinook_Sqlite.sqlite, from the lerocha/chinook-database releases on GitHub) and write 20 questions with answers you work out in your own SQL.
Give the agent list_tables, describe_table and run_sql, with run_sql opening the file read-only and returning errors the model can act on.
Write the loop yourself, with a 10-step limit, a token budget and a final_answer tool as the only way to finish.
Add one write tool, create_playlist, that takes an idempotency key, and show that a call retried after a forced timeout adds the playlist once.
Write one JSON line per step (messages, tool call, result, tokens and time), so any failed run can be read back.
Run each question five times, work out per-run success and pass^3, read the failed traces, fix the most common cause and run again.
How you'll know it worksA table of per-run success, pass^3, average steps and tokens over the 20 questions, before and after your fix, with no run past 10 steps and exactly one playlist row after the retry test.
Python and SQLite (both free, sqlite3 is in the standard library), with a tool-calling model in Ollama such as qwen2.5:7b, or your own API key. No framework needed. Uses lessons 01 to 17.
Project: A database agent with traces and limits
From insideout, Agents: An agent that answers questions about a music store's database with tools you designed, inside a step limit, with a trace of every run.
Steps:
1. Download the Chinook sample database (Chinook_Sqlite.sqlite, from the lerocha/chinook-database releases on GitHub) and write 20 questions with answers you work out in your own SQL.
2. Give the agent list_tables, describe_table and run_sql, with run_sql opening the file read-only and returning errors the model can act on.
3. Write the loop yourself, with a 10-step limit, a token budget and a final_answer tool as the only way to finish.
4. Add one write tool, create_playlist, that takes an idempotency key, and show that a call retried after a forced timeout adds the playlist once.
5. Write one JSON line per step (messages, tool call, result, tokens and time), so any failed run can be read back.
6. Run each question five times, work out per-run success and pass^3, read the failed traces, fix the most common cause and run again.
How you'll know it works: A table of per-run success, pass^3, average steps and tokens over the 20 questions, before and after your fix, with no run past 10 steps and exactly one playlist row after the retry test.
Runs on: Python and SQLite (both free, sqlite3 is in the standard library), with a tool-calling model in Ollama such as qwen2.5:7b, or your own API key. No framework needed.
Help me build this on my machine, one step at a time. Ask what I've tried first, give hints before full answers, and check my work against "How you'll know it works".
Another project · on your own machine
A knowledge base your coding agent maintains
A folder of plain markdown pages that your coding agent writes and links from sources you collect, with an index, a log, disagreements written down, and answers that cite the pages they came from.
Make a git repo with raw/ for sources (never edited) and wiki/ for the agent's pages, put 15 to 20 articles or papers on one topic in raw/, and write a context file under 100 lines with the page rules: one idea per page, frontmatter with a type and dates, a link on the first mention of anything that has a page, and never overwrite a claim another source disputes.
Write an ingest skill and run it on one source at a time: each run writes a source page, creates or updates the concept pages it touches, links them both ways, adds new pages to index.md, appends one line to log.md and commits.
Add a hook that runs your own link-check script after every edit (broken links, pages nothing links to, pages missing from index.md, frontmatter that doesn't parse) and hands any failure back to the agent.
Plant two sources that disagree on one number, ingest both, and check that the page keeps both claims with their sources and dates instead of quietly keeping the newer one.
Write 15 questions, each with the page that should answer it and 3 that no source covers, and have a read-only subagent answer them by reading index.md first and citing pages.
Check every answer with a script (each cited page exists) and by reading (the page supports the claim), fix the most common cause of misses in the context file or the skill, and run the questions again.
How you'll know it worksEvery source ingested with zero broken links, zero orphan pages and every page in index.md, by your script rather than the agent's word; both planted claims on the page with their sources and dates; and on the 15 questions, every citation points to a real page, at least 10 of the 12 answerable ones cite a page that supports the answer, and all 3 uncovered ones say the knowledge base doesn't cover them.
Plain markdown in a git repo, your own coding agent (Claude Code, Codex or Cursor) on your own plan, key or local model, and a short standard-library Python script for the link check. Obsidian is optional, for browsing the links as a graph. The idea is Andrej Karpathy's LLM wiki; the vault template in Second Brain OS (MIT, github.com/undefined-ui/second-brain-os) makes a good starting point. Uses lessons 04 to 14.
Project: A knowledge base your coding agent maintains
From insideout, Agents: A folder of plain markdown pages that your coding agent writes and links from sources you collect, with an index, a log, disagreements written down, and answers that cite the pages they came from.
Steps:
1. Make a git repo with raw/ for sources (never edited) and wiki/ for the agent's pages, put 15 to 20 articles or papers on one topic in raw/, and write a context file under 100 lines with the page rules: one idea per page, frontmatter with a type and dates, a link on the first mention of anything that has a page, and never overwrite a claim another source disputes.
2. Write an ingest skill and run it on one source at a time: each run writes a source page, creates or updates the concept pages it touches, links them both ways, adds new pages to index.md, appends one line to log.md and commits.
3. Add a hook that runs your own link-check script after every edit (broken links, pages nothing links to, pages missing from index.md, frontmatter that doesn't parse) and hands any failure back to the agent.
4. Plant two sources that disagree on one number, ingest both, and check that the page keeps both claims with their sources and dates instead of quietly keeping the newer one.
5. Write 15 questions, each with the page that should answer it and 3 that no source covers, and have a read-only subagent answer them by reading index.md first and citing pages.
6. Check every answer with a script (each cited page exists) and by reading (the page supports the claim), fix the most common cause of misses in the context file or the skill, and run the questions again.
How you'll know it works: Every source ingested with zero broken links, zero orphan pages and every page in index.md, by your script rather than the agent's word; both planted claims on the page with their sources and dates; and on the 15 questions, every citation points to a real page, at least 10 of the 12 answerable ones cite a page that supports the answer, and all 3 uncovered ones say the knowledge base doesn't cover them.
Runs on: Plain markdown in a git repo, your own coding agent (Claude Code, Codex or Cursor) on your own plan, key or local model, and a short standard-library Python script for the link check. Obsidian is optional, for browsing the links as a graph. The idea is Andrej Karpathy's LLM wiki; the vault template in Second Brain OS (MIT, github.com/undefined-ui/second-brain-os) makes a good starting point.
Help me build this on my machine, one step at a time. Ask what I've tried first, give hints before full answers, and check my work against "How you'll know it works".