Prompt chaining

01 One huge prompt

A “chat with your database” feature. One prompt gets everything and has to do everything.

02 Split it into steps

Each step does one job. Each step’s output is the next step’s input.

1. 1
   Pick the tablesLLM

   → ["orders", "customers", "refunds"]A small, cheap model is enough.
2. 2
   Write the SQLLLM

   → SELECT c.name, SUM(r.amount) FROM refunds r …Sees only those 3 schemas + the question.
3. 3
   Run the queryCODE · NO LLM

   → 12 rowsNot every step needs an LLM.
4. 4
   Explain the resultLLM

   → "3 customers got 60% of the refunds…"Gets the original question too, not just the rows.

03 The real power is between the steps

A check in plain code between two steps. Anthropic calls it a gate. Here it sits between step 2 and step 3.

The gate isn’t your only defence. A SELECT-only check can still be beaten by an expensive query. Run generated SQL as a read-only DB user, on a replica, with a statement timeout and a row limit.

04 What it looks like in code

A sketch in Python. llm_json, llm_text, run_readonly and explain are your own thin helpers over any LLM SDK and DB driver. The gate uses sqlglot to parse the SQL.

import sqlglot
from sqlglot import exp

MAX_RETRIES = 2

def answer(question: str) -> str:
    # Step 1 (LLM, small model): pick tables. JSON out, so code can check it.
    tables = llm_json(PICK_TABLES, question=question, tables=list(SCHEMA), model="small")
    if not tables or not set(tables) <= SCHEMA.keys():      # gate
        raise ChainError("step 1: unknown tables", tables)

    # Step 2 (LLM): write SQL from only those schemas. Gate it; retry only this step.
    error = None
    for _ in range(MAX_RETRIES + 1):
        sql = llm_text(WRITE_SQL, question=question,
                       schemas={t: SCHEMA[t] for t in tables}, last_error=error)
        error = check_sql(sql, tables)                         # gate
        if error is None:
            break
    else:
        raise ChainError("step 2: no valid SQL", error)        # fail clearly

    # Step 3 (code, no LLM): read-only user, statement timeout, row limit.
    rows = run_readonly(sql, timeout_s=5, max_rows=500)

    # Step 4 (LLM): explain. Pass the original question, not just the rows.
    return llm_text(EXPLAIN_ROWS, question=question, rows=rows)


def check_sql(sql: str, tables: list[str]) -> str | None:
    """The gate: plain code. Returns an error for the LLM to fix, or None."""
    try:
        stmts = sqlglot.parse(sql, read="postgres")
    except sqlglot.errors.ParseError as e:
        return f"parse error: {e}"
    if len(stmts) != 1 or not isinstance(stmts[0], exp.Select):
        return "send exactly one SELECT statement"
    known = {col for t in tables for col in SCHEMA[t]}
    aliases = {e.alias for e in stmts[0].expressions if e.alias}  # ORDER BY total
    missing = {c.name for c in stmts[0].find_all(exp.Column)} - known - aliases
    if missing:
        return f"columns not found: {sorted(missing)}"
    try:
        explain(sql)                       # EXPLAIN plans the query without running it
    except DatabaseError as e:
        return f"EXPLAIN failed: {e}"
    return None

• Structured output between steps. Step 1 returns JSON, so the gate can check it. Free text between steps is where chains break.
• The retry carries the error. Step 2 gets last_error, so the model fixes the actual problem instead of guessing again.
• Retries are bounded. After 2, the chain fails with a clear error instead of looping or returning a confident wrong answer.

05 Why it matters: errors multiply

If each step is right 95% of the time, three steps in a row are right only about 86% of the time. Move the sliders.

06 The trade-off

07 Senior notes

08 Interview check

An LLM turns resumes into a JSON profile, then scores each profile against the job description. Where do the gates go?