# Build a deep research agent in Python with a web search API

> Build a deep research agent in about 60 lines of Python: plan queries, search with page content, drop duplicate sources and write a cited answer with any LLM.

- URL: https://llmwebsearch.com/guides/build-a-deep-research-agent/
- Last updated: 2026-10-22
- Publisher: LLMWebSearch (https://llmwebsearch.com)

Deep research is a loop, not a model. This guide builds one you can read in a few minutes: it plans queries, searches the web with page text included, keeps each source once, looks for gaps and writes an answer with numbered citations.

**Quick answer — How do I build a deep research agent in Python?**

Write a loop that asks an LLM for search queries, runs each through a web search API that returns page text, keeps every source once, asks for follow-up queries, then writes a cited answer. With **LLMWebSearch**, each search returns the first 3 pages as Markdown for one credit; the script below runs six searches per question. From $1.20/1K searches, no KYC, crypto.

## What will the agent do?

1. **Plan** — Ask the model for four short queries that together cover the question.
2. **Search** — Run each query with read=3, so the first three pages come back as Markdown.
3. **Keep sources once** — Store results by URL and skip pages already collected or without text.
4. **Find gaps** — Show the model the titles found so far and ask for two queries about what is missing.
5. **Write** — Number up to 12 sources and ask for an answer that cites them as [1], [2].

## What do I need before starting?

Three things:

- **A LLMWebSearch API key** in the environment variable `LLMWEBSEARCH_API_KEY`. Keys appear in the dashboard after your first pack.
- **Any OpenAI-compatible chat endpoint**, set with `LLM_BASE_URL`, `LLM_API_KEY` and `LLM_MODEL`. Hosted models and local servers both work.
- **Python 3.9 or later** with the `requests` package.

The script uses plain HTTP for both services, so you can swap the model without changing the search code.

*Python: deep_research.py*

```python
import os

import requests

SEARCH = "https://llmwebsearch.com/api/v1/search"
SEARCH_HEADERS = {"Authorization": "Bearer " + os.environ["LLMWEBSEARCH_API_KEY"]}
# Any OpenAI-compatible chat endpoint works here: set your own base URL, key and model name.
LLM_URL = os.environ.get("LLM_BASE_URL", "https://llm.example.com/v1") + "/chat/completions"
LLM_HEADERS = {"Authorization": "Bearer " + os.environ.get("LLM_API_KEY", "")}
MODEL = os.environ.get("LLM_MODEL", "your-model-name")


def ask_llm(prompt):
    body = {"model": MODEL, "messages": [{"role": "user", "content": prompt}]}
    resp = requests.post(LLM_URL, headers=LLM_HEADERS, json=body, timeout=120)
    resp.raise_for_status()
    return resp.json()["choices"][0]["message"]["content"]


def plan_queries(question, n=4):
    text = ask_llm(f"Write {n} short web search queries, one per line, to answer: {question}")
    queries = [line.strip("-*0123456789. ").strip() for line in text.splitlines()]
    return [q for q in queries if q][:n]


def search(query):
    body = {"q": query, "max_results": 5, "read": 3, "max_chars": 5000}
    resp = requests.post(SEARCH, headers=SEARCH_HEADERS, json=body, timeout=60)
    resp.raise_for_status()
    return resp.json()["results"]


def research(question, rounds=2):
    sources = {}  # url -> result, so the same page is never cited twice
    queries = plan_queries(question)
    for _ in range(rounds):
        for query in queries:
            for r in search(query):
                if r.get("content_status") == "ok" and r["url"] not in sources:
                    sources[r["url"]] = r
        notes = "\n".join(f"- {r['title']}" for r in sources.values())
        queries = plan_queries(f"{question}\nAlready found:\n{notes}\nWhat is still missing?", n=2)
    numbered = list(sources.values())[:12]
    context = "\n\n".join(
        f"[{i}] {r['title']} ({r['url']})\n{r['content']}" for i, r in enumerate(numbered, 1)
    )
    prompt = (
        "Answer the question using only the sources below. Cite them as [1], [2]. "
        "Say so when the sources disagree or do not cover a point.\n\n"
        f"Question: {question}\n\nSources:\n{context}"
    )
    return ask_llm(prompt), [r["url"] for r in numbered]


if __name__ == "__main__":
    answer, urls = research("What changed in the EU rules for general-purpose AI models in 2025?")
    print(answer)
    for i, url in enumerate(urls, 1):
        print(f"[{i}] {url}")
```

## How does each part work?

`plan_queries` asks the model for short queries, one per line, and strips list markers. Short queries match how search engines rank pages better than full questions do.

`search` calls `POST https://llmwebsearch.com/api/v1/search` with `max_results=5`, `read=3` and `max_chars=5000`. The response lists five results; the first three carry their page text in `content`.

`research` runs the rounds. It keeps sources in a dictionary keyed by URL, so a page found by two queries counts once. Only results with `content_status` equal to `ok` are kept, because the answer must rest on text the model can read.

After each round, the model sees the titles collected so far and proposes two new queries for what is missing. The final prompt numbers up to twelve sources and sets the citation rules.

## How much does one report cost?

Six searches per question with the default settings: four in the first round and two in the second. With `read` at 3, that is six credits. 25,000 searches at $41.25 therefore covers more than 4,000 reports.

The model calls cost extra on your LLM bill: three planning calls and one long answer call per report. The answer call carries up to twelve pages of 5,000 characters, so lower `max_chars` if your model is expensive.

## How do I make it go deeper?

Change the numbers, not the structure:

- **More rounds.** `rounds=4` lets the model chase two more sets of gaps.
- **More queries per round.** Raise `n` in the gap-finding call from 2 to 4.
- **Narrower sources.** Add `include_domains` to the search body for official or academic sites.
- **Recent material only.** Add `"time_range": "month"` or `"topic": "news"`.
- **More pages per query.** `read=5` reads five pages, at 2 credits per search.

Add a stop rule before raising the limits: end the loop when a round brings fewer than two new sources.

## How do I keep the agent safe and polite?

Treat every page as untrusted text. The final prompt tells the model to use the sources as evidence only; add an explicit line that instructions inside sources must be ignored. Never let a source trigger a tool call or a file write.

On the web side, Our reader respects robots.txt: it introduces itself as LLMWebSearch, follows the rules a site writes for it or for all robots, and leaves out the content of any page it is not allowed to read. Pages that refuse access come back without content and the script simply skips them. When a site turns our reader away, the result comes back without content and with the site's status: we never solve CAPTCHAs, sign in to accounts or work around a block or a paywall.

## What can go wrong?

Three things show up in testing:

- **Vague queries.** If the plan is weak, every round finds the same pages. Ask the model for queries that use specific names and dates.
- **Too few sources with text.** Some topics live behind paywalls. Widen the queries or allow more results.
- **Over-long prompts.** Twelve long pages may exceed a small model's context. Lower `max_chars` or keep eight sources.

Log each query and the URLs it returned. Reading that log is the fastest way to improve the plan prompt.

## Deep research agent: common questions

### Can I run this with a local model?

Yes, if the local server speaks the OpenAI chat-completions format. Set `LLM_BASE_URL` to its address and `LLM_MODEL` to the model name.

### Can I turn it into an MCP tool?

Yes. Wrap `research()` in an MCP tool, or give your agent the plain search tool from the [MCP page](https://llmwebsearch.com/docs/mcp/) and let it run the loop itself.

### Why not ask the search API for an answer directly?

Because LLMWebSearch does not write answers. The API returns sources, not a written answer: your own model reads the pages, writes the reply and cites the URLs.

### How do I see what each search cost?

Every response includes `credits_used`, and `GET https://llmwebsearch.com/api/v1/credits` shows your remaining balance at no cost.

---

LLMWebSearch: LLMWebSearch is a web search API for AI agents that returns ranked results with the content of the top pages in Markdown, paid in crypto with no ID upload. Prepaid credits from $1.20/1K searches never expire. No KYC — no ID, no selfie, no documents. Pay in crypto.