# RAG from the live web: retrieve with a search, answer with citations

> Retrieval-augmented generation from the live web: search, read the top pages as Markdown, chunk them and answer with citations. Python code. No KYC, crypto.

- URL: https://llmwebsearch.com/use-cases/rag/
- Last updated: 2026-10-22
- Publisher: LLMWebSearch (https://llmwebsearch.com)

A vector index holds what you loaded last month. Some questions need what the web says today. Live-web RAG retrieves at question time: one search, the top pages as Markdown, a few chunks into the prompt, and citations back to each page.

**Quick answer — How do I do RAG with live web search results?**

Run a web search at question time, read the top pages, split them into chunks, keep the most relevant ones and ask the model to answer with numbered citations. **LLMWebSearch** handles the first two steps in one call: ranked results with the first 3 pages as Markdown, for one credit, from $1.20/1K searches. No KYC; pay in crypto.

## What is live-web RAG?

It is retrieval-augmented generation where the retrieval step is a web search instead of a lookup in your own index. The question goes to a search API, the API returns pages, and the model answers from those pages.

The pattern suits questions about things that change: prices, releases, rules, schedules and news. It also suits long-tail questions your own documents never covered. The cost is one search per question, or two when the first one misses.

## When should I fetch fresh and when should I store?

Store what is stable and yours; fetch what changes. Product manuals, internal policies and your own docs belong in a vector index, refreshed on a schedule. Competitor prices, regulations and news belong to live retrieval.

Many teams combine both. They search their own index first and call the web only when the index has no good match, or when the question mentions a recent date.

If you decide to keep pages you retrieved, note our position: We do not train any model on your queries or on what the API returns, and we add no rule of our own against storing results; the rights of the sites whose pages you read still apply. Check each site's terms before building a permanent copy of its content.

## How do I turn search results into chunks?

Split each page on its headings, then cut long sections to a fixed length. Markdown makes this easy, because headings are lines that start with `#`. Keep the source URL and title on every chunk so citations survive the split.

The example below searches, reads three pages, chunks them and picks the six chunks that share the most words with the question. Replace the word count with your embedding model in production; the structure stays the same.

*Python: live-web retrieval with chunks and numbered sources*

```python
import os
import re

import requests

API = "https://llmwebsearch.com/api/v1/search"
HEADERS = {"Authorization": "Bearer " + os.environ["LLMWEBSEARCH_API_KEY"]}


def live_chunks(query, read=3, chunk_len=1200):
    """Search, then cut each page that was read into heading-based chunks tagged with its source."""
    body = {"q": query, "max_results": 5, "read": read, "max_chars": 12000}
    results = requests.post(API, headers=HEADERS, json=body, timeout=60).json()["results"]
    chunks = []
    for r in results:
        if r.get("content_status") != "ok":
            continue
        for section in re.split(r"\n(?=#{1,3} )", r["content"]):
            section = section.strip()
            for start in range(0, len(section), chunk_len):
                chunks.append({"url": r["url"], "title": r["title"], "text": section[start:start + chunk_len]})
    return chunks


def top_chunks(chunks, question, k=6):
    """A plain keyword score; swap in your embedding model for better ranking."""
    words = set(re.findall(r"\w{4,}", question.lower()))

    def score(chunk):
        return sum(1 for w in re.findall(r"\w{4,}", chunk["text"].lower()) if w in words)

    return sorted(chunks, key=score, reverse=True)[:k]


def build_prompt(question):
    picked = top_chunks(live_chunks(question), question)
    urls = list(dict.fromkeys(c["url"] for c in picked))
    blocks = [f"[{urls.index(c['url']) + 1}] {c['title']}\n{c['text']}" for c in picked]
    sources = "\n".join(f"[{i}] {u}" for i, u in enumerate(urls, 1))
    return (
        "Use only the context to answer and cite sources as [n].\n\n"
        + "\n\n".join(blocks)
        + f"\n\nSources:\n{sources}\n\nQuestion: {question}"
    )
```

## How many pages should each question read?

Three is a good start, and it keeps the search at one credit. Most factual questions are answered on the first two or three pages of a good query.

Ask for longer pages than you would in a chat agent, because the chunker will discard most of the text anyway. A `max_chars` of 10,000 to 15,000 per page gives the ranker enough to choose from. If the answer depends on one known site, add it with `include_domains` rather than reading more pages.

## How do I make the model cite its sources?

Number the sources in the prompt and ask for the numbers in the answer. A short instruction works well: use only the context, cite each claim as [n], and say when the context does not cover the question.

Keep the source list at the end of the prompt, with one URL per number. Then render [n] as a link in your interface. Users trust answers they can check, and wrong citations are easy to spot in review.

## What if a page is blocked or empty?

Skip it and use the others. Every result carries a `content_status`; only `ok` results have text. A page may be missing because Our reader respects robots.txt: it introduces itself as LLMWebSearch, follows the rules a site writes for it or for all robots, and leaves out the content of any page it is not allowed to read.

When all pages of a search come back empty, retry once with a different query or add `include_domains` for a site you know is open. When a site turns our reader away, the result comes back without content and with the site's status: we never solve CAPTCHAs, sign in to accounts or work around a block or a paywall.

## What does live-web RAG cost?

One search per question in most cases. A support assistant that answers 5,000 questions a month, with a second search for one question in five, uses about 6,000 searches. 25,000 searches at $41.25 covers four such months.

Your model also reads the chunks you send it. Six chunks of 1,200 characters add roughly 2,000 tokens per question, which you control through the chunk size and count.

## Which pack suits a RAG workload?

One credit per question for most pipelines. Search credits have no expiry date: what you do not use this month is still there next year.

| Plan | Quantity | Price | Price per unit |
| --- | --- | --- | --- |
| Try | 5,000 searches | $9.25 | $1.85/1K searches |
| Builder | 25,000 searches | $41.25 | $1.65/1K searches |
| Production | 100,000 searches | $145 | $1.45/1K searches |

In every plan: Top 3 pages read as Markdown · Country, language, time and domain filters · Failed calls not charged · Credits never expire

**Before you pay**

- Minimum top-up $25. Top up once in crypto, then buy any plan from your balance.
- Money on your balance does not expire and buys search credits whenever you want. When a pack is cheaper than the minimum top-up, the part you paid only to reach that minimum comes back to you on request within 30 days of payment, in USDT or USDC, provided it is still unspent. The rest of a balance is not returned unless the law requires it.
- Credited once your payment is checked on the blockchain. Typical network confirmation time: BTC ≈ 20 min · USDT TRC-20 ≈ 1 min · USDT ERC-20 ≈ 5 min · USDC ERC-20 ≈ 5 min · ETH ≈ 5 min · SOL ≈ 1 min · LTC ≈ 15 min

## Live-web RAG: common questions

### Can I use LangChain or LlamaIndex?

Yes. The [LangChain and LlamaIndex page](https://llmwebsearch.com/docs/langchain/) wraps the search as a retriever and as a tool.

### Can I retrieve from URLs I already know?

Yes, with the [extract endpoint](https://llmwebsearch.com/docs/extract/). Reading pages whose URLs you already have costs 1 credit for each started group of 3 pages returned with content, up to 20 URLs per call; refused or failed URLs are not counted.

### Does LLMWebSearch store the pages it reads for me?

No. Each call leaves one log line: the time, the account, the first 60 characters of the query, how many results and pages were returned, and the credits. Lines are deleted after 30 days. The content of the pages we read is not kept. Account number, orders, top-ups and credit totals stay while the account exists, then only as long as accounting law asks. Authorities receive records only when a law compels us.

### Can I restrict retrieval to one language or country?

Yes. Set `language` and `country`, for example `language=de` and `country=de` for German results from Germany.

---

LLMWebSearch: LLMWebSearch is a web search API for AI agents that returns ranked results with the content of the top pages in Markdown, paid in crypto with no ID upload. Prepaid credits from $1.20/1K searches never expire. No KYC — no ID, no selfie, no documents. Pay in crypto.