Try
5,000 searches
$9.25
$1.85/1K searches
- Wire it into one agent
- Same API as the largest pack
- Credits never expire
Quick answer
How do I do RAG with live web search results?
Run a web search at question time, read the top pages, split them into chunks, keep the most relevant ones and ask the model to answer with numbered citations. LLMWebSearch handles the first two steps in one call: ranked results with the first 3 pages as Markdown, for one credit, from $1.20/1K searches. No KYC; pay in crypto.
What is live-web RAG?
It is retrieval-augmented generation where the retrieval step is a web search instead of a lookup in your own index. The question goes to a search API, the API returns pages, and the model answers from those pages.
The pattern suits questions about things that change: prices, releases, rules, schedules and news. It also suits long-tail questions your own documents never covered. The cost is one search per question, or two when the first one misses.
When should I fetch fresh and when should I store?
Store what is stable and yours; fetch what changes. Product manuals, internal policies and your own docs belong in a vector index, refreshed on a schedule. Competitor prices, regulations and news belong to live retrieval.
Many teams combine both. They search their own index first and call the web only when the index has no good match, or when the question mentions a recent date.
If you decide to keep pages you retrieved, note our position: We do not train any model on your queries or on what the API returns, and we add no rule of our own against storing results; the rights of the sites whose pages you read still apply. Check each site's terms before building a permanent copy of its content.
How do I turn search results into chunks?
Split each page on its headings, then cut long sections to a fixed length. Markdown makes this easy, because headings are lines that start with #. Keep the source URL and title on every chunk so citations survive the split.
The example below searches, reads three pages, chunks them and picks the six chunks that share the most words with the question. Replace the word count with your embedding model in production; the structure stays the same.
import os
import re
import requests
API = "https://llmwebsearch.com/api/v1/search"
HEADERS = {"Authorization": "Bearer " + os.environ["LLMWEBSEARCH_API_KEY"]}
def live_chunks(query, read=3, chunk_len=1200):
"""Search, then cut each page that was read into heading-based chunks tagged with its source."""
body = {"q": query, "max_results": 5, "read": read, "max_chars": 12000}
results = requests.post(API, headers=HEADERS, json=body, timeout=60).json()["results"]
chunks = []
for r in results:
if r.get("content_status") != "ok":
continue
for section in re.split(r"\n(?=#{1,3} )", r["content"]):
section = section.strip()
for start in range(0, len(section), chunk_len):
chunks.append({"url": r["url"], "title": r["title"], "text": section[start:start + chunk_len]})
return chunks
def top_chunks(chunks, question, k=6):
"""A plain keyword score; swap in your embedding model for better ranking."""
words = set(re.findall(r"\w{4,}", question.lower()))
def score(chunk):
return sum(1 for w in re.findall(r"\w{4,}", chunk["text"].lower()) if w in words)
return sorted(chunks, key=score, reverse=True)[:k]
def build_prompt(question):
picked = top_chunks(live_chunks(question), question)
urls = list(dict.fromkeys(c["url"] for c in picked))
blocks = [f"[{urls.index(c['url']) + 1}] {c['title']}\n{c['text']}" for c in picked]
sources = "\n".join(f"[{i}] {u}" for i, u in enumerate(urls, 1))
return (
"Use only the context to answer and cite sources as [n].\n\n"
+ "\n\n".join(blocks)
+ f"\n\nSources:\n{sources}\n\nQuestion: {question}"
)How many pages should each question read?
Three is a good start, and it keeps the search at one credit. Most factual questions are answered on the first two or three pages of a good query.
Ask for longer pages than you would in a chat agent, because the chunker will discard most of the text anyway. A max_chars of 10,000 to 15,000 per page gives the ranker enough to choose from. If the answer depends on one known site, add it with include_domains rather than reading more pages.
How do I make the model cite its sources?
Number the sources in the prompt and ask for the numbers in the answer. A short instruction works well: use only the context, cite each claim as [n], and say when the context does not cover the question.
Keep the source list at the end of the prompt, with one URL per number. Then render [n] as a link in your interface. Users trust answers they can check, and wrong citations are easy to spot in review.
What if a page is blocked or empty?
Skip it and use the others. Every result carries a content_status; only ok results have text. A page may be missing because Our reader respects robots.txt: it introduces itself as LLMWebSearch, follows the rules a site writes for it or for all robots, and leaves out the content of any page it is not allowed to read.
When all pages of a search come back empty, retry once with a different query or add include_domains for a site you know is open. When a site turns our reader away, the result comes back without content and with the site's status: we never solve CAPTCHAs, sign in to accounts or work around a block or a paywall.
What does live-web RAG cost?
One search per question in most cases. A support assistant that answers 5,000 questions a month, with a second search for one question in five, uses about 6,000 searches. 25,000 searches at $41.25 covers four such months.
Your model also reads the chunks you send it. Six chunks of 1,200 characters add roughly 2,000 tokens per question, which you control through the chunk size and count.
Which pack suits a RAG workload?
One credit per question for most pipelines. Search credits have no expiry date: what you do not use this month is still there next year.
Builder
25,000 searches
$41.25
$1.65/1K searches
- A coding agent or a research assistant
- MCP, LangChain and n8n ready
- Credits never expire
Production
100,000 searches
$145
$1.45/1K searches
- Agents in production
- Higher rate limit on request
- Credits never expire
In every plan: Top 3 pages read as Markdown · Country, language, time and domain filters · Failed calls not charged · Credits never expire
Live-web RAG: common questions
Can I use LangChain or LlamaIndex?
Yes. The LangChain and LlamaIndex page wraps the search as a retriever and as a tool.
Can I retrieve from URLs I already know?
Yes, with the extract endpoint. Reading pages whose URLs you already have costs 1 credit for each started group of 3 pages returned with content, up to 20 URLs per call; refused or failed URLs are not counted.
Does LLMWebSearch store the pages it reads for me?
No. Each call leaves one log line: the time, the account, the first 60 characters of the query, how many results and pages were returned, and the credits. Lines are deleted after 30 days. The content of the pages we read is not kept. Account number, orders, top-ups and credit totals stay while the account exists, then only as long as accounting law asks. Authorities receive records only when a law compels us.
Can I restrict retrieval to one language or country?
Yes. Set language and country, for example language=de and country=de for German results from Germany.