Top Highlights
-
The agent didn’t clearly beat plain search: it found the right paper in the top five for 96.9% of questions versus 95.6%—a difference the test couldn’t distinguish from chance.
-
A simple reranker was the standout: it moved relevant papers higher in the results, with a clear improvement in ranking quality and no model fees.
-
More searches brought trade-offs: the agent sometimes rescued missed papers but also pushed relevant ones down; rewriting questions performed worse than plain search in this test.
-
Bottom line: On 298 questions about recent arXiv papers, extra agent searches weren’t clearly worth their added calls and delay. The test measured retrieval—not the quality of final answers.
Does Agentic RAG Actually Retrieve Better Evidence? I Tested It on 298 Questions
More searches did not guarantee better evidence
Retrieval-augmented generation, or RAG, gives an AI system documents before it writes an answer. Agentic RAG adds a step: An AI agent reviews search results, then decides whether to stop or search again.
That sounds promising. But every extra search can add cost and delay—and may send the system in the wrong direction.
I compared four search methods on 298 questions and 2,000 recent research papers. Plain search looked up each question once. An agent could search up to three times. I measured whether the correct paper appeared among the first five results. None of the methods wrote an answer.
The reranker led; the agent’s gain stayed uncertain
The agent found the right paper in its top five for 96.9% of questions. Plain search did so for 95.6%. That small difference fell within the range where chance could explain the result. The agent also found some papers plain search missed, while losing others it had found.
A question-rewriting method performed worse, reaching 93.6%. The clearest improvement came from a reranker. It reordered the first 50 search results and moved the right paper higher. It improved ranking quality without language model fees, though it added about 1.5 seconds of delay on the test laptop.
Test before you add an agent
The results offer a practical lesson: Start with your own questions. If plain search already finds the right evidence most of the time, an agent has little room to help. Then test a reranker before adding more model calls.
Still, this test covered only research abstracts and questions written from those abstracts. Real users may ask shorter, messier questions. The test also measured retrieval, not the quality of final answers.
Agentic RAG may help in other settings. But here, extra searches did not clearly retrieve better evidence. Measure the benefit—and the added delay and cost—before adopting it.
Discover More Technology Insights
Explore the future of technology with our detailed insights on Artificial Intelligence.
Explore past and present digital transformations on the Internet Archive.
AITechV1
