I indexed thousands of documents into a SQLite database with an FTS5 index, plugged it into DeepSeek v4 Flash and got better much better results than any other commercial solutions my company has tried in the past.
The trick was just to let the LLM come up with its own SQL queries for searching... and the results are impressive.
Just wondering on the reported accuracy of the SQL queries generated by SQL based on RAG. Reports have been not encouraging and also confirmed by these results but perhaps your methodology is different from these reports [1],[2],[3].
>On this benchmark, a pure LLM generated an accuracy score of zero. Adding RAG, prompt engineering, and agentic AI raised accuracy to the 10+% range.
[1] Any text-to-SQL benchmark should address difficulties of real-world data stores (acm.org) (21 comments):
I'm now using this approach too, and it feels better than any sorting method I've used before. The only thing I'm thinking about now is: if the LLM makes a mistake, how can I provide feedback and verify it?
I log all toolcalls to a file, I think others have said the same. But I'm a bit leery of letting a hallucinating LLM write SQL queries. I think most I've spoken with, agree that an LLM is like a 20 year old, eager intern. Well meaning, but left unrestrained capable of immensely inexperienced mistakes.
Before a lot of frameworks existed, you'd see DEVs taking user input on a web form, and then just throwing it directly at the MTA. So spammers could submit email@address\nCC: persontospam@address, and the like.
Now LLMs are a different beast, but you have input validation for LLMs, unique to all other validation methods. Yet there's actually no safe way to ever validate user input for a LLM, except for very rigid input validation on single words. Take the email example above. You'd need a regex to only validate an email address (and that isn't simple), but once you expand it to actually allowing sentences?
The LLM is now input validation vulnerable.
And that means no user input can be used in unvalidated commands.
And then just random hallucinations. I'm curious how the gp managed weirdo LLM behaviour, like out of the blue 'drop table' or accidental select into as opposed to just select.
Its a read-only SQLite file. And I mean the people using the chatbot knows it uses AI so just like Google they shouldnt pick the first result, but forcing the LLM to mention the sources and not assume acronyms works amazingly well
I have a web user interface connected to a coding agent (OMP) running inside a container, so if any of the tool calls fail or something happens, usually my model recovers autonomously from these situations. The capabilities of models like DS4 Flash are those of frontier models from months ago, so its recovery and autonomous capabilities are quite impressive.
Not really, I just made a really simple Python library the agent can use and modify as he wishes within a sandboxed environment to explore the SQLite database. When the user prompts the agent, the prompt goes into my coding agent, omp, and starts running Python scripts and throwing SQL sentences until coming up with the answer. It works extremely well in our experience.
The trick was just to let the LLM come up with its own SQL queries for searching... and the results are impressive.