One of the humans mentioned in the article, Sebastian Rowan, a Ph.D. candidate at the University of New Hampshire, is quoted as saying “But a fundamental problem with using AI, even specialized tools, for anything is their tendency to hallucinate, which as far as I know is believed to be an unsolvable problem.”
I don’t believe that’s an unsolvable problem.
I’ve done some private experiments where I had Claude, over many sessions, design and implement original research projects in fields in the humanities and social sciences, with a research paper as the final product of each project.
I included detailed instructions on research ethics in CLAUDE.md and other project documentation, insisting, among other things, that it double- and triple-check all citations and references to make sure not only that the referenced papers existed but also that they actually said what the paper Claude wrote claimed they said. I gave Claude an OpenRouter API key and a daily budget, and it outsourced much of that adversarial checking to models made by other companies. In my checks of its final papers, I found no instances of hallucinated or otherwise invalid references. At various steps in the research process, Claude also, on its own initiative, had other models critique its research designs, data analyses, written drafts, etc., and it made changes based on that feedback.
The final papers were, to my eye, well done, if quite narrow in their scope and claims. The ethical alignment I imposed on it, as well as its lack of ego, seemed to keep it from making even weak claims if they could not be rigorously justified.
I’m a retired academic, and I no longer have any need or desire to produce research papers myself, with or without AI. Rather, I am interested in the impact of AI tools on the research process, particularly on how research and researchers will be evaluated in the years ahead.
I did not, and will not, release any of those papers written for me by Claude, though I have released what it produced for another research project I had it do, on a topic in the philosophy of language. In case anyone is interested:
> insisting, among other things, that it double- and triple-check all citations and references to make sure not only that the referenced papers existed but also that they actually said what the paper Claude wrote claimed they said
So your solution to get it to not hallucinate was to ask it not to hallucinate? And you suppose that this was a novel approach? What, that people just hadn’t tried that before?
I didn’t ask it not to hallucinate. I had it prompt other LLMs to do web searches to confirm whether the citations, references, and other information that it had initially produced were correct.
I certainly don’t claim that this technique was original. There have been many reports of people using similar methods:
“CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era”
I also don’t claim that such techniques are 100% effective. While the chance of different LLMs producing the same hallucinated reference is extremely small, they still rely on bibliographic information found on the web being accurate. But a human checking a paper’s references for accuracy is likely to rely on the same sources.
I don’t believe that’s an unsolvable problem.
I’ve done some private experiments where I had Claude, over many sessions, design and implement original research projects in fields in the humanities and social sciences, with a research paper as the final product of each project.
I included detailed instructions on research ethics in CLAUDE.md and other project documentation, insisting, among other things, that it double- and triple-check all citations and references to make sure not only that the referenced papers existed but also that they actually said what the paper Claude wrote claimed they said. I gave Claude an OpenRouter API key and a daily budget, and it outsourced much of that adversarial checking to models made by other companies. In my checks of its final papers, I found no instances of hallucinated or otherwise invalid references. At various steps in the research process, Claude also, on its own initiative, had other models critique its research designs, data analyses, written drafts, etc., and it made changes based on that feedback.
The final papers were, to my eye, well done, if quite narrow in their scope and claims. The ethical alignment I imposed on it, as well as its lack of ego, seemed to keep it from making even weak claims if they could not be rigorously justified.
I’m a retired academic, and I no longer have any need or desire to produce research papers myself, with or without AI. Rather, I am interested in the impact of AI tools on the research process, particularly on how research and researchers will be evaluated in the years ahead.
I did not, and will not, release any of those papers written for me by Claude, though I have released what it produced for another research project I had it do, on a topic in the philosophy of language. In case anyone is interested:
https://tkgally.github.io/ai-semantics/