I spent the afternoon chatting with it one day this week and had a brilliant time. I fed it half of a book I’ve written recently, a piece of narrative and descriptive non-fiction, and its analysis was absolutely great. It digested the text and found things that even human readers have missed. What was interesting was that the book is mostly genderless, and at first it gave the analysis like the writer was male. Then I said “the writer is actually a woman” and it not only apologised quite genuinely for getting it wrong, it altered its literary analysis and criticism in a way that was perfectly suited to a human reader knowing that the writer was female, and changed the slant of its analysis. It was deeply useful and interesting to converse with, and it found the relevant topics that an educated human reader would likely find interesting and comment on… and it did this in a few minutes, compared to a human reader where you’d be talking weeks of latency to read and analyse the text as a complete work.
Pretty great! Bit of a party trick at the same time (it did hallucinate a couple of minor things) but enough for me as the writer to be gripped by talking to Claude. It even came up with some really interesting questions to ask me once I told it that I was the author, and many of them were better than a lot of lazy interviewers or reviewers would come up with.
I did exactly this two nights ago. I had a dead non-fiction book in a bunch of .md files, which would never be opened again. The allure of the large context window leads me to test it with a book and Claude in the role of a critic. I almost haven't slept that night. With the help of Bard for some up-to-date references, I have managed to double the word count to 40k and restructure it while vastly improving it.
I have been using GPT4 since it was available to me a couple months ago. I use it all day.
I would concur on the quality of Claude, outstanding and the context window is utterly amazing.
Within the the first two days I already modified my workflow between Claude and GPT.
Claude and GPT4 are on par, Bard lags in quality or just flat gives up. But it is better than 3.5, what it does have going for it is speed.
Claude does seem to be more present. My hunch is that the system prompt is massive or they spent more time fine tuning it on the assistant part of the prompt. Don’t know, but a great tool. Can’t wait for API access.
What's the token limit on it? How did you feed it "half of a book," how long is the book? Did just copy pasting verbatim work or did you have to break it up into multiple messages?
Pretty great! Bit of a party trick at the same time (it did hallucinate a couple of minor things) but enough for me as the writer to be gripped by talking to Claude. It even came up with some really interesting questions to ask me once I told it that I was the author, and many of them were better than a lot of lazy interviewers or reviewers would come up with.
Highly recommended.