Absolutely surreal. Thank you for sharing, this is very nice timing, as I'm giving a presentation on the basics of "AI" at a library tomorrow, and have been looking for a good edge case conversation - and this is it! I've saved the text as well, in case OpenAI deletes it.
Sent the Monty Hall one verbatim to OpenOrca for the lulz and man these are some olympic gold winning mental gymnastics:
> You initially choose door number one, because you want the car and can see it behind that door.
> Monty Hall opens door number two (which has a goat) and shows that there are no more goats left. Since you already saw what was behind door number one, this doesn't change anything about your initial choice of doors.
> The fact that door number two is opened does not affect the probability of the car being behind either door number one or three (since both have been hidden from Monty Hall). However, it does give you new information: since there are no more goats left to be found and only one door can hide a car, the other remaining door must contain the car.
> Therefore, with this new information, your best option is to switch doors and choose door number three, as that's where the car must now be located given what you know about Monty Hall's choices.
> It is important to remember that the car cannot magically move between doors after being seen or hidden; it always remains in its original position behind one of the doors. The information provided by Monty Hall simply helps you make a more informed decision based on your initial choice.
I love these open source models, they're an endless source of hilarity. Not much use for anything else though :P
It’s an iterative, semi-deterministic process. It’s simply a place where context and generation lead to weird outputs.
You can get similar outputs by asking OpenAi to repeat a number 100 times. It will eventually get into some weird, low probability paths and generate non-sense output.
This type of complete garbage is not uncommon in AI. It's simply the nature of asking a non-intelligent system to generate human readable content.
Maybe this is a different way to think about it. In most of the country, your cellphone has _amazing_ coverage. It can talk clearly with a cell tower. Your data and calls work perfectly.
In some parts of the country, you're going to have no service. Your cell phone won't work. It doesn't have cell towers to talk to.
At the intersection of service and no-service, you'll find an area where your cell phone works sporadically. It might barely have 1 bar of service. You might need to hold your phone a certain way. It will work seemingly randomly. Calls might have a few words go through.
That edge of service is essentially where the LLM is at. Its in an internal state where it has enough signal to attempt to generate a response, but not a strong enough signal to generate a meaningful response. It ends up falling back to something it's "memorized".
I find your example surreal as well... I get the surreal feel from these kinds of technological liminal spaces where hard and fast rules break down into seemingly black magic.
"You might be able to get cell service by holding your phone differently. Try waving it randomly around the room, one corner might work better than others."
"The USB stick enters on the third try."
"An iterative semi-deterministic bag of matrix multiplications can convincingly communicate. Undefined behavior appears schizophrenic."
On an intellectual level, I get it, but it's still fuckin' weird.
Which bolsters the claim that it doesn't really understand what it's generating in a way humans understand language. Sure, it can be said to have a kind of understand for the rules of language it's been trained successfully on, but when you get it in a weird place, that understanding breaks down in a way it doesn't for people.
I tend to agree with the sentiment here, but reading the output made me think of disorders like schizophrenia, which is to say sometimes humans enter this mode too.
For a different reason though having to do with a brain disorder where they have trouble distinguishing their thoughts and imagination from perception.
LLMs generate one token at a time, so anytime they happen to generate some garbage for any reason, you get "garbage in" for further tokens for as long as it stays in the context window.
Not totally sure, but based on other folks' comments a number of external sources of noise are possible, e.g. from other conversations or from hitting a weird spot in the training data.
