I cannot stand the idea of this. I know it's coming, I know I can't stop it, but it will be a catastrophic loss.
Most people are bad at writing. This will make them "better" at writing, but only in a certain way. I love reading things people write, because it gives me a window into their mind, how they think, who they are at a deep level. That is the joy of reading, and the structure of writing is a big part of that.
Now, a lot of people's writing has become homogenized by the computerization of our world already, but only at a low level: spelling, basic grammar, and so on.
When people have the ability to inpaint whole paragraphs, dreamed from the blob of internet text (which is mostly corporatized, computerized, email-ized, sterile in the way described above) we will lose something essential.
And another problem arises. I send an email asking something, to a coworker or to a friend. They inpainted their response. Did they really understand what I was asking? Did we really communicate at all?
Images, I guess, suffer from the same problem. But images are less interpersonal. They are communication, but not communication like writing is communication. In nearly all circumstances, images are less subtle than words are (the subtlety of visual communication happens irl, where such machines have yet to insert themselves).
A century or so ago as business correspondence became standardised, what emerged were form letters and templates for various correspondence.
I'm not sure when this practice started, and it may have gone by several different terms --- "sample business letters", "model business letters", and "handbook of business letters" are terms I'm coming up with.
What all of these address, however, is the fact that most people aren't good at composing correspondence, and/or that businesses benefit by standardised forms of communications.
Worse yet, not only will the 'in-painting' obscure the writer's meaning, or even whether they even had meaning, it will ALSO render that meaning more generic, and eliminate the most information-dense surprising bits.
All of these tools are merely synthesizing new text/images/code/etc. from millions of existing examples. When there is any doubt about the intent or the output, it fills in the MOST EXPECTED output. It does NOT fill in the possibly intended but unique image, phrase, code that would be the output of a unique insight. Real brilliance will be simply lost in the sauce.
Ugh. I won't be using these, and I'll shun those who do.
Right - in terms of information theory, automatically-generated infilled data communicates zero bytes of additional information. It could be compressed away entirely losslessly and I could instead generate it myself. There is no need to waste bandwidth sharing it.
Communication only has value if it is surprising. That much has been known since Shannon.
Language models don't pick the top choice all the time, and even picking the top choice is some amount of information, more precisely -p_token*log(p_token). They return a distribution we sample from. Sampling adds surprise and detail that was not originally there. Even more, if you have some sort of verification going - a second model to rate the outputs of the first, a math verifier or code executer when available - then language models can surprise us with interesting solutions to our tasks.
That is great when we are looking for interesting solutions. for example, it looks like GPT3-type systems might be very good search engines for obscure topics, getting them to finish paragraphs with their enormous search base. I've found a few tantalizing bits that I couldn't find on Google or DDG searches.
But, when they are trying to express what WE are saying, it looks like a very lossy solution at best.
I've had a deposition taken with an "AI" stenographer, and it was horrific, frequently reversing the meaning of sentences I said, or replacing an uncommon name with a common name (e.g., "John Kemeny" replaced with "Jack Kennedy"). Of course the transcript LOOKS great, it doesn't have any of the "(unintellegibile)" notations of a human transcript. It also does not go back at break points and ask for proper spellings of names, addresses, etc. like a human transcriber.
This is in the context of a legal trial with consequences, and I'm horrified to see this kind of crap passing for usable products, and here we are looking to foist it off on the general public as writing tools. We're forking doomed by smart idiots looking to make a quick buck with novel "tools".
I don't know if this is quite correct. The inpainted text has zero bits of information only if you already have on your end the exact, very large model they would have used. If you don't, and they only sent their prompt rather than the full text, you're in for a bit of a download.
Put another way, their prompt is arguably a pointer to data buried in that model. You're raising the question of whether they should only send the pointer, or just cache the query result, in a sense, and save you the trouble of looking it up.
It's interesting to imagine what the interaction model for this will be -- is there a sense in which this auto-generation will be like collaborating, and so it just changes what we think of to constitute the process of creation?
And maybe that's not as bad as it seems right this moment. Once upon a time great painters actually made their own paints. These days we wouldn't think about that skill as in the necessary catalogue of an artist, and in a bunch of ways -- most ways -- we're the better for it. Perhaps something similar will unfold here.
You know, something interesting is happening to me here.
I've never been upset about Stable Diffusion, because as you say, once upon a time, you had to physically paint an image to be visually creative. Never posted a comment about SD with "I cannot stand the idea of this. I know it's coming, I know I can't stop it, but it will be a catastrophic loss." Now, at the suggestion that it might happen for writing, all the sudden it seems wrong! Someone else noticed this in the other thread about Copilot: HN is not upset about Stable Diffusion, but seems to frequently be upset about Copilot! I think you may be right actually -- calm down, it's just a tool.
I guess I want to revise my prior comment. I don't just want to talk to a machine. Just like I don't just want to see Stable Diffused images. The human part is essential, it's the heart of the thing. My intuition is that you're supposed to augment the human part, not replace it.
I totally get your (original) point -- I'm an (amatuer) fiction writer trying to make myself feel better, a bit. But everything evolves. Think of how Magnus Carlson trains for and plays chess, vs how it was historically done. No point trying to resist the forces loose in the world, be one of the first to embrace them. I'm curious what this means right this moment for an artist upon the advent of SD and the like.
The difference is that images are expected to support the message. They're art, figures, illustrations. If I send someone a generated image, I still picked the image. It's still my message. If I send someone generated words, I did not pick the words. It's no longer my message. I'm tasking a machine with choosing my thoughts. It's no longer augmentation.
Yes, that's the obvious cynical interpretation. But our beliefs aligning with our self interest doesn't automatically make us wrong, especially in areas where we have expert knowledge. It could just be that our interest aligns with the general interest.
It will add a new level of suspense. Did they use passive voice, because they are trying to distance themselves from the decision or did algorithm thought that was the best choice here.
I agree with you overall. It will likely have similar impact to grammarly and similar services.
Images have much less of this problem. They are not the raw result of cognition. So changing images doesn't change someone's voice.
Instead images are always interprative. The camera doesn't capture what the eye sees. Often it takes a lot more interpretation than what a camera does to make a JPEG to end up with something like what the eye saw. In that process you can also change an image into something your eye would enjoy seeing.
Alternatively you can lie and say 'this is what I saw' about something that is not even close. Once images are used to promote falsehoods, your worry becomes true. But much editing, either for accuracy or for beauty, has no such harmful effects.
People go jogging and ride bicycles, even though cars exist, because they enjoy it and to tone their bodies. It's more effort, but it does things for them the car does not do, including provide thrills or promote well-being.
That is to say, these tools will exist and they will be used. But people will still write and make art - often perhaps using these tools in some way - because they'll have the time and find it rewarding to do so. And others will also assign some value to this. We might briefly get addicted, and then have a society-wide discourse on what healthy use is, similar to social media.
Isn't it compelling to wonder what humanity will decide to do with technology when technology were to be limitless? As in, what essentially human choices will we make in what, when and how to use technology? (Singularity-themed scifi tries to provide some answers since the 90s.)
Yes, and I mean, while you could argue that jogging or riding bicycles is a form of self expression, I don't consider them so the same way I consider writing to be so. Cars help you get from point A to point B in a way that is qualitatively different than walking; but both are ontologically different from writing, because writing is not a thing that a single person does -- it's an act of communication. Writing and reading, and the dynamic of both playing out among people, is related to the formation of the self for both parties in the deepest and most crucial way.*
Easier put: the joy of writing and the joy of reading are strongly linked. I don't write things with the expectation that nobody will read them, and I don't read things with the expectation that nobody wrote them. Or, in this case, a machine.
And I think a "writing photoshop" will be much harder to detect than an image Photoshop. Did I picture the author as a white collar Yale graduate because they are one, or because the machine told them that was best?
* For example: I would sooner give up my ability to walk than give up my ability to communicate myself to others; that's the difference I'm talking about.
What do you think about text transformations like the ones described in the essay? Not inpainting but tools that could help “good” writers, like changing narration in a chapter from first person to third person in one step?
Changing perspective is not as easy as changing personal pronouns. It also affects the knowledge of a character as well. First person perspective has insights of the main character's inner thoughts and feelings, since they are the narrator, third person has not. Then there is third person limited and third person omniscient, which require totally different approaches and need to have different twists and turns to be belieavable. Changing the pov basically requires total rewrite of the story.
The authors of most shared articles and most comments are not even passing a “turing test”. In the vast majority of cases the readers just consume the data.
With GPT-3 we can already make “helpful and constructive” seeming comments that 9 out of 10 times may even be correct and normal. But 1 out of 10 times be kind of crappy. Aby organization with an agenda can start spinning up bots for Twitter channels, Telegram channels, HN usernames and so on, and amass karma, followers, members. In short, we are already past this point: https://xkcd.com/810/
And the scary thing is that, after they have amassed all this social capital, they can start moving the conversation in whatever directions the shadowy organization wants. The bots will be implacable and unconvinced by any arguments to the contrary… instead they can methodically gang up on their opponents and pit them agaisnt each other or get them deplatformed or marginalized, and through repetition these botnet swarms can get “exeedingly good at it”. Literally all human discussion — political, religious, philosophical etc. - could be subverted in this way. Just with bots trained on a corpus of existing text on the web.
In fact, the amount of content on the Internet written by humans could become vanishingly small by 2030, and the social capital — and soon, financial capital — of bots (and bot-owning organizations) will dwarf all the social capital and financial capital of humans. Services will no longer be able to tell the difference between the two, and even close-knit online societies like this one may start to prefer bots to humans, because they are impeccably well-behaved etc.
I am not saying we have to invent AGI or sexbots to do this. Nefarious organizations can already create sleeper bot accounts in all services, using GPT-4.
Imagine being systematically downvoted every time you post something against the bot swarm’s agenda. The bots can recognize if what you wrote is undermining their agenda, even if they do have a few false positives. They can also easily figure out your friends using network analysis and can gradually infiltrate your group and get you ostracized or get the group to disband. Because online, when no one knows if you’re a bot… the botswarms will be able to “beat everyone in the game” of conversation.
Another reason the future of social media is invite-only. People obviously aren't invisible BS detectors, but if things really get that bad they're not just going to take it indefinitely. They'll notice and adjust.
By the way, if you are worried, it isn't true that this is "coming." A form of it will come but like self-driving cars, it will be more hype than substance.
Most people are bad at writing. This will make them "better" at writing, but only in a certain way. I love reading things people write, because it gives me a window into their mind, how they think, who they are at a deep level. That is the joy of reading, and the structure of writing is a big part of that.
Now, a lot of people's writing has become homogenized by the computerization of our world already, but only at a low level: spelling, basic grammar, and so on.
When people have the ability to inpaint whole paragraphs, dreamed from the blob of internet text (which is mostly corporatized, computerized, email-ized, sterile in the way described above) we will lose something essential.
And another problem arises. I send an email asking something, to a coworker or to a friend. They inpainted their response. Did they really understand what I was asking? Did we really communicate at all?
Images, I guess, suffer from the same problem. But images are less interpersonal. They are communication, but not communication like writing is communication. In nearly all circumstances, images are less subtle than words are (the subtlety of visual communication happens irl, where such machines have yet to insert themselves).