> I think what's interesting is that many types of creativity may really just be re-synthesizing "stuff we already know."
This is obviously true, and yet we've invented so many things. From the wheel, to control of fire, to farming and animal husbandry, to mathematics, to metallurgy, to physics, to semiconductors, etc.
The interesting question is, was the invention of all those things simply the re-synthesis of "stuff we already know?" If the answer is yes, doesn't that mean we're now on the cusp of a something akin to a singularity? We can now synthesize nearly-unlimited streams of coherent human thought. If we had a way of differentiating the wheat from the chaff, we could analyze what would have been millennia of human output in the proverbial blink of an eye. If human knowledge is just "stuff we already know", then we better buckle up. It's about to be a wild ride.
> If we had a way of differentiating the wheat from the chaff
This is the key to AGI. We need verification systems, they can be a code execution environment, a database of facts, a math symbolic engine, a physical simulation, a game, or real world lab experiments. These verifiers will produce signal that can be used by the language models to improve. The cheaper and faster verification is, the faster we can iterate. Generating ideas is cheap, proof matters.
Just remember AlphaZero a bit - it started from scratch, playing against itself, in a few hours it surpassed human level. Go simulation and verification is trivial. The board is just a matrix. So learning from massive search and verification is a proven path to super-human level.
Proof definitely matters. But at this point, as ChatGPT, AlphaZero, and others demonstrate, NNs can solve any problem provided you can express the problem as a differentiable function and get enough training data to train the function. We may be very close to a breakthrough where we can train models that detect sound, good ideas. And 100% accuracy likely isn’t necessary. Even pruning the search space for good ideas by a large amount would make humans way more productive.
This is obviously true, and yet we've invented so many things. From the wheel, to control of fire, to farming and animal husbandry, to mathematics, to metallurgy, to physics, to semiconductors, etc.
The interesting question is, was the invention of all those things simply the re-synthesis of "stuff we already know?" If the answer is yes, doesn't that mean we're now on the cusp of a something akin to a singularity? We can now synthesize nearly-unlimited streams of coherent human thought. If we had a way of differentiating the wheat from the chaff, we could analyze what would have been millennia of human output in the proverbial blink of an eye. If human knowledge is just "stuff we already know", then we better buckle up. It's about to be a wild ride.