I keep waiting for the post explaining that this is a joke

Well the name Pion translates to Pawn in French. So that's a good start.

Like polsia is aislop backwards?

I thought it was a joke and I laughed out loud

I always wonder who the hell is clicking these ads. I skip all you tube ads (thanks Adblock)… I never click banner ads on principle. I assume anything paying for my eyeball time is a scam.

So who is this shit working on? These companies must have detailed profiles on “people who do what we tell them” as a whole class of people. Scary.

My elderly parents regularly click the sponsored links in Google search results.

I don't think it even registers for them that they're ads (even though I have informed them multiple times).

Same for sponsored items in Amazon search results. As an adblock user it's really jarring to see their experience of the web compared to my own.

Honestly I wouldn't be surprised if the majority of actual end customers/humans (i.e. not bot farms or paid for actors) who click are not clicking on purpose. I've seen so many bad Android apps that have performance issues where content is still rendering and by the time you've tapped on where you go an add banner/link loaded up and registered you clicking on that. Zero purchase ever done but numerous 'ad clicks' generated

Can someone define “improvement” in this context? This concept feels like a buzz word otherwise.

Improvement means being able to do more complicated things more reliably. Relatedly, it means being able to learn to do new things with fewer and fewer examples. We are running out of easily verifiable or simulation-friendly or data-rich domains for LLMs to conquer. (Note that I didn't say "simple" or "easy" domains.)

I suppose the next (and more risky) step is to let AI conduct its own real-world experiments, so that it can generate data to learn more physical and social properties.

It is doing that already - every day 1B people or more use AI for the tune of a few trillion tokens. Imagine that much language flowing between brains and AI agents. It carries real our world problems to AI, their solutions back to us, and we act in the world and come back for more AI iteration. In the end AI gets inside the loop of real world actions and their consequences. AI logs stretch over years, tracking downstream effects. Hindsight can be used to track consequences of prior actions. It's a real data loop, an experience engine.

This same process has recently been under scrutiny when mathematicians claimed AI companies trained on their unpublished logs and later claimed merit for results. But the exchange of experience happens across all domains. Experience gets generated at amazing rates, and absorbed by models which get applied everywhere, collecting more experience.

That sounds terrifying.

Which means someone is already working on it.

Models that are strong enough to improve themselves without a human (I.e. ai researcher) in the loop

The trivial idea that there are some benchmarks for a tool and there can be even better inexplicit ones, and if the tool is capable of modifying itself towards better benchmarks results there could be a recursive climbing of those benchmarks through iterations of tool versions which are outputs of former tool versions.

Is this shocking? Grokipedia is an opaque system designed to confirm the opinions of its owner. Why should we expect any kind of coherent transparency?

This. It’s not even worth discussing.

As long as there’s a trade off between experimentation and performance, software will always be slightly too slow.

And now the other aspect of the trade off is token budget allocation.

The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.

(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)

I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.

This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).

I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)

These are all bots and shills

Voting with your wallet can work, but consolidation increasingly makes voting with your wallet a double or triple whammy.

1) alternatives may be more expensive by dollars

2) alternatives may be more expensive by time

3) time/money lost to altered consumption has proportional opportunity cost

Market consolidation reduces the cost of using “bad” services and increases the cost of using “good” services over time.

Or a shot in the back.

Not if your spells cast their own spells.

Read the article.

They are saying very clearly the models are not casting their own spells…yet. But looking at trends and speculating when they may start doing so.

How does he stay awake??