LFS is not OS development. You build a distribution by assembling all the components. If you want to develop an OS from scratch, perhaps you're looking for osdev.org.
When the sole focus is profit and to crush enemies and not to care for the people, the world will start to look like it's full of enemies and profit. It's a self perpetuating, self inflicted situation.
> your prompt is confusing, unfocused, and doesn't work right on any LLM
You are assuming the entirety of the prompt is human prose, but it could be sets of data so the agent doesn't have to collect it every time, like program interfaces, commands, views, databases, tables, data models etc...
I could see this scale to multiple kiltobytes of metadata in the prompt easily.
That doesn't make much sense to me because this is in nature much like how harnesses operate: launch a bunch of exploratory subagents to search and retrieve evidence to use in the actual prompt. Think of it as caching this end result so you don't have to re-fetch in the codebase.
That's the other way of doing it, which solves the context rot problem in a more complex way. The model at the top says, "hey, sub-agent, go figure out the answer to this question and give me the answer", and that sub-agent can go consume 250k+ context to return an answer that might be a couple of words, and thus not contaminate the main context with that now thrown-away context.
However, this is not something that is inherently part of models or inference engine, but part of the harness.
Harnesses are very hit and miss, and are not integrated into the stack, and I think that will have to happen eventually. Like, conceptually similar to an LLM performing a tool call that just calls itself recursively, I think this would go a long way to making LLMs more viable for being an actual product people could conceivably want.
You’re right about the harness being the issue. It’s really down to giving it functions specific to your use case that will let it surgically read/modify files, rather than needing to consume entire project folders. I built my own and for Python files some of the most helpful functions I provide are equivalent to:
inspect_function(filename, function, class)
replace_function(<same>)
call_graph(<same>)
And a few other convenient ones. Beyond that it’s trickery like if a function returns more than N lines I omit the result and auto-reply “Your function call was too verbose.” Typically that’s stuff like recursively listing every file in a repo to “see what it’s working with” or similar. When it emits the next call in response to it I clip the previous attempt (and my response) off the conversation and attach the new call/result as if that’s what it did in the first place. I also log that event so if the same type of thing happens often enough I’ll create a special function to address it, or modify an established one so it’s not tempted to do it again.
Language models don’t know what they know, they know what has been said. Even if you give it an entire Python environment it won’t reach for AST, but if you give it a function called Python_AST() it’ll use it every time.
I’ve never seen an off-the-shelf harness that approached it that way.
But usually verifying the answer is cheaper than finding the answer, so it can be done. The prompt given to the sub-agent can be a question with "and explain your answer", the answer to which will still consume far less context than figuring out the answer.
Of course it has to. The subagent might have misunderstood and spent 250k tokens smoking crack. If that output is blindly trusted the parent agent might go on to burn millions of tokens going in the wrong direction, completely negating the point of using a subagent in the first place.
Only if you think in terms of perceived raw intelligence, but self-update is a form of valuable self-improvement that could benefit current models a lot, if they could commit facts from context into their weights cheaply and reliably.
All you can really say is that negligence is corrosive to civilization, and maybe that drugs act as an amplifier of sorts for society's illnesses and qualities.
Right, and the elites can deal with it them selves, they have money. They are definitely not directly responsible for the "subterranean environments covered in art and strewn with drug paraphernalia".
It's not solved, I agree. But if we pretend it is we can prepare for when it actually becomes solved, if ever, and measuring sloppiness is a worthy pursuit even if we never "solve code".
reply