This site.

Hold on to that spirit and I think you'll genuinely do well in the world that's coming next.

Apparently shale oil mostly comes out as light, so our own production doesn't feed our refineries and we've increasingly taken to importing heavy crude.

It is very clear when you look at academic papers actually targeting problems specific to reasoning / intelligence (e.g. rotation invariance in images, adversarial robustness) that all the big companies are doing is just fitting more data / spending more resources on human raters and other things to boost performance on (open) metrics, but that clear actual gains in genuine intelligence are being made only by milking what we know very well to be a limited approach. I.e. there are trivially-basic problems that cannot be solved by curve-fitting models, which makes it clear most current advances are indeed coming from curve(manifold) fitting. It just isn't clear how far we can exploit these current approaches and in what domains this kind of exploitation is more than good enough.

EDIT: Are people unaware Google Scholar is a thing? It is trivial to find modern AI papers that can be read without requiring access to a research institution. And e.g. HuggingFace collects trending papers (https://huggingface.co/papers/trending), and etc.

But the bizarre thing is, even though the productivity of SWE's is increasing I dont believe there will be much happening in regards to lay offs due to the fact that there isn't complete trust in LLMs; I dont see this changing either. In which case the LLM producers will need to figure out a way to increase the value of LLMs and get users to pay more.

And, again, this is ignoring all the technical debt of produced code that is poorly understood, weakly-reviewed, and of questionable quality overall.

I still think this all has serious potential for net benefit, and does now in certain cases. But we need to be clearer about spelling out where that is (webshit, boilerplate, language-to-language translation, etc) and where it maybe isn't (research code, legacy code, large codebases, niche/expert domains).

https://youtu.be/tbDDYKRFjhk?si=gF4EN4ilogoam3hG

I am hopeful about LLMs for SWE, but the progress is currently contextual.

Even if LLMs could write great code with no human oversight, the world would not change over night. Human creativity is necessary to figure out what stuff to produce that will yield incremental benefits to what already exists.

The humans who possess such capability stand to win long-term; said humans tend to be those from the humanities and liberal arts.

You can work around a lot of the memory issues for large and complex tasks just by making the agent keep work logs. Critical context to keep throughout large pieces of work include decisions, conversations, investigations, plans and implementations - a normal developer should be tracking these and it's sensible to have the agent track them too in a way that survives compaction.

- `FEATURE_IMPL_PLAN.md` (master plan; or `NEXT_FEATURES_LIST.md` or somesuch)

- `FEATURE_IMPL_PROMPT_TEMPLATE.md` (where I replace placeholders with next feature to be implemented; prompt includes various points about being thorough, making sure to validate and loop until full test pipeline works, to git version tag upon user confirmation, etc.)

- `feature-impl-plans/` directory where Claude is to keep per-feature detailed docs (with current status) up to date - this is esp. useful for complex features which may require multiple sessions for example

- also instruct it to keep main impl plan doc up to date, but that one is limited in size/depth/scope on purpose, not to overwhelm it

- CLAUDE.md has summary of important code references (paths / modules / classes etc.) for lookup, but is also restricted in size. But it includes full (up-to-date) inventory of all doc files, for itself

- If I end up expanding CLAUDE.md for some reason or temporarily (before I offload some content to separate docs), I will say as part of prompt template to "make sure to read in the whole @CLAUDE.md without skipping any content"

They refuse to believe that it's possible to instruct these tools in terse plain English and get useful results.

None, that's what I'm trying to say. My favorite is just storing project context locally in docs that agents can discover on their own or I can point to if needed. This doesn't require me to upload sensitive code or information to anonymous people's side projects and has and equivalent amount of hard evidence for efficacy (zero), but at least has my own anecdotal evidence of helping and doesn't invite additonal security risk.

People go way overboard with MCPs and armies of subagents built on wishes and unproven memory systems because no one really knows for sure how to get past the spot we all hit where the agentic project that was progressing perfectly hits a sharp downtrend in progress. Doesn't mean it's time to send our data to strangers.

FWIW, I find this eventual degradation point comes much later and with fewer consequences when there are strict guardrails inside and outside of the LLM itself.

From what I've seen, most people try to fix only the "inside" part - by tweaking the prompts, installing 500 MCPs (that ironically pollute the context and make problem worse), yell in uppercase in hopes that it will remember etc, and ignore that automated compliance checks existed way before LLMs.

Throw the strictest and most masochistic linting rules at it in a language that is masochistic itself (e.g. rust), add tons of integration tests that encode intent, add a stop hook in CC that runs all these checks and you've got a system that is simply not allowed to silently drift and can put itself back on track with feedback it gets from it.

Basically, rather than trying to hypnotize an agent to remember everything by writing a 5000 line agents.md, just let the code itself scream at it and feed the context.

One thing I’d clarify about what we’re building is that it’s not meant to be “the best memory for a single agent.”

The core idea is portability and sharing, not just persistence.

Concretely:

- you can give Codex access to memory created while working in Claude

- Claude Code can retrieve context from work done in other tools

- multiple agents can read/write the same memory instead of each carrying their own partial copy

- specific parts of context can be shared with teammates or collaborators

That’s the part that’s hard (or impossible) to do with markdown files or tool-local memory, and it’s also why we don’t frame this as “breaking the context limit.”

Measuring impact here is tricky, but the problem we’re solving shows up as fragmentation rather than forgetting: duplicated explanations, divergent state between agents, and lost context when switching tools or models.

If someone only uses a single agent in a single tool and already are using their customized CLAUDE.md, they probably don’t need this. The value shows up once you treat agents as interchangeable workers rather than a single long-running conversation.

I'm confused because every single thing in that list is trivial? Why would Codex have trouble reading a markdown file Claude wrote or vice versa? Why would multiple agents need their own copy of the markdown file instead of just referring to it as needed? Why would it be hard to share specific files with teammates or collaborators?

Edit - I realize I could be more helpful if I actually shared how I manage project context:

CLAUDE.md or Agents.md is not the only place to store context for agents in a project, you can just store docs at any layer of granularity you want. What's worked best for me is to:

1. Have a standards doc(s) (you can point the agents to the same standards doc in their respective claude.md/agents.md)

2. Before coding, have the agent create implementation plans that get stored in to tickets (markdown files) for each chunk of work that would take about a context window length (estimated).

3. Work through the tickets and update them as completed. Easy to refer back to when needed.

4. If you want you can ask the agent to contribute to an overall dev log as well, but this gets long fast. Is useful for agents to refer to the last 50 lines or so to immediately get up to speed on "what just happened?", but so could git history.

5. Ultimately the code is going to be the real "memory" of the true state, so try to organize it in a way that's easy for agents to comb through (no 5000 lines files that agents have trouble trying to carefully jump around in to find what they need without eating up their entire context window immediately).

Where it stopped being trivial for us was once multiple agents were working at the same time. For example, one agent is deciding on an architecture while another is already generating code. A constraint changes mid-way. With a flat file, both agents can read it, but you’re relying on humans as the coordination layer: deciding which docs are authoritative, when plans are superseded, which tickets are still valid, and how context should be scoped for a given agent.

This gets harder once context is shared across tools or collaborators’ agents. You start running into questions like who can read vs. update which parts of context, how to share only relevant decisions, how agents discover what matters without scanning a growing pile of files, and how updates propagate without state drifting apart.

You can build conventions around this with files, and for many workflows that works well. But once multiple agents are updating state asynchronously, the complexity shifts from storage to coordination. That boundary - sharing and coordinating evolving context across many agents and tools — is what we’re focused on and what an external memory network can solve.

If you’ve found ways to push that boundary further with files alone, I’d genuinely be curious - this still feels like an open design space.

> With a flat file, both agents can read it, but you’re relying on humans as the coordination layer: deciding which docs are authoritative, when plans are superseded, which tickets are still valid, and how context should be scoped for a given agent.

So the memory system also automates project management by removing "humans as the coordination layer"? From the OP the only details we got were

"What it does: (1) persists context between sessions (2) semantic & temportal search (not just string grep)"

Which are fine, but neither it nor you explain how it can solve any of these broader problems you bring up:

"deciding which docs are authoritative, when plans are superseded, which tickets are still valid, and how context should be scoped for a given agent, questions like who can read vs. update which parts of context, how to share only relevant decisions, how agents discover what matters without scanning a growing pile of files, and how updates propagate without state drifting apart."

You're claiming that semantic and temporal search has solved all of this for free? This project was presented as a memory solution and now it seems like you're saying its actually an agent orchestration framework, but the gap between what you're claiming your system can achieve and how you claim it works seems vast.

All of these systems are for managing context.

You can generally tell which ones are actually doing something if they are using skills, with programs in them.

Because then, you're actually attaching some sort of feature to the system.

Otherwise, you're just feeding in different prompts and steps, which can add some value, but okay, it doesn't take much to do that.

Like adding image generation to claude code with google nano banana, a python script that does it.

That's actually adding something claude code doesn't have, instead of just saying "You are an expert in blah"

another is one claude code ships with, using rip grep.

Those are actual features. It's adding deterministic programs that the llm calls when it needs something.

> Otherwise, you're just feeding in different prompts and steps

"skills" are literally just .md files with different prompts and steps.

> That's actually adding something claude code doesn't have, instead of just saying "You are an expert in blah"

It's not adding anything but a prompt saying "when asked to do X invoke script Y or do steps Z"

You're packaging the tool with the skill, or multiple tools to do a single thing.

There's no inherent magic to skills, or any fundamental difference between them and "just feeding in different prompts and steps". It literally is just feeding different prompt and steps.

Also, the pick up or not pick up, or discover or may not discover is solved as well. It's handled by my router, which I wrote about here - https://vexjoy.com/posts/the-do-router/

So these are solved problems to me. There are many more problems which are not solved, which are the interesting space to continue with.

So a better question to ask is - Do you have any ideas for an objective way to a measure a performance of agentic coding tools? So we can truly determine what improves performance or not.

I would hope that internal to OpenAI and Anthropic they use something similar to the harness/test cases they use for training their full models to determine if changes to claude code result in better performance.

I often use restore conversion checkpoint after successfully completing a side quest.

Because experts snd vets often use these tools and find them extremely lacking?

I’m not sure where the ‘despite’ comes in. Experts and vets have opinions and this is probably the best online forum to express them. Lots of experts and vets also dislike extremely popular unrelated tools like VB, Windows, “no-code” systems, and Google web search… it’s not a personality flaw. It doesn’t automatically mean they’re right, either, but ‘expert’ and ‘vet’ are earned statuses, and that means something. We’ve seen trends come and go and empires rise and fall, and been repeatedly showered in the related hype/PR/FUD. Not reflexively embracing everything that some critical mass of other people like is totally fine.

https://xkcd.com/1319/

Wait really, just for thinking someone is hot and telling you, who's presumably their friend?

To be clear, I’m also not saying anyone would ostracized for this, nor that anyone would ostracize me if I said this. But if one of my more casual friends randomly commented that their hairdresser is hot, I’d give them a bit of a sideways look, yeah.

There are receptors specifically for dopamine on the amygdala neurons. Dopamine molecules are released by the pre-synaptic neurons, travel across the synapse, and bind to these receptors.

Dopamine’s role in the nervous system is not simply an intermediate on the pathway to produce epinephrine or norepinephrine. If you thought like this you’d reach the conclusion that testosterone is simply a precursor to estrogen because the pathway to convert it exists in some tissues of the body.

The way that I’ve learned to think about it is that the brain is made up neurons, and they perform specific functions, technically individually but more usefully understood in regional groupings (primarily figured out via fMRI/blood flow studies and lesion experiments).

Each neuron’s activity is regulated by specific neurotransmitters, and the type of receptors expressed in neurons also correlates with these functional areas (figured out through PET/radio-tagged molecule scans and biopsies). Regarding dopamine specifically, the area that is responsible for effortful attention (prefrontal cortex) as well as for reward (in a general sense, broader “good” not only simpler treats) processing (nucleus accumbens) have high concentrations of dopamine receptors.

Therefore, drugs that interact with dopamine receptors or with chemical chains that involve dopamine can affect these functions.

Neurotransmitters are just chemicals and they go through many complex and interrelated metabolic chains, and at baseline (in a typical individual, barring specific genetic differences) it is often most useful to assume they’re all there and instead understand where they’re used.

This comment might not be the most succinct and I’m just started my education on the subject so I’m sure there are inaccuracy’s and I’d be happy if they’re pointed out, but I do hope that it helps you get a somewhat clearer picture and realize that you’re not dumb for being confused about this.

Understandable writing, easily digestible and - for me - thought-provoking. Thanks!