Claude Code, Codex, opencode (Z.ai GLM-5.x), sometimes Gemini -- most of my work is at Mac iTerm2 CLI or Arch Linux terminal. I sometimes use emacs as the coding agent, but rarely JetBrains IDE or vscode (more for reading/debugging code).

I supplement this with my own vibe-coded tools that help agents plan, perform sagas/steps, stay on track, tools that check the code produced, tools that check the produced documentation, tools/process to limit AI coding agent write access to files outside their assigned project, and tools that allow coding agents to collaborate (shared wiki, mailboxes, gh cli issues, etc.). I have written test frameworks that my AI agents use to detect functional regressions.

On top of this I used web-based or Desktop: Claude, ChatGPT, and/or Gemini to do research, architecture, requirements, planning, design, and later review; to suggest improvements, answer questions, etc.

And I am trying to switch to opencode using local LLMs, served by llama.cpp, Freetoken, or vLLM, on 12G, 16G, and 24G VRAM GPUs. So far the cloud LLMs are faster/better, but recent qwen3.x MoE models look very promising.

This IBM Research looks very promising: https://research.ibm.com/blog/northpole-llm-inference-result...

which I found after watching this video: https://www.youtube.com/watch?v=G4nEpY1LYzM - The AI Chip With No Ram (and still 22x Faster)

I am curious about what Rust-based extensions I can add next to my custom machine learning programming language.

> https://github.com/sw-ml-study/demo-extensions shows my first attempt at an extension, desktop graphics. README shows a Conway's Life "Glider" on an interactive view of a 3D torus (and other desktop examples).

The underlying language is implemented in Rust, and supports running in a browser via WASM, but the language itself is an array programming language I created to help me visualize my study of ML. (This is my non-Python response to Jupyter notebooks and Google Colab)

Then I wanted to visualize ML-related Math concepts so I added some demo repos.

And then I added language features to make it a more general purpose application development language, with libraries as a way to share code. So I added I/O, functional programming features, etc. and related demos. Dogfooding!

And more recently the dynamic Rust extension mechanism to add things that cannot be written easily in the array language itself (or for features that would clutter the language).

Maybe I can add a networking extension or an infinite precision math library.

So my curiosity is around what belongs in the base language and what belongs in optional dynamic extensions.

why do you need special language to understand algorithms like a softmax dag or whatever algorithm u can explain in any language that already exists

I am a single person LLC so my "production" is just me using AI to build things.

I use MCP for some things, like Playwright; for web UI development, but I also use CLIs, especially custom CLIs I create that have built-in help for the AI coding agent to use. I just to run my command (in the path) with --help and it figures out how to use. This seems more straightforward and efficient than MCP.

I have written MCP servers for all sorts of things, like monitoring remote systems, operating a PvP game as PvAI, others but lately I just create a new CLI with help. I use this with Claude, Codex, Gemini, and opencode (z.ai GLM) instead of dealing with how each agent registers MCPs.

I am trying to make it easier to use LLMs on older, cheaper, smaller GPUs. I'm taking a similar approach (move MoE expert weights to disk, avoid wasting VRAM on these). My goal is also to run models that do not fit. My work also suffers from AI documentation issues. Where my approach differs is that instead of running an LLM that doesn't fit slowly, run many agents in parallel sharing the streams of MoE experts weights, to increase throughput. I envision a team of AI agents sharing a pretty-good-at-coding LLM that does not fit to collaborate on a set of related features, being developed in parallel.

My work is showing promising results (if you can get past the way the AI tries to describe what I am doing). https://sw-ml-study.github.io/emufpga/index.html

I am doing this work initially on a 6-Xeon-cores Linux workstation with an RTX5060-16G to run MoE models larger than that. Then I will be moving this to a server with a lot more cores (Dual 32-cores) and a mix of SAS HD and SSD drives, using older GPUs.

Ultimately, I hope to build some FPGA/MCU "accelerators" that process the expert weights on systems with not enough CPU cores to offload the experts. If I can enable large capable models to run on older hardware, keeping the limited GPU VRAM for context and things that must be in VRAM, I can get useful work out of my old refurbished systems without paying today's RAM and VRAM/GPU prices.

I think AI is more of a radial arm saw. Very flexible, adaptable, capable, but very dangerous in untrained hands.

I tried nono to limit commands an agent can run but that proved to be cumbersome.

I ended up creating multiple logins on a Linux system. I used my developer account to clone git repos and then create local bare repos in a read only dir. The new linux user accounts can clone from these bare repos but not push. This way, each coding agent (installed per linux account) can only work within the file system for that user on the repo assigned to that user.

Each of these user has no github permissions (cannot clone from github, cannot create pull requests, etc.). They can run tools (that I have installed) but cannot generally install software or push changes (no sudo access, no github credentials). So I can ask each coding agent to make local changes, and when they are done they create a pull request branch and wait for it to be accepted.

Then my single authorized user can take that pull request branch and merge it and push it (or create a pull request). This prevents any agent from working on a repo other than the one it has access to.

Doing this allows me to run several agents at once, one per repo, and not interfere with each other. Before I did this the agents would overreach and start modifying other repos while the correct agent was also making changes. Now, each agent creates a document requesting changes from other agents in their respective repo.

So these low privilege users have no credentials (other than to use a coding agent). This setup works for me.

This looks extremely helpful, thanks!

Understood, and I like a challenge. I've also solved some tough engineering problems in the past (before AI). With AI it seems (and I emphasize seems) that I can tackle much harder problems. That's why I am proceeding to measurable experiments and not just taking AI chats as accurate.

I have a homelab, electronics lab, and software dev experience (electronics is a hobby, and an early career before I pivoted to O/S development). Before the recent RAM/GPU price jumps I had invested in a lot of used ECC RAM and many older GPUs (and some new GPUs) on many older servers that I refurbished and upgraded. It is hard to justify the prices of new GPUs/RAM going forward. It is cheaper to upgrade my CPUs (and I have) to have more cores.

Yes, my older hardware is slow by today's standards, but it is at least affordable. Yes, old hardware is power inefficient, but I justify that by using solar panels. I cannot do anything about the current supply/demand problems, but I can perhaps help with reuse and upcycling. Maybe my work will help students and junior programmers learn on old/used hardware.

The value proposition I am pursuing is not faster speed, but increased throughput (and possibly reduced kWh).

I'm trying to understand the benefits of streaming expert weights through hardware that offloads the math and avoids storing all of the weights in RAM at once.

But that's not the only thing that can be streamed and offloaded. '

Finally, I'm trying to come up with approaches to reuse old hardware, old GPUs, old RAM instead of paying today's prices for GPU VRAM or unified memory. Even if I do not end up showing any particular FPGA benefit, I might be able to better run very large models on systems without GPUs or without unified RAM.