My team is in the agentic orchestrator phase. I like this software factory pattern in concept, but our biggest challenges in development are acceptance testing of anything UI-related. Mobile app testing in particular is still a huge bottleneck that requires a human. AI models really suck at identifying poor usability and jank, particularly because they only typically process snapshots of the app from an instance in time.
I know there are traditional testing frameworks that can detect jitter and frame drop to a certain level. We could potentially start having agents build that in.
If we had concrete designs and specs on every project, that would also be helpful, but in a fast-moving startup, that gets delegated to the builders. That puts a human back in the loop every time.
Curious to hear what anyone else does to fully adopt a software factory pattern.
I totally agree with this. Agents seem tremendously bad at UI to me. Maybe it's just because I am a back end guy.
Right now I'm working on a declarative UI framework which can help me along here. My thought is that if I sacrifice a little control for sane primitives, that will make that spec /build loop easier.
I think ClayUI is a really interesting "reduced instruction set" for UI. I don't know that immediate mode UI is the right call for anything web related (that's how you get React lol) but his reduced primitive layer is very interesting to me
> Right now I'm working on a declarative UI framework which can help me along here
After your first para, I was about to suggest exactly that (well, maybe not writing your own). I find that frameworks (both front-end and back-end) constrain the LLM's choices and result in both sensible defaults and improved consistency.
Of course, you will immediately hit the problem all frameworks have: customer requirements that the framework components don't quite meet.
Probably one of those things where it's best to try it yourself, because I think anything that constrains the LLM is an improvement.
Think up a simple CRUD problem (lets say a book database - Book *-* Author, Book 1-* Publisher, Book 1-* Shelf). Write a loose markdown spec, and have the chatbot one-shot it in [language of choice]. Then do the same thing, but tack "build it in Filament/Avo/Django/AdonisJS" on the end.
I'd be interested to see if your results match mine - everything's just more consistent, and smoother, and you start getting freebies like autocomplete. The framework acts as guardrails and the LLM has to make fewer choices, so less of the token budget goes on bikeshedding.
I think Google/Meta surveillance just feels like something painless, benign, and invisible. You couldn’t see a reason a huge corporation would really want to target you or put you in danger. With Flock, your local law enforcement has access, and that feels like a much more real threat (warrantless tracking of your real-world activity with the ability to ruin your day/life), not to mention how it’s already been abused to track exes of officers, had be easily hacked in the past, and that AI face recognition matched the wrong people.
Honestly I think the only significant difference is in branding: Flock isn't consumer-facing and is explicitly marketed for tracking people's movements. The major tech companies that make your phone, browser, maybe a few smart home devices with always-on microphones or cameras, all have more data about you and your habits than Flock could ever obtain, and with us paying for the privilege no less. I get that the average person doesn't care when the product works as advertised, but I don't think that's healthy. People should care as much about privacy when it's Google or Meta as when it's Flock.
The U.S. leadership has a bad habit of starting conflicts that the bulk of the population doesn't want, which leads to large political costs and resulting half-measures. Our current leadership literally ran on "no more wars." Morons.
I wonder if this is a sign of things to come for dirt-cheap model hosting: no servers running old versions, only new versions. Just to keep costs down.
Gemini is still my top choice within production software for typical data extraction from unstructured data. Gemini Flash Lite feels like a cheat code for speed, and it's really cheap.
Some other Chinese models are also fast and cheap, but a harder sell in a U.S. production environment.
Speaking from experience here, flash lite models have amazing price, speed, and perform far above their size, but are susceptible to very bad instruction following and recall when either complexity or context size inch up. They’ll just forget to apply your instructions to portions of the input, and repeat parts of the input that should be returned verbatim as direct quotes but with subtle changes (breaking urls, for example).
This was 3.5 flash lite, actually, and after prompt tuning. It was very clearly an issue that correlated with input (JSON array) size, the more elements in the batch, the higher the error rate.
3.0 flash (not lite) handled it like a champ though, fwiw.
Hi, I’m the author of this blog. It depends on how strong of a model you need, but in general, Qwen is easily the best among the Chinese models right now.
Googles local gemma models which target roughly the same parameter count range, are known for being a lot better at vision tasks than qwen, no idea if 3.8 has changed that though
It’s an extremely consistent experience, and you can sit in the front seat (something I like personally). With Uber and taxi services, you never know what kind of driver or ride you’re going to get.
They use to be (along with Lyft). But it's been a very long time since that was clearly true.
In reality I think you can get better experiences than a taxi with a rideshare service (which they really haven't been for a very long time), but it's very hit or miss.
So less consistently bad as a taxi service... but far from consistently good.
You really have to believe your own narrative to do well as a startup founder, even when evidence is initially scant. This likely also has a tendency to slide into self-delusion.
I'm tired of being strung along on these silly narratives. I can't wait for open-weight models to be deployed around the world just so people like Dario will shut up about the mystical levels of power these models have.
I know there are traditional testing frameworks that can detect jitter and frame drop to a certain level. We could potentially start having agents build that in.
If we had concrete designs and specs on every project, that would also be helpful, but in a fast-moving startup, that gets delegated to the builders. That puts a human back in the loop every time.
Curious to hear what anyone else does to fully adopt a software factory pattern.
reply