Local models are meaningfully useful* for agentic work now. This is a realization I came to over the last day or so.
I wrote a lot of custom mods for a game that recently switched engines. The migration is going rather smooth for most people because of wise architectural choices from the dev early on, and because they prioritized backwards compatibility in the update. But there are still some rough edges, especially if you had your hooks deep into the engine.
I recently vibeforked Unbiased Bob’s fork of Strata to make it compatible, easy to use and optimized for my OS and hardware (NixOS 5800x3d 64gb, 7900xtx 24gb). Sol 6.1 got it working with Qwen 3.8 Flash Next IQ3 XSS (~75GB) at roughly 46tps 164k ctx with enough headroom to launch a heavy browser along with a triple-A game set to low in 720p for testing when I need to (plus space for zram swap)
I set it up in deepseek harness, gave it a 14MB error log in the game’s working directory, then went to sleep. I woke up and it had run for 3 hours. The game launched with only two errors, which it quickly fixed from the new, vastly smaller logfile.
It hammered down every issue, making changes across 30 separate mod repos. Practically a one-shot, and all the features worked as originally intended.
I left a local coding agent running completely on its own for three hours across multiple repos across a dozen auto-compactions with a log file and no ability for it to verify its work besides a compiler. And it produced meaningful results. I can’t speak to the code quality, but it worked. This was completely unthinkable even just two years ago. Even one year ago, the cloud frontier wouldn’t be able to do this on its own, and now a local model can, running on my consumer gaming hardware from one or two gens back.
“Huh… So… We’re here now…" I said to myself, with a bit of stunned disbelief, wondering what the future one or two years from now might hold.
Sounds acceptable. I mean as long as you don’t have to intervene all that much. And you said electricity is affordable where you live… So all good.
Yeah, I think hallucinating isn’t all that bad in reasoning steps or in-between. As long as it manages to then home in on the correct thing.
Electricity is affordable anywhere there is space for a few PV modules. At least, as long as the sun shines.
I feel that could be a bit of an issue with letting the agent run overnight 😆
But generally, yes. If someone can afford a graphics card, I suppose they could afford some 450W (peak) solar modules at like 100€ as well.
Sadly it’s the cold season now. With lots of overcast or rainy days and the sun angle isn’t ideal either. Lots of atmosphere absorbing the light, and days are shorter.
I paid 82 EUR for a 500 Wp bifacial when picking up four of them about a week ago. A 2 kW Hoymiles microinverter was 202 EUR, 10 m plug in cable 45 EUR. Flat roof mount 145 EUR. I will need to buy some 36x concrete plate ballast for about 25 EUR. Some 4x 3 m and 4x 1 m DC4 module cables, cheap. Yes, winter months are thin, but my meter is old enough to run backwards. When they exchange it, I will buy a couple 10 kWh battery.
I intend to run a local LLM during daytime anyway.
Lucky you! They replaced my good old electricity meter right when I was willing to buy some solar panels. And now I keep postponing it because it’s not that attractive anymore. The street is the wrong way, so all I can easily do is place them eastwards, with the roof casting shadows at a bit after noon. And not sure about the cost of paying someone to bolt them to the top of the roof… Guess I could do some east/west configuration there.