>I used Qwen2.5-Coder-0.5B as the starting point. I needed a small model to get close to realtime performance. Being a coding model, it had also already seen plenty of CLI usage.
Interesting. I wanted to ask, how that model compares with the fine-tuned version, but I just saw the author provided benchmarks. Interesting that fine-tuning improved the memory so significantly, wouldn't have expected that.
Though, on second thought, I had an experience where I was able to, as a human, fine-tune my memory beyond what I previously thought was possible!
Reminds me of one of my favorite jokes.
Excuse me, can you tell me how to get to Carnegie Hall?
I regretfully suspect this actually is the future of UI. You just need smart local models running fast enough to keep the framerate high, but that’s inevitable in the long term, isn’t it?
Why would you do that instead of having the model do the task you're using the UI for? It could generate UI widgets on-the-fly as needed to take richer inputs.
It might be fine for one-off applications, but it's incredibly inefficient for something that has to be done over and over.
Why would you use a GPU and gigabytes of VRAM running a non-deterministic process for something a program can do on the CPU in milliseconds with kilobytes of memory?
We had decades to work on making software more performant, but instead most users' space-age 2nm chips spend most of their time running Javascript because it was faster to make software that way.
That seems as likely as anything, but that just moves the problem to counting, doesn't it? The model gets the duration, but I was under the impression LLMs struggled with the arithmetic that would make that useful.
Idk about Qwen 2.5 Coder that this is based on, but modern models don't struggle with arithmetic. I see Qwen 3.8 do long-hand math in thinking traces (ie. it outputs 17+35+12=64 in the thinking trace) so I would imagine this is something they train into the model specifically. Sometimes it creates and runs a simple python script to check its work.
Terminal world models have many applications, but empirical observation of successful pretraining or fine-tuning can be significant for encapsulating from rather expensive Bash traces, such as ones obtained from systems with scarce accelerators, etc.
Interesting. I wanted to ask, how that model compares with the fine-tuned version, but I just saw the author provided benchmarks. Interesting that fine-tuning improved the memory so significantly, wouldn't have expected that.
Though, on second thought, I had an experience where I was able to, as a human, fine-tune my memory beyond what I previously thought was possible!
Reminds me of one of my favorite jokes.
Excuse me, can you tell me how to get to Carnegie Hall?
—Practice, man... Practice...
Why would you use a GPU and gigabytes of VRAM running a non-deterministic process for something a program can do on the CPU in milliseconds with kilobytes of memory?
It’s cheaper to hire developers, and quicker to ship software. The user experience be damned.
Edit: oh, I saw you wrote a related comment after I posted this. So you already know that direct LLMs for UIs are all but inevitable...
If I had to complain about any site/app it'd be datadog... each tab uses like 1gb in firefox.
https://www.microsoft.com/en-us/research/articles/whamm-real...
Time is an input too. If nothing is pressed, the model gets a WAIT token with a duration instead of a key, which is how snake keeps moving on its own.
https://www.youtube.com/watch?v=z3pV6FHvcgM
The tmux thing had me confused for a moment until i realized it was emulating tmux. Super fun. Is the model window width aware?