Ask HN: How close are we to local LLMs being useful? What's the impact?

Feels to me like local models are an under-covered aspect of this whole AI boom.

If everything improves over time, at some point a good chunk of tasks won’t need to be done in data centers or be subject to the whims of a few frontier AI labs.

How close are we to that? Or is my thinking flawed?

5 points | by AbstractH24 5 hours ago

5 comments

  • segmondy 1 hour ago
    Local LLMs have been useful since 2024. If you don't know this then you are just far behind. Catch up!
  • david927 4 hours ago
    I think we're past that point; they're absolutely useful already for a lot of tasks. I think it's about costs, convenience, and benefits of a frontier model for what you're doing.
    [-]
    • AbstractH24 4 hours ago
      Which ones are most useful? Any suggestions on where to go to start exploring this world?
  • PaulHoule 4 hours ago
    I do classification with SLMs and for my tasks when I have a few thousand samples the frontier models in zero-shot and few-shot modes are embarassingly bad in comparison.
  • buffer_overlord 5 hours ago
    until its cheaper to train and infer than 100k gpu data centers...i doubt it will ever compete.
  • tylerhackbart 4 hours ago
    [flagged]