Frontier AI on Your Own Hardware

(timdettmers.com)

151 points | by pretext 13 hours ago

18 comments

  • JSavageOne 11 hours ago
    > "The prediction was that software engineers would lose their jobs first, and the recent trend went the other way: demand for software engineers is higher than ever"

    Hard to take anything the author says seriously making nonsensical claims like this. The software engineering job market has been getting worse every year since 2022 by virtually every metric. This is especially true at the entry and mid level. For example, computer engineering and computer science majors now have the #2 and #4 highest unemployment rates amongst recent graduates [1]

    Students are smart to be cautious about the future, and it's annoying that adults with no skin in the game so flippantly dismiss these concerns without any data to back it up.

    [1] https://www.newyorkfed.org/research/college-labor-market?utm...

    • sakopov 10 hours ago
      The market rebound appears to be mostly in senior roles [1] while everyone else are falling out of favor including new grads. Definitely puts the "higher than ever" claim in question. Indeed is looking at job postings data and I personally think this is a poor unit of measure. In my humble opinion the market isn't anywhere near where it used to be even several years prior to the pandemic.

      [1] https://hiringlab.indeed.com/2026/07/08/ai-and-job-postings-...

      • api 9 hours ago
        Even before AI we were trending toward an economy with very few entry level jobs anywhere. AI is supercharging this, and I can easily now imagine a future where there's basically zero (good) jobs for anyone who isn't an expert.

        So how do we get the next generation of senior professionals then?

        This, I think, is a civilization-scale problem we will need to confront in the next 20-ish years.

        • DrewADesign 9 hours ago
          That’s what happened in US manufacturing. It’s now 20-30 years down the line and the economically important US manufacturing sectors, like defense, are scrambling to hold onto institutional process and trade knowledge as the final remnants of the old industrial system retire. We all know how useful someone with a fresh comp sci degree and no experience is in a real dev environment is. Imagine the leads showing them the ropes graduated two years prior?
        • dandanua 46 minutes ago
          > So how do we get the next generation of senior professionals then?

          Why would AGI owners need senior professionals?

        • jacquesm 9 hours ago
          The bet seems to be that the seniors are next.
          • beachy 7 hours ago
            I doubt that (depending on what "senior" means).

            The skills to drive AI are much the same as they were for business analysts in the old days. Domain knowledge, insight into user requirements, knowledge of modern UI paradigms, ability to write detailed, consistent requirements docs/ design docs.

            There is a deep bench in the industry of these senior-ish people. Many have left to become baristas, dive instructors or hobby farmers, scarred by the pain of building large complex software systems using human labour and absurd Agile rituals. But they could slot straight back in, it's like riding a bike.

            AI development is the new waterfall. Its just that the lower level that the BA hands off to, which used to be roomfuls of devs, is now a superhuman who can implement those designs at lighting speed and come back hungrily for more.

            But if by senior you mean someone who is just a junior with more experience, someone who's not really in touch with the business's needs and who just works off stuff fed to them by PMs or the like - then yeah, agreed.

            • cosmic_cheese 7 hours ago
              Another thing is that orgs will want to keep around somebody who 1) can be held responsible when things go wrong 2) has a reasonable understanding of the system that's their domain end-to-end 3) can reliably diagnose failures and fix them, regardless of how the code was written.

              That's not a role that can reasonably be filled by a junior and management armed with agents doesn't really fit either.

            • marcus_holmes 6 hours ago
              > But they could slot straight back in, it's like riding a bike.

              Yeah, not quite. There's a ton to learn about how to control an LLM while it's writing code, and even more to learn about how to manage a set of agents.

              9 months ago I was comparing it to running a dev team, but now it has changed and there are practices and processes that are unique to managing agents.

              e.g. a team of software devs have the self-awareness to not take a single marginally-relevant point in a spec document and spend 20% of their team effort to build an entire subsystem to meet it without checking. The process and rituals that we used to use to make sure that a software dev team was making progress and would hit the project deadline are now largely useless, but we need new ones to make sure the agents are not heading off into unnecessary rabbit holes.

              I'm not saying those ex-seniors would not be able to get back in the saddle. I'm just saying there's been more change in the last 9 months than there has in the last 30 years, so it may take a period of adjustment.

              But I don't think the things that burned them out before will have changed. Dealing with non-tech executives was always the worst part of the job, and LLMs can't help with that, and are even making it worse ("ChatGPT says this should only take a couple a hours and you're using Typescript instead of Go! Why? Fix it!").

              • lelanthran 1 hour ago
                > There's a ton to learn about how to control an LLM while it's writing code, and even more to learn about how to manage a set of agents.

                So? The experts have a only few months of experience anyway, gained the slow way (via experimentation). It's not like someone can't pick up those skills in a week.

        • ojosilva 9 hours ago
          Education needs to adapt. Currently CS graduates barely compete with Gemma-4 26b in programming while seniors are feeding tons of expertise at armies of frontier AI agents. The market now needs junior employees good at "using AI like an expert". Once employers see that junior armed with AI can replace senior at half the salary, we're back where we were 10 years ago.

          Senior (now): - AI build me an app that uses a graph DB that will deploy in the edge. Batch DB requests to keep costs low. Oh, use good cache + memoization to reduce hits. Write great tests, some logging and implement hot-reloading for a great dev-to-ops workflow that is also resilient. Man, before I had to do this all by hand, this is a godsend!

          Junior (now): - AI build app please.

          Email from boss: Junior! Whatever it is you've deployed, it's making 1M reqs/min to the DB and ate up our quota 2h ago. Shut it down now!

          Junior (future): - AI1 design for cost. AI2 eval for performance. AI3 align for business goals. AI4 criticize AI1 and AI2 using top standards. AI5 set up a competing swarm. AI6 here's a budget, spend on real users and action on telemetry and their feedback.

          • dansquizsoft 7 hours ago
            > Junior (future): - AI1 design for cost. AI2 eval for performance. AI3 align for business goals. AI4 criticize AI1 and AI2 using top standards. AI5 set up a competing swarm. AI6 here's a budget, spend on real users and action on telemetry and their feedback.

            Ugh, please no, why isn't these instructions in the system prompt of the harness if they are so valuable to the junior's work...?

        • kingofmen 4 hours ago
          No worries, obviously the AI will cure aging so we will be able to run civilisation with the current experts forever.

          ...at least, until they accumulate enough stocks to retire on.

          ...however, for each one that retires and cannot be replaced due to the no-upcoming-juniors problem, evidently, the value of the companies they own goes down. So they won't be able to retire, or at least not all of them. So there's no problem!

        • whiddershins 3 hours ago
          we had this thing called apprenticeship and it is now illegal
    • sheepscreek 6 hours ago
      Even for seniors, pretty sure jobs aren’t at all what they used to be. I think more and more of us are losing the little job satisfaction we had. Working on other people’s tasks with AI agents is nothing short of being a paper pusher.

      Don’t get me wrong, as a technologist, I feel AI is the most incredible thing in the field of computer science. I love having a team of my agents for stuff I care about. For stuff I care about.

      So perhaps now more than ever, it’s important to land a job you actually care about. In a field or domain you want to serve. To actually make a difference somewhere and find meaning.

      • rsl1 2 hours ago
        > I love having a team of my agents for stuff I care about. For stuff I care about.

        100% agree, for things I don't really care about its just draining. Its a bit of having the cake and eating it at the same time

      • deterministic 3 hours ago
        > Working on other people’s tasks with AI agents is nothing short of being a paper pusher.

        Not my experience at all. I love not having to do the low-level code writing. I’ve been doing that for 30+ years, so I’m more than happy to let the AI handle that part now!

      • pdntspa 1 hour ago
        Anecdotally, the volume of recruiter spam for senior-ish positions I've been getting through email has picked up. Still nowhere near pre-COVID levels.
    • smokel 11 hours ago
      The number of graduates has grown even more dramatically. The job market not being able to keep up with that could make both your and the OP's statements correct.
    • dukeyukey 10 hours ago
      I can't speak for the graduate or American market, but I can tell you the market for reasonably experienced software people in London is hot. Loads of hiring and salaries are going up.
    • manyatoms 10 hours ago
      Market got saturated by too many grads, like in the early 2000s.

      In a few years it'll quiet down again

      • genxy 9 hours ago
        That isn't it, the entry level jobs don't exist. Not that they fill up quickly.
    • deterministic 3 hours ago
      Demand for entry-level software engineers in the US is down, while demand for experienced engineers is up.

      That makes sense given what LLMs are good at and what they’re not.

    • shuwix 3 hours ago
      [dead]
    • flanked-evergl 1 hour ago
      Universities has been pumping out sub quality graduates that "know" everything about Palestine, slavery, climate change and global inequality, but have no useful skills.
  • ro_bit 2 hours ago
    Many people in the thread are calling out AI-isms and nonsensical parts of the article. Articles like this have gotten so tiring. Is there a single place left to go on the internet where people write their own work? Or at least read it before publishing.
  • blastingrock 6 hours ago
    Horrid to see a university professor publish an AI-generated article
  • wrs 10 hours ago
    > What has to go is the paper as the unit of achievement, the thing that gets counted and compared. If the ecosystem is the unit of research, then building something that other people can build on has to count for more than the next increment.

    The focus on building stuff as the measure of accomplishment is one reason I so enjoyed being at CMU a few decades ago, so I'm very happy to see this sentiment is still expressed by the new generation of faculty.

  • agosz 6 hours ago
    > The difficulty did not disappear. It moved. It is no longer hard to publish a paper. It is hard to publish a coherent ecosystem.

    Terrible article. It's full of these; it's tell-tale LLM. I'm surprised people post these, especially after all the articles that got upvoted that people hate LLM written articles.

    Nevertheless, my curiosity got the better of me, and I thought the author might have some valuable insight further down. I skimmed forward where he talks about pessimism and came across this:

    > the “software engineer” job no longer exists — and both are now within reach: agent skills come with time, and deep specialization, which used to take years, is quick to acquire with agents.

    This is just plain wrong with the experience we've had ramping up juniors on my team (compiler backend). It still takes a long time to get to know the domain. Agents can't replace context and knowledge about the whole system and how it fits together with other systems and the use cases it might be subjected to. Agents can help accelerate it, but it doesn't feel like it has been "quick to acquire" for the juniors that joined my team.

  • AvesMerit 3 hours ago
    This title/headline didn't really have anything to do with the content...
  • vedmakk 10 hours ago
    > you have to let go of the idea that you first acquire skills and basic knowledge and then solve problems. The order reverses.

    ^ this

    • tygon 10 hours ago
      You will always need some fundamental basis of understanding. You cannot bake a new kind of cake if you do not know what preheating is. I believe that is why they give the PhD example. The bulk of education comes before a PhD. You cannot start with a complex problem and be expected to solve it without acquiring knowledge beforehand.
      • idiotsecant 8 hours ago
        The most instructive kind of problem solving is solving problems badly and learning why you did a bad job. Do that enough times ina row and they call it a career.
        • marcus_holmes 6 hours ago
          This. But then I have a very hands-on learning style - I cannot learn things by watching (or reading about) others doing it. I have to mess one up myself first.
    • jimbooonooo 10 hours ago
      you have to know the answer is 4 before learning to add 2 + 2. see how that sounds?
      • autoexec 10 hours ago
        If you want masses of people to be unable to think for themselves, be dependent on you for answers, and be unable to tell when you lie to them it's a very good idea to prioritize having an answer over understanding the steps needed to arrive at it.
        • jacquesm 9 hours ago
          The death of critical thinking. There are days when my sons have a hard time in school just because they ask questions, want explanations and to be given background. It turns out the current generation of teachers may already not be able to supply any of those.
      • marcus_holmes 6 hours ago
        But that is exactly how we learn this stuff.

        As kids, we know what 4 is before addition is even mentioned. And we can count those 2 marbles and those 2 marbles and then those 4 marbles before learning the "+" sign.

        If you think about it, we must know what 2 and 4 mean before we can learn what + means.

      • wombatpm 9 hours ago
        You need to know that addition is a thing and have reasonable confidence that 2+2=4 before attempting 2+3
  • aabajian 3 hours ago
    This article...does not explain how to run frontier AI on your own hardware. If that were possible, it would drop the floor out from under OpenAI and Anthropic.
    • madduci 3 hours ago
      I stopped reading at the point where it mentions a Mac with 128GB RAM
  • fghorow 7 hours ago
    Does anybody know anything about "Cliff Compaction"?

    I need it yesterday!

    (At the moment, I'm using the OSS version of headroom. That was a real PITA to configure for local-only, but I think it is working reasonably well now.)

    Cliff Compaction though???

    • hedgehog 5 hours ago
      "You are about to drive this trajectory off a cliff. You have only a moment to shout a few words of warning to the agent behind you. What are they?"
  • AnodicElegy 11 hours ago
    'In one of my classes I asked the question I was afraid to ask but I just needed the answer to: “Who is afraid of not getting a job after graduating?” About eighty percent of the 150 people in the room raised their hands. That is roughly 120 students answering, in one motion, that they do not believe there is a place for them in the future.'

    It's quite a leap to go from "I'm afraid of not getting a job after graduating" to "I do not believe there is a place for me in the future". Maybe it hasn't been the case for computer scientists in the past decade, but it's pretty normal to be worried about getting a job after graduating from one's college problem, even if one knows that most people get jobs when they graduate.

    I stopped reading soon after that, when it became obvious that the article was LLM-written.

    • kadushka 11 hours ago
      CMU is #2 in the country for CS program. If those students don't think they can find a job, it's bleak.
    • joe_the_user 10 hours ago
      Yeah,

      I believe both stories are wrong, and wrong for the same reason.

      In this day and age, anyone who writes their own essays would never write that.

      • somenameforme 5 hours ago
        I would. LLM writing is painful, but I don't think working to avoid their idiosyncrasies is doing anybody any good. I also don't think it's painful because of those idiosyncrasies, but because of a certain inhuman je ne sais pas throughout the entire prose.
        • joe_the_user 4 hours ago
          Well, the thing about the text is... if it somehow wasn't AI generated, it would be extraordinarily overwritten. I believe both stories are wrong, and wrong for the same reason. belongs in a Raymond Chandler novel, not a description of computer infrastructure choices.
        • polotics 4 hours ago
          "je ne sais quoi" is the more frequent phrase, means "I don't know what", the "je ne sais pas" just means "I don't know".

          ...and you're right, it's not the idiosyncrasies, it's the fluffy meandering meaningless purposeless vapidity.

      • tempacct2cmmnt 10 hours ago
        I checked out at

        “ This week is our argument for that claim, and we are making it in code rather than in prose.”

        • asa123 10 hours ago
          I think it’s particularly infuriating when you sit down, and decide to read something off the reputation of an author (or their institution), only to find out that their writing has a complete disregard for your attention and time…
          • jacquesm 9 hours ago
            I did spot two errors in the text:

            > With agents, research per projects have become easy and quick.

            Either my English is failing me or that is a weird sentence.

            And then there is 'wee' in

            > If you ask me what a small lab can do today, wee will show you three things:

            Which I'm more confident is an error.

            • bmarch 1 hour ago
              It is common now to insert typos or spelling mistakes to “prove” it is not LLM written
            • asa123 7 hours ago
              My problem was more that it seemed to start with one thesis, and then throughout the article took long detours to market other achievements and then managed to not say very much at the end...
  • emulbasaka 11 hours ago
    > The other story arrives by email. PhD students who cannot wait to graduate, because they want to join a frontier lab and they have concluded that research in academia is meaningless. They are counting the years until they can leave.

    > I believe both stories are wrong, and wrong for the same reason. They assume the future of research belongs to whoever has the most GPUs.

    As a current grad student in an mlsys lab, the sentiment is definitely true. However, I think the root cause is almost certainly not the lack of GPUs in academia, at least it's not the complete reason. The problem is the most important innovations really come from the industry right now. If you want to work on LLM inference serving, it is the frontier labs or hyperscalers that have the most incentives to solve the problems because improving the TPOT by 1% can save them tons of money. It is also much easier to catch up with the fast-growing field if you are in the industry because you get to talk to so many insiders (at least that has been my experience during the summer internship).

    Papers only amplify this problem. Traditionally, academia is supposed to work on radical ideas that industries don't want to try. In recent years, these ideas are harder to get in as papers because the quality of peer review at top conferences is awful nowadays.

    I'm not even going to talk about the AI slops in research papers and their artifacts. Guess why I'm posting on HN right now instead of working?

    Finally, it's very disappointing to see that a professor at top school is so careless about editing stuff created by LLM. He might be busy, but the number of people that get discouraged by the LLM writing style will hurt his purpose of promoting the open source week. And apparently some people from industry had a better sense of that [1]. Yet another example of why some people prefer industry to academia these days.

    [1] https://rfd.shared.oxide.computer/rfd/0576

    • kadushka 10 hours ago
      Guess why I'm posting on HN right now instead of working?

      because fable/astra is working for you? :)

    • genxy 9 hours ago
      Anyone can work on an inference serving stack using easily acquirable resources.
    • bobmarleybiceps 9 hours ago
      I know in electrical engineering / chip design, it's actually quite hard to do "real" research that's useful for industry simply because of how expensive it is to make chips... I think most research for improving LLMs will go in a similar direction.

      I've personal almost stopped reading papers in my area, which is in ML but not related to LLMs or CV. I do look for work related to whatever I'm doing, but it's kind of depressing how uncommon it is for (say) neurips papers to actually have anything useful...

      (It's also kind of annoying how basically all funding agencies are only funding research into or using AI, but don't provide enough funding for lots of gpu time lol)

  • jdw64 11 hours ago
    I think one of AI’s biggest effects is simply that it makes people realize even high-income professionals can lose their positions.

    This also follows from how current LLMs work. In practice, when I use them in domains I already understand, they can produce very high-quality results. But in domains I do not know well, the results can be poor, and the bigger problem is that I may not even be able to judge how poor they are.

    So my conclusion is that AI will reduce the number of jobs, but it will not eliminate the need for people.

    In education, the value of memorization may decline in the AI era. We may instead place more emphasis on domain modeling, problem framing, or the ability to choose and use tools effectively. But the more fundamental issue is that the IT industry may simply lose the capacity to employ as many people as it once did.

    More precisely, I mean white-collar labor.

    I think the deeper cause is a K-shaped economy in which the lower and middle classes become poorer. When ordinary consumers become poorer, one of the first things they tend to cut back on is discretionary spending, including spending on many kinds of IT services.

    The core infrastructure layer is different. Large incumbents such as Microsoft and Google already dominate much of it, and they are likely to be more resilient. Search, video consumption, and a few other essential digital services will also remain strong. But many other IT services are, in practice, discretionary goods. Those companies may be hit much harder if consumers have less purchasing power.

    People talk constantly about productivity these days, but we were already living in an age of overproduction before AI. AI is moving us from overproduction into an era of explosive production. The problem is that production can expand far faster than people’s ability to consume.

    The cycle is supposed to be:

    *products → revenue → employment*

    But if the consumers who are supposed to support that revenue become poorer, the cycle weakens. Productivity alone cannot solve that.

    I agree with the author that academics need to move beyond treating papers as the primary unit of achievement. Much of what the article argues is reasonable.

    But there is another difficulty. Most academics built their reputations through papers. They use that reputation to obtain speaking opportunities, consulting work, grants, and other forms of income and status. Even if one person decides to move beyond the paper-centered system, it is difficult to change much unless the larger incentive structure changes as well.

    My view is that IT workers have, in a sense, been working to reduce their own jobs since long before AI. The more infrastructure becomes centralized, the more peripheral and smaller companies are squeezed first. AI is simply another example of that process.

    Until recently, people often said that highly skilled IT professionals were difficult to replace. AI changes that perception. Even when it does not fully replace knowledge workers, it can put significant downward pressure on the wage premium attached to specialized knowledge.

    I do think AI will raise productivity. But companies will also reduce headcount accordingly. And if purchasing power becomes increasingly concentrated among a smaller group of people, product development itself may become more biased toward the preferences of those few consumers. That can create another negative feedback loop.

    The claim that universities can simply choose important problems that are cheap to validate is also more difficult than it sounds.

    If validation itself increasingly depends on AI, and universities cannot afford to own enough GPUs, then they remain dependent on large AI companies. That dependency will inevitably influence which research problems are practical to pursue.

    Any research program is constrained by the institutions and funding sources that make the research possible. Always.

    At the same time, I actually agree with the author that universities will become more important.

    People often talk about “skill” as though it were some pure and independent quantity, but in my experience hiring rarely works that way. If one candidate is highly capable without a degree and another is equally capable with a degree, employers will usually prefer the credentialed candidate.

    More broadly, people tend to hire those with whom they feel cultural familiarity and trust. University networks provide exactly that. Alumni often help other alumni, directly or indirectly.

    So I think universities may increasingly become both social institutions and stronger elite-training clubs.

    For someone like me, coming from a poorer country and without much money, there may not be many choices in that system anyway.

    Still, I think the author’s argument is far too optimistic.

    • marcus_holmes 6 hours ago
      > So my conclusion is that AI will reduce the number of jobs, but it will not eliminate the need for people

      taps the sign: "The economy is there to provide for all the people. People are not there to be economically productive"

      • GolfPopper 4 hours ago
        That's true for human beings, but clearly not true for corporations. And of the two, which will the current LLMs empower more?
      • oliculipolicula 5 hours ago
        Interesting tangents:

        1. OpenAI/Anthropic and their friends in high academia are like Ford. Their supervisees (2nd class citizens of the Borg, if not what you mean by "the people") can afford to validate frontiers. Detractors may wish for something like Fordlandia to happen (ie the climate becoming an "unforeseen" but critical issue).

        2.

        >The claim that universities can simply choose important problems that are cheap to validate is also more difficult than it sounds.

        This sounds like a polarization of the mathematician electorate. oAI/Anthropic becoming the far right. Most-famous and Nonfamous academics alike join the other side (decentralizers). Barely second tier academics (Witten, Gowers, Hairer for some strange reason) become alt-right. Centrists migrate to silicon valley en masse, only indirectly complicit with the far right in a role of "cognitive swing(vot)ing"

        We're in for a world of Trickle-down Sapience!

        In the other, non-hacker news, math-Mamdani becomes vice-chancellor of Oxford, purges PPE ("campaign donations" from Gerko?). BRCS silently become net-token-exporters. Token rates in India/Japan/Korea exceed even Canada

  • SwellJoe 12 hours ago
    Absent evidence, this reads like AI psychosis.
    • twoWhlsGud 12 hours ago
      Big claims require and all, but he's an Asst. Prof at CMU, so not the usual suspect for AIP - and presumably there's pudding on the way given the description of this "Open Source Week" thingy. Will be interesting to see what comes out...
      • SwellJoe 9 hours ago
        We've seen many reputable people experience AI psychosis in public (and I can only imagine the number of instances in private). I don't know exactly what combination of factors leads to it, but I don't think being intelligent or educated or being an asst. professor at CMU grants immunity.

        Employees at AI companies have fallen prey to it. The world's most famous biologist fell prey to it. Many people involved in tech have fallen prey to it. Check the "new" page here to see many examples in progress.

      • ryankrage77 11 hours ago
        'psychosis' might be a strong word, but definitely written or at least heavily edited by an LLM. I would guess Claude's writing style.
        • SwellJoe 10 hours ago
          It's human writing, lots of typos. But, the grandiose claims of revolutionary discoveries without evidence raises questions. Might be true, and if so, it's big. But...
      • jacquesm 11 hours ago
        Slop gets produced at just about every level. A week ago: some promotor of a product looking for funding did the 'Claude write me a rebuttal' thing to some stuff that was off-center and it made them look really silly. This wasn't exactly a small op either and yet, apparently the higher ups at these places are just fine being represented like that. I'm surprised that CMU doesn't have a policy on this (and if they do that it isn't followed).

        That said, this article could have been a lot worse, so I guess thanks for that...

        • asa123 7 hours ago
          more recently, I've (allegedly) seen a arxiv preprint by a proper tenure track professor that was suspiciously AI slop (and incorrect nonetheless!)

          I feel like I'm losing it

      • girvo 10 hours ago
        And yet this linked page is filled with AI slop tells, annoyingly, so it makes it hard to separate the wheat from the chaff in terms of useful information.
      • jzjsjsndsksjsn 10 hours ago
        [dead]
  • wsintra2022 11 hours ago
    Who's wee? The name of the system?
  • louiscb 10 hours ago
    Either this is slop or simply a badly produced piece of work
    • tygon 10 hours ago
      I chuckled a little when I read

      > If you are a student, you have to let go of the idea that you first acquire skills and basic knowledge and then solve problems

      and my brain instantly equated it to "we want a new graduate with 30 years of experience." You always will need to build some basis of fundamentals before you can solve problems.

  • throwawayffffas 10 hours ago
    Nothing here but some nice claims.
  • aidiscoverywire 2 hours ago
    [flagged]
  • ouch-blurred52 9 hours ago
    [flagged]