3 ms·

Even if something like Gas Town isn't remotely affordable today it could potentially be a useful glimpse at what can be done in principle and what might be affo

by leoc 9mo ago

Even if something like Gas Town isn't remotely affordable today it could potentially be a useful glimpse at what can be done in principle and what might be affordable in, say, 10 years. There's a long history of this in computing, of course https://en.wikipedia.org/wiki/Expensive_Typewriter https://en.wikipedia.org/wiki/Expensive_Typewriter https://en.wikipedia.org/wiki/Sketchpad https://en.wikipedia.org/wiki/Sketchpad https://en.wikipedia.org/wiki/Xerox_Alto https://en.wikipedia.org/wiki/Xerox_Alto . OTOH it could certainly make the approach totally unsuitable for VC funding at present, and that's without even considering the other reasons to be wary of Gas Town and Beads.

lovich 9mo ago
Nothing I have read about LLMs and related makes it seem like they will be affordable in the future when it comes to software specifically. I will preface this that I think AI agents can accomplish impressive things, but in the same way that the Great Pyramid of Giza was impressive while not being economically valuable. Software is constantly updating. For LLMs to be useful they need to be relatively up to date with software. That means training and from what I understand the training is the vast majority of costs and there is no plausible technical solution around this. Currently LLMs seem amazing for software because AI companies like OpenAI or Anthropic are doing what Uber and Lyft did in their heyday where they sold dollars for pennys, just to gain market share. Mr. Yegge and friends have made statements about if cost scares you, then step away. Even in the article this thread is about he has this quote > Jeffrey, as we saw from his X posts, has bought so many Claude Pro Max $200/month plans (22 so far, or $4400/month in tokens) that he got auto-banned by Anthropic’s fraud detection. And so far what I’ve seen is that he’s developed a system that lets him scale out the equivalent of interns/junior engineers en masse under his tentative supervision. We already had the option to hire a ton of interns/junior engineers for every experienced engineer. It was quite common 1.5-3 decades ago. You’d have an architect who sketched out the bones of the application down to things like classes or functions, then let the cheap juniors implement. Everyone moved off that model because it wasn’t as productive per dollar spent. Mr. Yegge’s Gas Town, to me, looks like someone thought “what if we could get that same gaggle of juniors, for the same cost or more, but they were silicon instead of meat” Nothing he’s outlined has made me convinced that the unit economics of this are going to work out better than just hiring a bunch of young bright people right out of college, which corporations are increasingly loathe to do. If you have something to point to for why they thought is incorrect, in regards to this iteration of AI, then please link it.
- leoc 9mo ago
  But why should one expect no future improvement in training costs (all else being equal) from Moore's Law, never mind any future use of eg. more efficient algorithms or more specialised hardware?
  - lovich 9mo ago
    If you are quoting Moore's Law in 2026 as the reason LLMs will be profitable, I don’t know how to interact with you. I guess we’ll make up the losses per unit at scale, and grow to infinity.
    - leoc 9mo ago
      I'm not making some general claim about LLMs being profitable. To be clear, are you claiming with high confidence that LLM training costs will show no meaningful reduction, for some fixed quality, over roughly the next 10 years?

by leoc 9mo ago

lovich 9mo ago

Nothing I have read about LLMs and related makes it seem like they will be affordable in the future when it comes to software specifically. I will preface this that I think AI agents can accomplish impressive things, but in the same way that the Great Pyramid of Giza was impressive while not being economically valuable. Software is constantly updating. For LLMs to be useful they need to be relatively up to date with software. That means training and from what I understand the training is the vast majority of costs and there is no plausible technical solution around this. Currently LLMs seem amazing for software because AI companies like OpenAI or Anthropic are doing what Uber and Lyft did in their heyday where they sold dollars for pennys, just to gain market share. Mr. Yegge and friends have made statements about if cost scares you, then step away. Even in the article this thread is about he has this quote > Jeffrey, as we saw from his X posts, has bought so many Claude Pro Max $200/month plans (22 so far, or $4400/month in tokens) that he got auto-banned by Anthropic’s fraud detection. And so far what I’ve seen is that he’s developed a system that lets him scale out the equivalent of interns/junior engineers en masse under his tentative supervision. We already had the option to hire a ton of interns/junior engineers for every experienced engineer. It was quite common 1.5-3 decades ago. You’d have an architect who sketched out the bones of the application down to things like classes or functions, then let the cheap juniors implement. Everyone moved off that model because it wasn’t as productive per dollar spent. Mr. Yegge’s Gas Town, to me, looks like someone thought “what if we could get that same gaggle of juniors, for the same cost or more, but they were silicon instead of meat” Nothing he’s outlined has made me convinced that the unit economics of this are going to work out better than just hiring a bunch of young bright people right out of college, which corporations are increasingly loathe to do. If you have something to point to for why they thought is incorrect, in regards to this iteration of AI, then please link it.

leoc 9mo ago
But why should one expect no future improvement in training costs (all else being equal) from Moore's Law, never mind any future use of eg. more efficient algorithms or more specialised hardware?
- lovich 9mo ago
  If you are quoting Moore's Law in 2026 as the reason LLMs will be profitable, I don’t know how to interact with you. I guess we’ll make up the losses per unit at scale, and grow to infinity.
  - leoc 9mo ago
    I'm not making some general claim about LLMs being profitable. To be clear, are you claiming with high confidence that LLM training costs will show no meaningful reduction, for some fixed quality, over roughly the next 10 years?