4 ms·

A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips cou

by hughw 3mo ago

A Fable 5 model running at 9,000 tokens/s on an ASIC rather than 150 tokens/s on electricity chugging Nvidia GPUs, or even giant SRAM Cerebras or Groq chips could be good enough to meet the majority of demand. 640K ought to be enough for anybody.

JacobAsmuth 3mo ago
Agreed 100%. This guy thinks there's a limit on the demand for intelligence. You think that Fable 7 which can run a billion dollar corporation on its own has no consumer demand just because we have fable 5 at 9k tok/s? Who do you think will be the biggest customer of such a model? Fable 7, obviously.
- LarsDu88 3mo ago
  Certainly there will be demand for Fable7, but that demand is context specific. Frontier labs' profit is dependent on there being sufficient demand for the next layer of capability and whether the premium consumers are willing to pay for that. The incremental unlock of capability by ever increasing frontier model sizes will eventually reach diminishing returns. I would argue tnference speed increases would actually unlock a different kind of more meaningful value for a wider audience.
- drob518 3mo ago
  Of course not. But many tasks won’t require Fable 7 level intelligence and many people won’t want to pay for it. Honestly, I’m using Deepseek v4 Flash a LOT lately to do more mundane tasks because it’s so nearly free and I don’t need Fable or even Opus. Serving those mid-level models at high speeds and low prices is a definite winner for lots of applications. And sure, the frontier models will continue to drive the frontier forward.
- sdfefcxv 3mo ago
  theoretically theres a no limit on the demand of anything if the price is right pretty stupid statement lmao
  - JacobAsmuth 3mo ago
    Marco econ 101 disagrees :) Ask your favorite LLM to explain the "yield curve" to you
- jackb4040 3mo ago
  Sorry, which billion-dollar corporation is Fable running "on its own"?
  - blep_ 3mo ago
    Fable 7 isn't a thing, 5 is the one people are currently excited about. They're talking about a hypothetical more-advanced future version.
  - aaronharnly 3mo ago
    They imagined a "Fable 7" model (i.e. two generations hence) which would be capable of such feats.
foobit-dev 3mo ago
> 640K ought to be enough for anybody. I get this reference!
dymk 3mo ago
I am really confused about the point you're trying to make. 150tok/s is slow, but so is 9000tok/s? Or they're both fast? Or 150tok/s should be enough?
- hughw 3mo ago
  probably no number you come up with will be adequate for very long. [edit] it's a reference to this possibly apocryphal prediction https://www.computerworld.com/article/1563853/the-640k-quote-won-t-go-away-but-did-gates-really-say-it.html https://www.computerworld.com/article/1563853/the-640k-quote...
- oliyoung 3mo ago
  The "640k should be enough for anyone" quote (even if Gates didn't exactly say it) is making the point is that _right now_ we have no idea about what our future needs and capabilities will be, we can't imagine what "should be enough" will be 640k was enough ... in 1981 ... almost fifty years later is 50,000 lower than a standard off the shelf PC now
  - dymk 3mo ago
    If that’s the case I still don’t get it. 640k was enough in 1981, same as how 150 (or 9k) tok/sec would suffice for 2026 The comment seems like the nerd equivalent of 6 7
    - everyday7732 3mo ago
      The original quote was supposedly Bill Gates saying that "640kb of memory should be enough for anyone". The quote became famous because it's a failure to imagine that people would find new uses for computer memory if it became plentiful and cheap. In the quote Gates is not expecting people to come up with more demanding applications for computer memory (RAM) than the ones which were available in 1981. Computer memory did in fact become plentiful and cheap after this, and computer programs became more complex and memory-hungry and today we wouldn't consider a 640kb an acceptable amount of ram for even the lowest-end device. OP is repeating the quote about memory to indicate that they think that modern LLMs will be a similar resource. We should not expect that demand for LLM inference will stay flat, and that once everyone has cheap access to Fable level, we won't find new more demanding uses for it.
stickfigure 3mo ago
> 640K ought to be enough for anybody. The question is really whether 640k is enough to last you until your next hardware upgrade.

by hughw 3mo ago

JacobAsmuth 3mo ago

Agreed 100%. This guy thinks there's a limit on the demand for intelligence. You think that Fable 7 which can run a billion dollar corporation on its own has no consumer demand just because we have fable 5 at 9k tok/s? Who do you think will be the biggest customer of such a model? Fable 7, obviously.

LarsDu88 3mo ago
Certainly there will be demand for Fable7, but that demand is context specific. Frontier labs' profit is dependent on there being sufficient demand for the next layer of capability and whether the premium consumers are willing to pay for that. The incremental unlock of capability by ever increasing frontier model sizes will eventually reach diminishing returns. I would argue tnference speed increases would actually unlock a different kind of more meaningful value for a wider audience.
drob518 3mo ago
Of course not. But many tasks won’t require Fable 7 level intelligence and many people won’t want to pay for it. Honestly, I’m using Deepseek v4 Flash a LOT lately to do more mundane tasks because it’s so nearly free and I don’t need Fable or even Opus. Serving those mid-level models at high speeds and low prices is a definite winner for lots of applications. And sure, the frontier models will continue to drive the frontier forward.
sdfefcxv 3mo ago
theoretically theres a no limit on the demand of anything if the price is right pretty stupid statement lmao
- JacobAsmuth 3mo ago
  Marco econ 101 disagrees :) Ask your favorite LLM to explain the "yield curve" to you
jackb4040 3mo ago
Sorry, which billion-dollar corporation is Fable running "on its own"?
- blep_ 3mo ago
  Fable 7 isn't a thing, 5 is the one people are currently excited about. They're talking about a hypothetical more-advanced future version.
- aaronharnly 3mo ago
  They imagined a "Fable 7" model (i.e. two generations hence) which would be capable of such feats.

foobit-dev 3mo ago

> 640K ought to be enough for anybody. I get this reference!

dymk 3mo ago

I am really confused about the point you're trying to make. 150tok/s is slow, but so is 9000tok/s? Or they're both fast? Or 150tok/s should be enough?

hughw 3mo ago
probably no number you come up with will be adequate for very long. [edit] it's a reference to this possibly apocryphal prediction https://www.computerworld.com/article/1563853/the-640k-quote-won-t-go-away-but-did-gates-really-say-it.html https://www.computerworld.com/article/1563853/the-640k-quote...
oliyoung 3mo ago
The "640k should be enough for anyone" quote (even if Gates didn't exactly say it) is making the point is that _right now_ we have no idea about what our future needs and capabilities will be, we can't imagine what "should be enough" will be 640k was enough ... in 1981 ... almost fifty years later is 50,000 lower than a standard off the shelf PC now
- dymk 3mo ago
  If that’s the case I still don’t get it. 640k was enough in 1981, same as how 150 (or 9k) tok/sec would suffice for 2026 The comment seems like the nerd equivalent of 6 7
  - everyday7732 3mo ago
    The original quote was supposedly Bill Gates saying that "640kb of memory should be enough for anyone". The quote became famous because it's a failure to imagine that people would find new uses for computer memory if it became plentiful and cheap. In the quote Gates is not expecting people to come up with more demanding applications for computer memory (RAM) than the ones which were available in 1981. Computer memory did in fact become plentiful and cheap after this, and computer programs became more complex and memory-hungry and today we wouldn't consider a 640kb an acceptable amount of ram for even the lowest-end device. OP is repeating the quote about memory to indicate that they think that modern LLMs will be a similar resource. We should not expect that demand for LLM inference will stay flat, and that once everyone has cheap access to Fable level, we won't find new more demanding uses for it.

stickfigure 3mo ago

> 640K ought to be enough for anybody. The question is really whether 640k is enough to last you until your next hardware upgrade.