3 ms·

I would readjust your convictions. We are only 2-4 years away from consumer grade immutable-weight ASICs.

by christopherwxyz 5mo ago

slashdave 5mo ago
We are discussing how rapid development has been, and now you want to freeze your model in silicon?
- rogerrogerr 5mo ago
  Genuine question from a place of ignorance: what in the silicon pipeline makes it take 2-4years to produce chips with a new model on them? Curious what the process bottleneck is.
  - jazzyjackson 5mo ago
    Without being an insider, I imagine that most global fab capacity is contracted out several years in advance. You might be interested in the tiny tape out project, which guides you through the process of getting your own design etched on silicon. If you only need larger features and not the next gen single digit nanometer stuff, you may not be so supply constrained. https://tinytapeout.com/ https://tinytapeout.com/
  - pjc50 5mo ago
    I think you could get it down to three months between weight changes, if you can encode it in metal layers only. The remaining limits are the fab lead time, and the cost of a metal respin (hundreds of thousands to millions of dollars depending on process).
  - happosai 5mo ago
    I think that comment meant it's 2-4 years until local models are good enough that it's worthwhile to burn an ASIC of them. Not that it takes 2-4 years to make an ASIC chip.
- nixon_why69 5mo ago
  Why not have a bunch of SRAM and various operations like "Q4 matmul" in silicon? Model weights and even architectures could still evolve on a platform like that.
  - deleted 5mo ago
    [deleted]
  - ac29 5mo ago
    Doesnt "a bunch of SRAM" top out at maybe a few gigs per chip (with zero area used for logic)? You'd need an order of magnitude more to fit even a fairly weak general purpose LLM model.
  - throwa356262 5mo ago
    I belive that is what NPUs are. The issue is the very huge amount of DRAM and high bandwidth these model require.
- dangus 5mo ago
  If the silicon costs $200-300 and the company throws it away every two years that’s cheaper than a subscription. Also, how many companies will just buy an M6/M7 MacBook Pro with 32GB+ of RAM in a couple of years and get “free” AI along with the workstation they were going to buy anyway?
m101 5mo ago
This will be interesting. I can see some world where it’s used with consumers, but for the most part I think it will be in the cloud and that would make most sense

I would readjust your convictions. We are only 2-4 years away from consumer grade immutable-weight ASICs.

slashdave 5mo ago

We are discussing how rapid development has been, and now you want to freeze your model in silicon?

rogerrogerr 5mo ago
Genuine question from a place of ignorance: what in the silicon pipeline makes it take 2-4years to produce chips with a new model on them? Curious what the process bottleneck is.
- jazzyjackson 5mo ago
  Without being an insider, I imagine that most global fab capacity is contracted out several years in advance. You might be interested in the tiny tape out project, which guides you through the process of getting your own design etched on silicon. If you only need larger features and not the next gen single digit nanometer stuff, you may not be so supply constrained. https://tinytapeout.com/ https://tinytapeout.com/
- pjc50 5mo ago
  I think you could get it down to three months between weight changes, if you can encode it in metal layers only. The remaining limits are the fab lead time, and the cost of a metal respin (hundreds of thousands to millions of dollars depending on process).
- happosai 5mo ago
  I think that comment meant it's 2-4 years until local models are good enough that it's worthwhile to burn an ASIC of them. Not that it takes 2-4 years to make an ASIC chip.
nixon_why69 5mo ago
Why not have a bunch of SRAM and various operations like "Q4 matmul" in silicon? Model weights and even architectures could still evolve on a platform like that.
- deleted 5mo ago
  [deleted]
- ac29 5mo ago
  Doesnt "a bunch of SRAM" top out at maybe a few gigs per chip (with zero area used for logic)? You'd need an order of magnitude more to fit even a fairly weak general purpose LLM model.
- throwa356262 5mo ago
  I belive that is what NPUs are. The issue is the very huge amount of DRAM and high bandwidth these model require.
dangus 5mo ago
If the silicon costs $200-300 and the company throws it away every two years that’s cheaper than a subscription. Also, how many companies will just buy an M6/M7 MacBook Pro with 32GB+ of RAM in a couple of years and get “free” AI along with the workstation they were going to buy anyway?

m101 5mo ago

This will be interesting. I can see some world where it’s used with consumers, but for the most part I think it will be in the cloud and that would make most sense