4 ms·

I hope we can buy Cerebras cards one day. Imagine buying a ~$500 AI card for your desktop and having easy access to 70B+ models (the price is speculative/made u

I hope we can buy Cerebras cards one day. Imagine buying a ~$500 AI card for your desktop and having easy access to 70B+ models (the price is speculative/made up).

chessgecko 2y ago
One day is doing some heavy heavy lifting here, we’re currently off by ~3-4 orders of magnitude…
- accrual 2y ago
  Thank you, for the reality check! :)
  - thomashop 2y ago
    We have moved 2 orders of magnitude in the last year. Not that unreasonable
- grahamj 2y ago
  So 1000-10000 days? ;)
  - Yizahi 2y ago
    In a few thousand days (c) St. Altman
    - grahamj 2y ago
      lol I almost said that too
killingtime74 2y ago
Maybe not $500, but $500,000
danpalmer 2y ago
I believe pricing was mid 6 figures per machine. They're also like 8U and water cooled I believe. I doubt it would be possible to deploy one outside of a fairly top tier colo facility where they have the ability to support water cooling. Also imagine learning a new CUDA but that is designed for another completely different compute model.
- initplus 2y ago
  Yeah you can see the cooling requirements by looking at their product images. https://cerebras.ai/wp-content/uploads/2021/04/Cerebras_Product_CS1-1.png https://cerebras.ai/wp-content/uploads/2021/04/Cerebras_Prod... Thing is nearly all cooling. And look at the diameter on the water cooling pipes. Airflow guides on the fans are solid steel. Apparently the chip itself measures 21.5cm^2. Insane.
- bboygravity 2y ago
  That means it'll be close to affordable in 3 to 5 years if we follow the curve we've been on for the past decades.
  - schoen 2y ago
    How have power and cooling been doing with respect to chip improvements? Have power requirements per operation been coming down rapidly, as other features have improved? My recollection from PC CPUs is that we've gotten many more operations per second, and many more operations per second per dollar, but that the power and corresponding cooling requirements for the CPUs have tended to go up as well. I don't really know what power per operation has looked like there. (I guess it's clearly improved, though, because it seems like the power consumption of a desktop PC has only increased by a single order of magnitude, while the computational capacity has increased by more than that.) A reason that I wonder about this in this context is that people are saying that the power and cooling requirements for these devices are currently enormous (by individual or hobbyist standards, not by data center standards!). If we imagine a Moore's Law-style improvement where the hardware itself becomes 1/10 or 1/100 of its current price, would we expect the overall power consumption to be similarly reduced, or to remain closer to its current levels?
visarga 2y ago
You still have to pay for the memory. The Cerebras chip is fast because they use 700x more SRAM than, say, A100 GPUs. Loading the whole model in SRAM every time you compute one token is the expensive bit.

I hope we can buy Cerebras cards one day. Imagine buying a ~$500 AI card for your desktop and having easy access to 70B+ models (the price is speculative/made up).

chessgecko 2y ago

One day is doing some heavy heavy lifting here, we’re currently off by ~3-4 orders of magnitude…

accrual 2y ago
Thank you, for the reality check! :)
- thomashop 2y ago
  We have moved 2 orders of magnitude in the last year. Not that unreasonable
grahamj 2y ago
So 1000-10000 days? ;)
- Yizahi 2y ago
  In a few thousand days (c) St. Altman
  - grahamj 2y ago
    lol I almost said that too

killingtime74 2y ago

Maybe not $500, but $500,000

danpalmer 2y ago

I believe pricing was mid 6 figures per machine. They're also like 8U and water cooled I believe. I doubt it would be possible to deploy one outside of a fairly top tier colo facility where they have the ability to support water cooling. Also imagine learning a new CUDA but that is designed for another completely different compute model.

initplus 2y ago
Yeah you can see the cooling requirements by looking at their product images. https://cerebras.ai/wp-content/uploads/2021/04/Cerebras_Product_CS1-1.png https://cerebras.ai/wp-content/uploads/2021/04/Cerebras_Prod... Thing is nearly all cooling. And look at the diameter on the water cooling pipes. Airflow guides on the fans are solid steel. Apparently the chip itself measures 21.5cm^2. Insane.
bboygravity 2y ago
That means it'll be close to affordable in 3 to 5 years if we follow the curve we've been on for the past decades.
- schoen 2y ago
  How have power and cooling been doing with respect to chip improvements? Have power requirements per operation been coming down rapidly, as other features have improved? My recollection from PC CPUs is that we've gotten many more operations per second, and many more operations per second per dollar, but that the power and corresponding cooling requirements for the CPUs have tended to go up as well. I don't really know what power per operation has looked like there. (I guess it's clearly improved, though, because it seems like the power consumption of a desktop PC has only increased by a single order of magnitude, while the computational capacity has increased by more than that.) A reason that I wonder about this in this context is that people are saying that the power and cooling requirements for these devices are currently enormous (by individual or hobbyist standards, not by data center standards!). If we imagine a Moore's Law-style improvement where the hardware itself becomes 1/10 or 1/100 of its current price, would we expect the overall power consumption to be similarly reduced, or to remain closer to its current levels?

visarga 2y ago

You still have to pay for the memory. The Cerebras chip is fast because they use 700x more SRAM than, say, A100 GPUs. Loading the whole model in SRAM every time you compute one token is the expensive bit.