3 ms·
Yes, but: 1) They are investing Loony Toons levels of money and using Loony Toons financing a lot of the time for DC buildout. Those are actual physical buildi
by FabCH 3mo ago
Yes, but:
1) They are investing Loony Toons levels of money and using Loony Toons financing a lot of the time for DC buildout. Those are actual physical buildings that once built out, exist and don’t need to be built again.
2) They are pointedly ignoring FPGA and ASIC. With the current model quality, would it really be so bad to burn Claude irrevocably on a chip and have a non-modifiable, cheap to mass produce, order of magnitude faster Claude-in-a-chip? This is what happened to Bitcoin ultimately, there are huge performance gains we know exist lying on the table just because they don’t exist for training. And even for training, TPUs make the first steps in that direction.
- marginalia_nu 3mo ago> They are pointedly ignoring FPGA and ASIC. With the current model quality, would it really be so bad to burn Claude irrevocably on a chip and have a non-modifiable, cheap to mass produce, order of magnitude faster Claude-in-a-chip? This is what happened to Bitcoin ultimately, there are huge performance gains we know exist lying on the table just because they don’t exist for training. And even for training, TPUs make the first steps in that direction. Yeah this is copium if I ever saw it.
- fragmede 3mo agoThat's the approach https://www.cerebras.ai/ https://www.cerebras.ai/ is taking.
- rightbyte 3mo agoIsn't GPUs a purpose made ASIC for parallell matrix math allready?
- FabCH 3mo agoNo? You can program a GPU, you can’t program ASIC. TPUs are closer to ASIC than GPUs.
- 3836293648 3mo agoAn ASIC won't help until models are compute bound. They're still memory bound at all levels.
- FabCH 3mo agoModels are memory bound. Startups burning through VC money to build massive warehouses full of hardware those models need to run on currently seem to be quite power grid connector bottlenecked. A more efficient chip would help.
- pier25 3mo agoInference is probably the cheapest part of it all. The buildings will last for years, decades even, but the GPUs need to be replaced every 2-3 years in perpetuity.
- FabCH 3mo agoTrue, but only because they are choosing to actively burn VC money on training. But it’s not like the models we have now would stop to exist if training stopped. Other than the occasional retraining to get the latest data in, if they stopped wanton experimentation with models, that admittedly is pushing the models forward, the training costs could plummet and inference would be the thing to optimize and scale.