3 ms·

they already did put a model into the silicon and it's crazy fast. https://chatjimmy.ai/ https://chatjimmy.ai/ I'm pretty sure there's a 3 year design goal sta

they already did put a model into the silicon and it's crazy fast. https://chatjimmy.ai/ https://chatjimmy.ai/ I'm pretty sure there's a 3 year design goal starting this year that'll do that to any of the qwen, deepseek, etc models. There's a lot you could do with sped up models of these quality. It might even be bad enough that the real bubble is how much we don't need giant data centers when 80-90% of use cases could just be a silicon chip with a model rather than as you say, bloated SOTA

clickety_clack 6mo ago
It would be pretty cool to have interchangeable usb keys with models on them.
LarsDu88 6mo ago
And this is an asic that is still operating digitally. Imagine a chip with baked it weights that does its math analogue with 20x reduction in number of circuit elements needed to do a multiplication op. If there's a breakthrough in memristors, you could end up with another 20x reduction in circuit elements (get rid of memory bottlnecks, start doing multiplication ops as log transform voltage addition) The ceiling is ultra high for how far AI can go.
- mordae 6mo ago
  Except weights will be unstable - temperature and frequency dependent - and we still have issues delivering analog circuits reliably to the spec. So it would take multiple attempts. But yeah, as soon as the digital models start to plateau, ASICs and then this will happen.
  - ACCount37 6mo ago
    You could SFT for temperature and frequency stability. I'm not even kidding. Modern ML systems already eat errors - what's one more error type for them to eat?
- imtringued 6mo ago
  fp8 and fp4 are already the standard data types, where are you going to find a 20x reduction? At 1 sign bit, 1 exponent and 2 mantissa bits, there is barely any work done during multiplication into a bf16 accumulator. You're performing two shifted adds at this point.

clickety_clack 6mo ago

It would be pretty cool to have interchangeable usb keys with models on them.

LarsDu88 6mo ago

And this is an asic that is still operating digitally. Imagine a chip with baked it weights that does its math analogue with 20x reduction in number of circuit elements needed to do a multiplication op. If there's a breakthrough in memristors, you could end up with another 20x reduction in circuit elements (get rid of memory bottlnecks, start doing multiplication ops as log transform voltage addition) The ceiling is ultra high for how far AI can go.

mordae 6mo ago
Except weights will be unstable - temperature and frequency dependent - and we still have issues delivering analog circuits reliably to the spec. So it would take multiple attempts. But yeah, as soon as the digital models start to plateau, ASICs and then this will happen.
- ACCount37 6mo ago
  You could SFT for temperature and frequency stability. I'm not even kidding. Modern ML systems already eat errors - what's one more error type for them to eat?
imtringued 6mo ago
fp8 and fp4 are already the standard data types, where are you going to find a 20x reduction? At 1 sign bit, 1 exponent and 2 mantissa bits, there is barely any work done during multiplication into a bf16 accumulator. You're performing two shifted adds at this point.