3 ms·

Their numbers look too good to be true, they have no identified customers and the whole site is generated, but I think the principle behind it is good. If they

by tancop 2mo ago

Their numbers look too good to be true, they have no identified customers and the whole site is generated, but I think the principle behind it is good. If they can pull off the error correction needed to make analog reliable we might have a great new option for cheaper more eco friendly AI. Then again it could turn out to be a total scam.

vatsachak 2mo ago
Like the numbers they claim could literally make LLMs 20x profitable. If it were true then why isn't every AI company trying to buy them out?
- refulgentis 2mo ago
  c.f. https://news.ycombinator.com/item?id=49403836 https://news.ycombinator.com/item?id=49403836, then from there, you'd need to see a couple of orders of magnitude before it's tractable for LLMs. Bottom of page linked from HN (currently https://www.mythic.ai/ https://www.mythic.ai/) indicates they're hoping to demonstrate something that could that in 2028 or later, and both Nvidia and Cerebra are looking at 10x'ing models to 10T+ plus in 2027. So they may never catch up on LLMs. They're a good fit for the companies they're working with and have taken investment from, ex. Toyota, that aren't doing LLMs.
  - vatsachak 2mo ago
    They plan to be able to run 1T parameter models next year https://www.mythic.ai/vanguard https://www.mythic.ai/vanguard Seems big, IF true
- mdp2021 2mo ago
  Notice that they do not talk about SRAM when they present the M1 chiplets - but you have to store the kv-cache etc. somewhere to run LLMs. The technology that could run LLMs should be the "Vanguard", but as the homepage says, "the M1 (scope: Edge/Cameras/Drones) is there, the Vanguard should be a reality in 2027".
- imtringued 2mo ago
  I'm pretty sure the answer is that their chip is uniquely unsuited for LLMs, probably because the write speed might be horrendously slow. Most likely too slow for storing the context window in multi user workloads.
mdp2021 2mo ago
> Their numbers look too good to be true Why? I have not seen anything outlandish for a NN implementation (vs a NN simulation). > can pull off the error correction There lie the issues that have not been mentioned, the solutions not explained. Analog computing means: * costly digital-to-analog at the input and analog-to-digital at the output; * sensitivity to environmental conditions such as temperature; * signal dispersion hence the need to boost it in the path. Maybe checking the patents they registered?
- dgfl 2mo ago
  Temperature is a relatively trivial issue because it can either be compensated or the chip simply kept at a constant temperature (they’re low power chips anyway, throttling could be skipped to simplify the design). The ADCs and DACs are indeed the main thing though. The whole analog compute game is about making those power efficient and scaled enough that the math still works out in your favor. The demos do work though, this company is far from the only one; see [1] or [2] for example. Flash NAND can routinely be bought with 4 bits per cell, perhaps even 5 soon (QLC and PLC drives). Since it has been proven to store 4 digital bits at production scale, I’m willing to bet an analog architecture running an LLM should be able to yield the performance analog of a 4-8 bit quantized model. Where in that 4-8 range is pretty crucial, but it depends on the specific design. [1] https://www.nature.com/articles/s41928-023-01010-1 https://www.nature.com/articles/s41928-023-01010-1 [2] https://www.nature.com/articles/s41586-022-04992-8 https://www.nature.com/articles/s41586-022-04992-8
refulgentis 2mo ago
My 15 second read of just the front page aligned with you, but when I saw replies pushing back, I went back and loaded News, then cross-verified some of the claims. It's real.
SkyPuncher 2mo ago
I went down this rabbit hole earlier in the year. You generally don’t need error correction. Instead, you do a retraining of sorts where you basically fine tune the model on the target chip architecture. Because these systems are so complex, errors tend to localize without affecting overall output. My understanding is the hardest part is actually noise in the analog system.
mdp2021 2mo ago
(Moderation, the post https://news.ycombinator.com/item?id=49404099 https://news.ycombinator.com/item?id=49404099 from SkyPuncher is legit. Automated filter? Maybe you can give us the ability to vouch?)
orbital-decay 2mo ago
Bulk bitwise operations in a memory array driven by an out-of-spec controller is a thing, but I'm not sure why they're using NAND for this and not DRAM like everyone else, and why it's so small (just 80M weights). And the page is insufferable and doesn't tell anything about how it works. Everything about it seems off.

by tancop 2mo ago

vatsachak 2mo ago

Like the numbers they claim could literally make LLMs 20x profitable. If it were true then why isn't every AI company trying to buy them out?

refulgentis 2mo ago
c.f. https://news.ycombinator.com/item?id=49403836 https://news.ycombinator.com/item?id=49403836, then from there, you'd need to see a couple of orders of magnitude before it's tractable for LLMs. Bottom of page linked from HN (currently https://www.mythic.ai/ https://www.mythic.ai/) indicates they're hoping to demonstrate something that could that in 2028 or later, and both Nvidia and Cerebra are looking at 10x'ing models to 10T+ plus in 2027. So they may never catch up on LLMs. They're a good fit for the companies they're working with and have taken investment from, ex. Toyota, that aren't doing LLMs.
- vatsachak 2mo ago
  They plan to be able to run 1T parameter models next year https://www.mythic.ai/vanguard https://www.mythic.ai/vanguard Seems big, IF true
mdp2021 2mo ago
Notice that they do not talk about SRAM when they present the M1 chiplets - but you have to store the kv-cache etc. somewhere to run LLMs. The technology that could run LLMs should be the "Vanguard", but as the homepage says, "the M1 (scope: Edge/Cameras/Drones) is there, the Vanguard should be a reality in 2027".
imtringued 2mo ago
I'm pretty sure the answer is that their chip is uniquely unsuited for LLMs, probably because the write speed might be horrendously slow. Most likely too slow for storing the context window in multi user workloads.

mdp2021 2mo ago

> Their numbers look too good to be true Why? I have not seen anything outlandish for a NN implementation (vs a NN simulation). > can pull off the error correction There lie the issues that have not been mentioned, the solutions not explained. Analog computing means: * costly digital-to-analog at the input and analog-to-digital at the output; * sensitivity to environmental conditions such as temperature; * signal dispersion hence the need to boost it in the path. Maybe checking the patents they registered?

dgfl 2mo ago
Temperature is a relatively trivial issue because it can either be compensated or the chip simply kept at a constant temperature (they’re low power chips anyway, throttling could be skipped to simplify the design). The ADCs and DACs are indeed the main thing though. The whole analog compute game is about making those power efficient and scaled enough that the math still works out in your favor. The demos do work though, this company is far from the only one; see [1] or [2] for example. Flash NAND can routinely be bought with 4 bits per cell, perhaps even 5 soon (QLC and PLC drives). Since it has been proven to store 4 digital bits at production scale, I’m willing to bet an analog architecture running an LLM should be able to yield the performance analog of a 4-8 bit quantized model. Where in that 4-8 range is pretty crucial, but it depends on the specific design. [1] https://www.nature.com/articles/s41928-023-01010-1 https://www.nature.com/articles/s41928-023-01010-1 [2] https://www.nature.com/articles/s41586-022-04992-8 https://www.nature.com/articles/s41586-022-04992-8

refulgentis 2mo ago

My 15 second read of just the front page aligned with you, but when I saw replies pushing back, I went back and loaded News, then cross-verified some of the claims. It's real.

SkyPuncher 2mo ago

I went down this rabbit hole earlier in the year. You generally don’t need error correction. Instead, you do a retraining of sorts where you basically fine tune the model on the target chip architecture. Because these systems are so complex, errors tend to localize without affecting overall output. My understanding is the hardest part is actually noise in the analog system.

mdp2021 2mo ago

(Moderation, the post https://news.ycombinator.com/item?id=49404099 https://news.ycombinator.com/item?id=49404099 from SkyPuncher is legit. Automated filter? Maybe you can give us the ability to vouch?)

orbital-decay 2mo ago

Bulk bitwise operations in a memory array driven by an out-of-spec controller is a thing, but I'm not sure why they're using NAND for this and not DRAM like everyone else, and why it's so small (just 80M weights). And the page is insufferable and doesn't tell anything about how it works. Everything about it seems off.