4 ms·
People are working on that [1]. In some sense, it's a step back to analog computing. Add/multiply is possible to do directly in memory with voltages, but it's l
by sigmoid10 3y ago
People are working on that [1]. In some sense, it's a step back to analog computing. Add/multiply is possible to do directly in memory with voltages, but it's less versatile (and stable) than digital computing. So you can't do all calculations in a neural network that way, meaning some digital components will always be necessary. But I'm pretty sure analog will make a comeback for AI chips sooner or later.
[1] https://www.nature.com/articles/s41586-023-06337-5 https://www.nature.com/articles/s41586-023-06337-5
- irrelative 3y agoHadn't thought about it this way before, but given that LLMs are auto regressive (use their own data for next data), they're sensitive to error drift in ways that are rather similar to analog computers.
- trebligdivad 3y agoTrinary however is an interesting middle; people have built trinary hardware long ago; it feels like you could make natively trinary hardware for something like this; it might even be quite a win.
- thsksbd 3y agoCan you make a "CMOS" three voltage level circuit though? One where the only current flow is when the state changes? Im not in this field but that's a question that's been bugging me for a while. Off you can't do this wouldn't energy consumption balloon?
- deleted 3y ago[deleted]
- neomantra 3y agoMy friend was working on this in the mid-90s at Texas Instruments. Not sure what the underlying semiconductors were, but it did involve making ternary logic via voltage levels. Just searched a bit and found this TI datasheet which might be an example of it (high logic, low logic, high impedance), but maybe not: https://www.ti.com/lit/ds/symlink/sn74act534.pdf https://www.ti.com/lit/ds/symlink/sn74act534.pdf
- int_19h 3y agoPeople haven't built reliable ternary electronics, though. Soviets tried with Setun, but they eventually had to resort to emulating each trit with two hardware bits (and wasting one state out of the possible four).
- zcw100 3y agoReminds me of my father saying something about how vacuum tubes are great integrators.