4 ms·
Electrical analog systems are since there's 1001 ways to create or conduct away electricity that adds noise, chemical ones might not suffer nearly as much too.
by moffkalast 2y ago
Electrical analog systems are since there's 1001 ways to create or conduct away electricity that adds noise, chemical ones might not suffer nearly as much too.
There's also an interesting bit I've observed with LLMs, quantization in range of 4-8 bits doesn't reduce performance as much as it mainly reduces consistency. If you generate an answer a bunch of times and take the average you'll end up with roughly the same result as an fp16/32 would do every time. In nature, being inconsistent is usually weighed negatively with death... so that's probably why it hasn't caught on even if it is more efficient. Or this is enough of a different abstraction that we can't draw parallels anyway.