3 ms·

I think the problem is that SNNs aren't suited to standard von neuman architecture. I'm in total agreement. That amount of power and compute thrown at language

by cglan 3y ago

I think the problem is that SNNs aren't suited to standard von neuman architecture. I'm in total agreement. That amount of power and compute thrown at language models, and ML in general is shocking relative to the capabilities. For GPT4, it's staggering. Probably on the order of KWHs in power alone for basic tasks. I think a properly modeled architecture would look something like the actor model represented in hardware, where each actor is basically a small extremely power efficient cpu but unfortunately you wouldn't be able to pull that off the shelf.

bob1029 3y ago
> I think the problem is that SNNs aren't suited to standard von neuman architecture. This is where we begin to disagree. If you are content with moving from actual real time to synthetic time, all of the potential frustrations fall away again. Think about online vs offline rendering in computer graphics. Theoretically, you could render all of Toy Story on a RasPi4. Most of us would be willing to wait days for answers, assuming the batch size is acceptable and the answers are correct.