5 ms·
I wonder if it's possible to design the next AI systems along with the hardware at the same time. For example, maybe by focusing on more approaches like mixture
by ilaksh 3y ago
I wonder if it's possible to design the next AI systems along with the hardware at the same time. For example, maybe by focusing on more approaches like mixture of experts or similar, there are ways to keep much of the data close to the cores that operate on it.
- CyberDildonics 3y agoThat's called CPU cache. It doesn't require "mixture of experts" (whatever that would mean) it just needs transistors for SRAM.
- ilaksh 3y agoThat's one example of the more general category of what I am talking about. But I was trying to get just a little more specific.
- CyberDildonics 3y agoCan you give another example and explain how "mixture of experts" gets data closer to a CPU?
- ilaksh 3y agoI'm talking about GPUs and don't know the details very well. It was a rough idea.
- CyberDildonics 3y agothe more general category of what I am talking about. But I was trying to get just a little more specific Can you get more specific then? Can you give any details or any overview? There must have been some information that led you to post this originally, can you link it?
- yazaddaruvala 3y agoThere are many possible future architectural changes that could help. Some are more feasible than others but all require fundamental advancement to be useful and cost effective. Some of them include: - GPU - CPU shared memory (already possible just not yet the standard) - Higher DRAM bandwidth (already possible, just not yet a priority) - system on chip FPGAs (always possible just very expensive to fit “AI models”) - SOC NVM. Ideally even NVM on the same wafer as the GPU and CPU (possible today but requires a lot of work on the yield. NVM would take up a lot of real estate that could ruin yield). - analog circuits - new semi-conductors / photonics - memristors