4 ms·
Also, isn't memory a bit moot? At scale I thought that the ASICs frequently sat idle waiting for memory.
by butvacuum 11mo ago
Also, isn't memory a bit moot? At scale I thought that the ASICs frequently sat idle waiting for memory.
- pests 11mo agoYou're doing operations on the memory once it's been transferred to gpu memory. Either shuffling it around various caches or processors or feeding it into tensor cores or other matrix operations. You don't want to be sitting idle.