4 ms·
Maybe AMD will do a 3D memory layer for this chip for the kv cache. Possibly even something more dense than SRAM, like cgram, when it's ready. Could this enabl
by petra 2mo ago
Maybe AMD will do a 3D memory layer for this chip for the kv cache. Possibly even something more dense than SRAM, like cgram, when it's ready.
Could this enable a reasonable context size ?