48 ms·
It uses variable grouped query attention, so it reduces the memory required to fetch during inference.
by GaggiX 3y ago
It uses variable grouped query attention, so it reduces the memory required to fetch during inference.
loading thread…
48 ms·