3 ms·You won't be RAM caching much of anything with experts that are 220b parameters worth of layers.by bigyabai 6mo agoYou won't be RAM caching much of anything with experts that are 220b parameters worth of layers.