3 ms·I would figure it needs a lot of kV cache and context size which can be much bigger than the model itselfby alightsoul 3mo agoI would figure it needs a lot of kV cache and context size which can be much bigger than the model itself