3 ms·
Can you explain how context fits into this picture by any chance? I sort of understand the vram requirement for the model itself, but it seems like larger cont
by johntash 1y ago
Can you explain how context fits into this picture by any chance? I sort of understand the vram requirement for the model itself, but it seems like larger context windows increases the ram requirement by a lot more?