For 128k context (even with a 7B model), I don't think 8GB is nearly enough. I've heard there might be tricks to get it under 24GB... but I haven't personally s

For 128k context (even with a 7B model), I don't think 8GB is nearly enough. I've heard there might be tricks to get it under 24GB... but I haven't personally seen it under about 28GB, IIRC. Definitely not heard of it being possible with 8GB.