3 ms·
Does keeping it on one stick make it more performant? I have a epyc server with 1tb of 64 gig sticks and a 3060, looking to maximize what models I can run on it
by ramgine 7mo ago
Does keeping it on one stick make it more performant? I have a epyc server with 1tb of 64 gig sticks and a 3060, looking to maximize what models I can run on it
- nurettin 7mo agoIn my case it definitely helps, but keeping 64k context window would probably require more than 64 GB.