2 ms·
yes, but it uses 17gb VRAM when running a 50 inference steps generation from prompt. Which is more than I expected since a 7B model sounds little.
by finnjohnsen2 5d ago
yes, but it uses 17gb VRAM when running a 50 inference steps generation from prompt. Which is more than I expected since a 7B model sounds little.