6 ms·
You definitely need more RAM if you are not satisfied with small context windows, especially if the weights take a large % of the total memory to boot.
by orsorna 11d ago
You definitely need more RAM if you are not satisfied with small context windows, especially if the weights take a large % of the total memory to boot.