2 ms·
I think it obviously will, what would be the benefit to spinning up more than one copy?
by nsvd2 6mo ago
I think it obviously will, what would be the benefit to spinning up more than one copy?
- wtallis 6mo agoIt should only need to load one copy of the weights, but each tab/site will need a separate context and KV cache.