2 ms·
In my experience with other regular models, once the context starts to fill up, quality starts to degrade. wouldn't getting batched at the end of a batch, have
by tough 1y ago
In my experience with other regular models, once the context starts to fill up, quality starts to degrade.
wouldn't getting batched at the end of a batch, have a similar -effect- on the results, where your prompt might recieve overall less attention focused into it, if the context window is almost full?
Idk just going by the vibes