3 ms·
I think they may be using low precision quants during peak load, which makes the output sometimes complete garbage. I'm most convinced though that the main way
by taylorfinley 3y ago
I think they may be using low precision quants during peak load, which makes the output sometimes complete garbage. I'm most convinced though that the main way they destroyed gpt-4 is the extremely-long system prompt they've been using since making it all-in-one, it has made gpt-4 all but unusable for even mildly complex tasks. Those system prompt tokens are powerful tokens and they're wasted on telling the model how to prompt dall-e without offending anyone when my prompt is actually about generating a unittest. Using the "gpt classic" "gpt" it's much better, but the usage limit hits fast and all the network errors are irritating.