3 ms·
I agree. But I think do think they use a tiered approach: they have an uncensored model at the very bottom, to which the queries go to first. And then there's
by precompute 4y ago
I agree. But I think do think they use a tiered approach: they have an uncensored model at the very bottom, to which the queries go to first. And then there's the "clean" model, and (for lack of a better explanation) the two outputs are then "XOR"d to get the clean public version.
- bob1029 4y agoI agree the core models are uncensored (as this would introduce a lot of noise), but I don't think any queries are allowed to touch them before a cheaper model first performs screening. If someone is spamming toilet humor, there is no reason to exercise the full 175B+ parameter model each time. The only reason ChatGPT is affordable is because of caching, filtering bad prompts, etc. The underlying LLM would be far too expensive if everyone was hitting it 1:1.
- precompute 4y agoAgree 100%. Have you noticed any lag with more "difficult" queries?
- bob1029 4y agoYes. I think there are multiple levels of LLM for each use case now. The stuff that absolutely must run on 100B+ models can be pre-classified by something like Ada/Babbage. Attempts at arithmetic or dimensional analysis can be binned and shipped to a monster model. Anything that is more routine information retrieval needs maybe goes to a 7B parameter model. Lots of models with lots of temperature levels/hyperparameters/etc is the only way to achieve the kinds of performance we are seeing. The secret sauce is looking increasingly like a huge classification layer. Nothing you see in ChatGPT is as it appears. Lengthy conversations are managed with recursive summarization. Every conversational turn is potentially handled by a different LLM.