3 ms·
Maybe it’s a cost thing. Short circuit inference whenever possible.
by jpalawaga 3y ago
Maybe it’s a cost thing. Short circuit inference whenever possible.
- ovohcoc 3y agoit actually causes more inference to take place. the prompt goes through a safety model in parallel with the real model, then the response goes through a safety model
- KTibow 3y agoThat part isn't what they were talking about, they were just talking about the practice of not responding at the LLM level