2 ms·
Title for the back of the class: "Prompts sometimes return null" I would be very cautious to attribute any of this to black box LLM weight matrices. Models li
by bob1029 7mo ago
Title for the back of the class:
"Prompts sometimes return null"
I would be very cautious to attribute any of this to black box LLM weight matrices. Models like GPT and Opus are more than just a single model. These products rake your prompt over the coals a few times before responding now. Telling the model to return "nothing" is very likely to perform to expectation with these extra layers.
- tiku 7mo agoThanks, I was already distracted after the first sentence, hoping there would be a good explanation.
- frde_me 7mo agoOut of curiosity, are there any sources to there being a significant amount of other steps before being fed into the weights Security guards / ... are the obvious ones, but do you mean they have branching early on to shortcut certain prompts?
- bob1029 7mo ago> do you mean they have branching early on to shortcut certain prompts? Putting a classifier in front of a fleet of different models is a great way to provide higher quality results and spend less energy. Classification is significantly cheaper than generation and it is the very first thing you would do here. A default, catch-all model is very expensive, but handles most queries reasonably well. The game from that point is to aggressively intercept prompts that would hit the catch-all model with cheaper, more targeted models. I have a suspicion that OAI employs different black boxes depending on things like the programming language you are asking it to use.
- frde_me 7mo agoAren't you describing why they use mixture of experts? Where a sub-set of weights are activated depending on the query?