3 ms·
That's true, but not my point. My point is that if the request specifies GPT-3.5, Nitro knows that it cannot possibly serve that model, so anything other than r
by p-e-w 3y ago
That's true, but not my point. My point is that if the request specifies GPT-3.5, Nitro knows that it cannot possibly serve that model, so anything other than returning an error is simply lying to the client, which is a really bad idea.
- trifurcate 3y ago> which is a really bad idea. Why?
- alpark3 3y agoBecause it's lying to the client?
- trifurcate 3y agoAnd why is that bad? Your mindset would mean that Windows would have next to no backwards compatibility, for instance.
- p-e-w 3y agoBecause if the client specifically requests GPT-3.5, but is silently being served something else instead, the client will rely on having GPT-3.5 capabilities without them actually being available, which is a recipe for breakage.
- brigadier132 3y agoYou do understand that the client will be written by the same people setting up the inference server?