4 ms·

My guess is that they are doing something like: * use some version of GPT, to preprocess the prompts * send preprocessed prompt to “real” model for inference

• jstarfish 3y ago
  In the last few years I've noticed lying by omission has become the new fun corporate/gen-z internet trend (see also: overfunding/gofundme fraud). Like priests and fund managers, their product is a black box, and there's a lot of mischief you can get into when you're entrusted with one of those. They play a fucked-up game of Rumpelstiltskin, where they mislead by default and only admit the truth if you can guess the right question to ask. You're on the right track, and I too think that's what their actual pipeline looks like, but you're missing a step. I think there's another step where they effectively alter the output of production models by hot-swapping different LORAs (or whatever) to them. This lets them plausibly claim they haven't changed the version of the model, because they haven't messed with the model. They messed with middleware, which nobody knows enough about to press them on. You ask them if anything changed with the model/API, they say no, and leave you to think you're going crazy because shit's just not working like it was last week. Nobody's asking them about changes to middleware though, which genuinely surprises me. I am never the smartest person in the room-- only the most skeptical.