4 ms·
How many GBs worth of logits would you need to reverse engineer their model? Also, if it’s a conglomerate of models that they’re using, you’d end up in a blind
by leobg 3y ago
How many GBs worth of logits would you need to reverse engineer their model? Also, if it’s a conglomerate of models that they’re using, you’d end up in a blind alley.
- gwern 3y agoConsidering how well simply reusing GPT-3.5/4 outputs has worked to juice rival model performance, at least in relatively narrow benchmarking, I dunno how many GBs it'd take, but probably not that many, and it's a straightforward easy way to turn money into performance at a much lower cost than buying a few thousand more H100s.
- leobg 3y agoOpenAI does not strike me as a company that would be naive about this. Didn’t they just recently manipulate the outputs of an endpoint when they realized people were misusing it? (“CatGPT”) The most sinister interpretation is that the logits are a red herring. People who are tied up in stealing them aren’t free to do actual rival work.
- joegibbs 3y agoWhat happened with CatGPT?
- mike_hearn 3y agoSomeone was abusing the ChatGPT backend to avoid paying for the API calls. When OpenAI's abuse team noticed this they modified the system prompt used for those requests to tell the AI to pretend it was a cat, and then monitored the Discord channel of the abusers for the lulz.
- msp26 3y ago>When OpenAI's abuse team noticed this they modified the system prompt used for those requests to tell the AI to pretend it was a cat, and then monitored the Discord channel of the abusers for the lulz. Please I need a source for this it sounds hilarious.
- gwern 3y agoWell, not recently. That was from at the ChatGPT launch when they were still using a quick hack that made the client too-privileged, so more or less a year ago. However, I definitely am wondering if they have poisoned the logits somehow. As long as the logits rank the tokens in order (are monotonic), preserving their utility for the suggested applications like ranking autocompletions, you presumably could screw with the magnitudes arbitrarily.