8 ms·

Note that this also applies to GPT models on the API > That risk extended to users of ChatGPT Free, Plus, and Pro, as well as users of OpenAI’s application pro

by efskap 1y ago

Note that this also applies to GPT models on the API > That risk extended to users of ChatGPT Free, Plus, and Pro, as well as users of OpenAI’s application programming interface (API), OpenAI said. This seems very bad for their business.

Kokouane 1y ago
If you were working with code that was proprietary, you probably shouldn't of been using cloud hosted LLMs anyways, but this would seem to seal the deal.
- larrymcp 1y ago
  I think you probably mean "shouldn't have". There is no "shouldn't of".
  - DecentShoes 1y ago
    Who cares?
  - knicholes 1y ago
    I care.
  - rimunroe 1y ago
    Which gives you an opening for the excellent double contraction “shouldn’t’ve”
    - b112 1y ago
      The letter H deserves better.
      - worthless-trash 1y ago
        I think we gave it too much leeway in the word sugar.
    - mananaysiempre 1y ago
      The funniest part is that in that contraction the first apostrophe does denote the elision of a vowel, but the second one doesn’t, the vowel is still there! So you end up with something like [nʔəv], much like as if you had—hold the rotten vegetables, please—“shouldn’t of” followed by a vowel. Really, it’s funny watching from the outside and waiting for English to finally stop holding it in and get itself some sort of spelling reform to meaningfully move in a phonetic direction. My amateur impression, though, is that mandatory secondary education has made “correct” spelling such a strong social marker that everybody (not just English-speaking countries) is essentially stuck with whatever they have at the moment. In which case, my condolences to English speakers, your history really did work out in an unfortunate way.
      - roywiggins 1y ago
        We had a spelling reform or two already, they were unfortunately stupid, eg doubt has never had the b pronounced in English. https://en.m.wiktionary.org/wiki/doubt https://en.m.wiktionary.org/wiki/doubt That said, phonetic spelling reform would of course privilege the phonemes as spoken by whoever happens to be most powerful or prestigious at the time (after all, the only way it could possibly stick is if it's pushed by the sufficiently powerful), and would itself fall out of date eventually anyway.
      - jdbernard 1y ago
        > but the second one doesn’t, the vowel is still there! Isn't the "a" in "have" elided along with the "h?" Shouldn't've Should not have What am I missing?
        jack09268 1y ago
        Even though the vowel "a" is dropped from the spelling, if you actually say it out loud, you do pronounce a vowel sound when you get to that spot in the word, something like "shouldn'tuv", whereas the "o" in "not" is dropped from both the spelling and the pronounciation.
        SAI_Peregrinus 1y ago
    - theoreticalmal 1y ago
      My favorite variation of this is “oughtn’t to’ve”
  - amanaplanacanal 1y ago
    That used to be the case, but "shouldn't of" is definitely becoming more popular, even if it seems wrong. Languages change before our eyes :)
- YetAnotherNick 1y ago
  Why not? Assuming you believe you can use any cloud for backup or Github for code storage.
  - solaire_oa 1y ago
    IIUC one reason is that prompts and other data sent to 3rd party LLM hosts have the chance to be funneled to 4th party RLHF platforms, e.g. Sagemaker, Mechanical Turks, etc. So a random gig worker could be reading a .env file the intern uploaded.
    - YetAnotherNick 1y ago
      What do you mean by chance? It's clear that if users have not opted out from training the models, it would be used. If they have opted out, it wont be used. And most of the users are in first bucket. Just because training on data is opt out doesn't mean business can't trust it. Not the best for user's privacy though.
- gpm 1y ago
  I think it's fair to question how proprietary your data is. Like there's the algorithm by which a hedge fund is doing algorithmic trading, they'd be insane to take the risk. Then there's the code for a video game, it's proprietary, but competitors don't benefit substantially from an illicit copy. You ship the compiled artifacts to everyone, so the logic isn't that secret. Copies of the similar source code have linked before with no significant effects.
  - FuckButtons 1y ago
    AFAIK, the actual trading algorithms themselves aren’t usually that far from what you can find in a textbook, their efficacy is mostly dictated by market conditions and the performance characteristics of the implementation / system as a whole.
    - short_sells_poo 1y ago
      This very much "depends". Many algo strategies are indeed programmatically simple (e.g. use some sort of moving average), but the parametrization and how it's used is the secret sauce and you don't want that information to leak. They might be tuned to exploit a certain market behavior, and you want to keep this secret since other people targeting this same behavior will make your edge go away. The edge can be something purely statistical or it can be a specific timing window that you found, etc. It's a bit like saying that a Formula 1 engine is not that far from what you'd find in a textbook. While it's true that it shares a lot of properties with a generic ICE, the edge comes from a lot of proprietary research that teams treat as secret and definitely don't want competitors to find out.
  - short_sells_poo 1y ago
    Most (all?) hedge funds that use AI models explicitly run in-house. People do use commercial LLMs, but in cases where the LLMs are not run in-house, it's against the company policy to upload any proprietary information (and generally this is logged and policed). A lot of the use is fairly mundane and basically replaces junior analysts. E.g. it's digesting and summarizing the insane amounts of research that is produced. I could ask an intern to summarize the analysis on platinum prices over the last week, and it'll take them a day. Alternatively, I can feed in all the analysis that banks produce to an LLM and have it done immediately. The data fed in is not a trade secret really, and neither is the output. What I do with the results is where the interesting things happen.
neilv 1y ago
Some established businesses will need to review their contracts, regulations, and risk tolerance. And wrapper-around-ChatGPT startups should double-check their privacy policies, that all the "you have no privacy" language is in place.
- Wowfunhappy 1y ago
  > And wrapper-around-ChatGPT startups should double-check their privacy policies, that all the "you have no privacy" language is in place. If a court orders you to preserve user data, could you be held liable for preserving user data? Regardless of your privacy policy.
  - woliveirajr 1y ago
    Yes. If your agreement with the end user says that you won't collect and store data, you're responsible for it. If you can't provide it (even if due to a court order), you have to adjust your contract. Your users aren't obligated to know that you're using open ai or other provider.
  - bilbo0s 1y ago
    No. It’s a legal court order. This, however, is horrible for AI regardless of whether or not you can sue.
    - dcow 1y ago
      In the US you absolutely can challenge everything up and including the constitutionality of court orders. You may be swiftly dismissed if nobody thinks you have a valid case, but you can try.
      - deleted 1y ago
        [deleted]
  - gpm 1y ago
    I don't think the suit would be against you preserving it, it would be against you falsely representing that you aren't preserving it. A court ordering you to stop selling pigeons doesn't mean you can keep your store for pigeons open and pocket the money without delivering pigeons.
dinobones 1y ago
How? This is retention for legal risk, not for training purposes. They can still have legal contracts with other companies, that stipulate that they don't train on any of their data.
- CryptoBanker 1y ago
  Right, because companies always follow the letter of their contracts.
- Take8435 1y ago
  ...Data that is kept can be exfiltrated.
  - fn-mote 1y ago
    Cannot emphasize this enough. If your psychologist’s records can be held for ransom, surely your ChatGPT queries will end up on the internet someday. Do search engine companies have this requirement as well? I remember back in the old days deanonymizing “anonymous” query logs was interesting. I can’t imagine there’s any secrecy left today.
    - SchemaLoad 1y ago
      I recently had a high school assignment document get posted on a bunch of sites that sell homework help. As far as I know that document was only ever submitted directly to the assignment upload page. So somewhere along the line, I suspect on the plagiarism checker service, there was a hack and then 10 years later some random school assignment with my name on it is all over the place.
      - genewitch 1y ago
        How did you find out?
- paxys 1y ago
  Your employees' seemingly private ChatGPT logs being aired in public during discovery for a random court case you aren't even involved in is absolutely a business risk.
m3kw9 1y ago
Not when people have nowhere else to go, pretty much you cannot escape it, it’s too convenient to not use now. You think no other AI chat providers doesn’t need to do this?
johnQdeveloper 1y ago
> This seems very bad for their business. Well, it is gonna be all _AI Companies_ very soon so unless everyone switches to local models which don't really have the same degree of profitability as a SaaS, its probably not going to kill a company to have less user privacy because tbh people are used to not having privacy these days on the internet. It certainly will kill off the few companies/people trusting them with closed source code or security related stuff but you really should not outsource that anywhere.
- csomar 1y ago
  Did an American court just destroy all American AI companies in favor of open weight Chinese models?
  - thot_experiment 1y ago
    afaik only OpenAI is enjoined in this
    - csomar 1y ago
      Sure. But this means the rest of the AI companies are exposed to such risk; and there aren't that many of them (grok/gemini/anthropic).
    - baby_souffle 1y ago
      > afaik only OpenAI is enjoined in this For now. This is going to devolve into either "openAI has to do this, so you do too" or "we shouldn't have to do this because nobody else does!" and my money is not on the latter outcome.
      - amanaplanacanal 1y ago
        It's part of preserving evidence for an ongoing lawsuit. Unless other companies are party to the same suit, why would they have to?
    - johnQdeveloper 1y ago
consumer451 1y ago
All GPT integrations I’ve implemented have been via Azure’s service, due to Microsoft’s contractual obligation for them not to train on my data. As far as I understand it, this ruling does not apply to Microsoft, does it?
- Descon 1y ago
  I think when you spin up open AI in azure, that instance is yours, so I don't believe that would be subject to this order.
  - tbrownaw 1y ago
    The plans scale down far enough that they can't possibly cover the cost of a private model-loaded-to-vram instance at the low end.
ukuina 1y ago
Aren't most enterprise customers using AzureOpenAI?
ivape 1y ago
Going to drop a PG tweet: https://x.com/paulg/status/1913338841068404903 https://x.com/paulg/status/1913338841068404903 "It's a very exciting time in tech right now. If you're a first-rate programmer, there are a huge number of other places you can go work rather than at the company building the infrastructure of the police state." --- So, courts order the preservation of AI logs, and government orders the building of a massive database. You do the math. This is such an annoying time to be alive in America, to say the least. PG needs to start blogging again about what's going on now days. We might be entering the digital version of the 60s, if we're lucky. Get local, get private, get secure, fight back.
bigfudge 1y ago
Will this apply to Azure OpenAI model APIs too?
merksittich 1y ago
Interesting detail from the court order [0]: When asked by the judge if they could anonymize chat logs instead of deleting them, OpenAI's response effectively dodged the "how" and focused on "privacy laws mandate deletion." This implicitly admits they don't have a reliable method to sufficiently anonymize data to satisfy those privacy concerns. This raises serious questions about the supposed "anonymization" of chat data used for training their new models, i.e. when users leave the "improve model for all users" toggle enabled in the settings (which is the default even for paying users). So, indeed, very bad for the current business model which appears to rely on present users (voluntarily) "feeding the machine" to improve it. [0] https://cdn.arstechnica.net/wp-content/uploads/2025/06/NYT-v-OpenAI-Preservation-Order-5-13-25.pdf https://cdn.arstechnica.net/wp-content/uploads/2025/06/NYT-v...
- Kon-Peki 1y ago
  Thank you for the link to the actual text! So, the NYT asked for this back in January and the court said no, but asked OpenAI if there was a way to accomplish the preservation goal in a privacy-preserving manner. OpenAI refused to engage for 5 f’ing months. The court said “fine, the NYT gets what they originally asked for”. Nice job guys.
  - noworriesnate 1y ago
    Nice find! Maybe this is a ploy by OpenAI to use API requests for training while blaming the courts?
- blackqueeriroh 1y ago
  That’s not an implicit admission, it’s refusing to argue something they don’t want to do.
jameshart 1y ago
Thinking about the value of the dataset of Enron’s emails that was disclosed during their trials, imagine the value and cost to humanity of all OpenAI’s api logs even for a few months being entered into court record..

by efskap 1y ago

Kokouane 1y ago

If you were working with code that was proprietary, you probably shouldn't of been using cloud hosted LLMs anyways, but this would seem to seal the deal.

larrymcp 1y ago
I think you probably mean "shouldn't have". There is no "shouldn't of".
- DecentShoes 1y ago
  Who cares?
- knicholes 1y ago
  I care.
- rimunroe 1y ago
  Which gives you an opening for the excellent double contraction “shouldn’t’ve”
  - b112 1y ago
    The letter H deserves better.
    - worthless-trash 1y ago
      I think we gave it too much leeway in the word sugar.
  - mananaysiempre 1y ago
    The funniest part is that in that contraction the first apostrophe does denote the elision of a vowel, but the second one doesn’t, the vowel is still there! So you end up with something like [nʔəv], much like as if you had—hold the rotten vegetables, please—“shouldn’t of” followed by a vowel. Really, it’s funny watching from the outside and waiting for English to finally stop holding it in and get itself some sort of spelling reform to meaningfully move in a phonetic direction. My amateur impression, though, is that mandatory secondary education has made “correct” spelling such a strong social marker that everybody (not just English-speaking countries) is essentially stuck with whatever they have at the moment. In which case, my condolences to English speakers, your history really did work out in an unfortunate way.
    - roywiggins 1y ago
      We had a spelling reform or two already, they were unfortunately stupid, eg doubt has never had the b pronounced in English. https://en.m.wiktionary.org/wiki/doubt https://en.m.wiktionary.org/wiki/doubt That said, phonetic spelling reform would of course privilege the phonemes as spoken by whoever happens to be most powerful or prestigious at the time (after all, the only way it could possibly stick is if it's pushed by the sufficiently powerful), and would itself fall out of date eventually anyway.
    - jdbernard 1y ago
      > but the second one doesn’t, the vowel is still there! Isn't the "a" in "have" elided along with the "h?" Shouldn't've Should not have What am I missing?
      - jack09268 1y ago
        Even though the vowel "a" is dropped from the spelling, if you actually say it out loud, you do pronounce a vowel sound when you get to that spot in the word, something like "shouldn'tuv", whereas the "o" in "not" is dropped from both the spelling and the pronounciation.
        SAI_Peregrinus 1y ago
  - theoreticalmal 1y ago
    My favorite variation of this is “oughtn’t to’ve”
- amanaplanacanal 1y ago
  That used to be the case, but "shouldn't of" is definitely becoming more popular, even if it seems wrong. Languages change before our eyes :)
YetAnotherNick 1y ago
Why not? Assuming you believe you can use any cloud for backup or Github for code storage.
- solaire_oa 1y ago
  IIUC one reason is that prompts and other data sent to 3rd party LLM hosts have the chance to be funneled to 4th party RLHF platforms, e.g. Sagemaker, Mechanical Turks, etc. So a random gig worker could be reading a .env file the intern uploaded.
  - YetAnotherNick 1y ago
    What do you mean by chance? It's clear that if users have not opted out from training the models, it would be used. If they have opted out, it wont be used. And most of the users are in first bucket. Just because training on data is opt out doesn't mean business can't trust it. Not the best for user's privacy though.
gpm 1y ago
I think it's fair to question how proprietary your data is. Like there's the algorithm by which a hedge fund is doing algorithmic trading, they'd be insane to take the risk. Then there's the code for a video game, it's proprietary, but competitors don't benefit substantially from an illicit copy. You ship the compiled artifacts to everyone, so the logic isn't that secret. Copies of the similar source code have linked before with no significant effects.
- FuckButtons 1y ago
  AFAIK, the actual trading algorithms themselves aren’t usually that far from what you can find in a textbook, their efficacy is mostly dictated by market conditions and the performance characteristics of the implementation / system as a whole.
  - short_sells_poo 1y ago
    This very much "depends". Many algo strategies are indeed programmatically simple (e.g. use some sort of moving average), but the parametrization and how it's used is the secret sauce and you don't want that information to leak. They might be tuned to exploit a certain market behavior, and you want to keep this secret since other people targeting this same behavior will make your edge go away. The edge can be something purely statistical or it can be a specific timing window that you found, etc. It's a bit like saying that a Formula 1 engine is not that far from what you'd find in a textbook. While it's true that it shares a lot of properties with a generic ICE, the edge comes from a lot of proprietary research that teams treat as secret and definitely don't want competitors to find out.
- short_sells_poo 1y ago
  Most (all?) hedge funds that use AI models explicitly run in-house. People do use commercial LLMs, but in cases where the LLMs are not run in-house, it's against the company policy to upload any proprietary information (and generally this is logged and policed). A lot of the use is fairly mundane and basically replaces junior analysts. E.g. it's digesting and summarizing the insane amounts of research that is produced. I could ask an intern to summarize the analysis on platinum prices over the last week, and it'll take them a day. Alternatively, I can feed in all the analysis that banks produce to an LLM and have it done immediately. The data fed in is not a trade secret really, and neither is the output. What I do with the results is where the interesting things happen.

neilv 1y ago

Some established businesses will need to review their contracts, regulations, and risk tolerance. And wrapper-around-ChatGPT startups should double-check their privacy policies, that all the "you have no privacy" language is in place.

Wowfunhappy 1y ago
> And wrapper-around-ChatGPT startups should double-check their privacy policies, that all the "you have no privacy" language is in place. If a court orders you to preserve user data, could you be held liable for preserving user data? Regardless of your privacy policy.
- woliveirajr 1y ago
  Yes. If your agreement with the end user says that you won't collect and store data, you're responsible for it. If you can't provide it (even if due to a court order), you have to adjust your contract. Your users aren't obligated to know that you're using open ai or other provider.
- bilbo0s 1y ago
  No. It’s a legal court order. This, however, is horrible for AI regardless of whether or not you can sue.
  - dcow 1y ago
    In the US you absolutely can challenge everything up and including the constitutionality of court orders. You may be swiftly dismissed if nobody thinks you have a valid case, but you can try.
    - deleted 1y ago
      [deleted]
- gpm 1y ago
  I don't think the suit would be against you preserving it, it would be against you falsely representing that you aren't preserving it. A court ordering you to stop selling pigeons doesn't mean you can keep your store for pigeons open and pocket the money without delivering pigeons.

dinobones 1y ago

How? This is retention for legal risk, not for training purposes. They can still have legal contracts with other companies, that stipulate that they don't train on any of their data.

CryptoBanker 1y ago
Right, because companies always follow the letter of their contracts.
Take8435 1y ago
...Data that is kept can be exfiltrated.
- fn-mote 1y ago
  Cannot emphasize this enough. If your psychologist’s records can be held for ransom, surely your ChatGPT queries will end up on the internet someday. Do search engine companies have this requirement as well? I remember back in the old days deanonymizing “anonymous” query logs was interesting. I can’t imagine there’s any secrecy left today.
  - SchemaLoad 1y ago
    I recently had a high school assignment document get posted on a bunch of sites that sell homework help. As far as I know that document was only ever submitted directly to the assignment upload page. So somewhere along the line, I suspect on the plagiarism checker service, there was a hack and then 10 years later some random school assignment with my name on it is all over the place.
    - genewitch 1y ago
      How did you find out?
paxys 1y ago
Your employees' seemingly private ChatGPT logs being aired in public during discovery for a random court case you aren't even involved in is absolutely a business risk.

m3kw9 1y ago

Not when people have nowhere else to go, pretty much you cannot escape it, it’s too convenient to not use now. You think no other AI chat providers doesn’t need to do this?

johnQdeveloper 1y ago

> This seems very bad for their business. Well, it is gonna be all _AI Companies_ very soon so unless everyone switches to local models which don't really have the same degree of profitability as a SaaS, its probably not going to kill a company to have less user privacy because tbh people are used to not having privacy these days on the internet. It certainly will kill off the few companies/people trusting them with closed source code or security related stuff but you really should not outsource that anywhere.

csomar 1y ago
Did an American court just destroy all American AI companies in favor of open weight Chinese models?
- thot_experiment 1y ago
  afaik only OpenAI is enjoined in this
  - csomar 1y ago
    Sure. But this means the rest of the AI companies are exposed to such risk; and there aren't that many of them (grok/gemini/anthropic).
  - baby_souffle 1y ago
    > afaik only OpenAI is enjoined in this For now. This is going to devolve into either "openAI has to do this, so you do too" or "we shouldn't have to do this because nobody else does!" and my money is not on the latter outcome.
    - amanaplanacanal 1y ago
      It's part of preserving evidence for an ongoing lawsuit. Unless other companies are party to the same suit, why would they have to?
  - johnQdeveloper 1y ago

consumer451 1y ago

All GPT integrations I’ve implemented have been via Azure’s service, due to Microsoft’s contractual obligation for them not to train on my data. As far as I understand it, this ruling does not apply to Microsoft, does it?

Descon 1y ago
I think when you spin up open AI in azure, that instance is yours, so I don't believe that would be subject to this order.
- tbrownaw 1y ago
  The plans scale down far enough that they can't possibly cover the cost of a private model-loaded-to-vram instance at the low end.

ukuina 1y ago

Aren't most enterprise customers using AzureOpenAI?

ivape 1y ago

Going to drop a PG tweet: https://x.com/paulg/status/1913338841068404903 https://x.com/paulg/status/1913338841068404903 "It's a very exciting time in tech right now. If you're a first-rate programmer, there are a huge number of other places you can go work rather than at the company building the infrastructure of the police state." --- So, courts order the preservation of AI logs, and government orders the building of a massive database. You do the math. This is such an annoying time to be alive in America, to say the least. PG needs to start blogging again about what's going on now days. We might be entering the digital version of the 60s, if we're lucky. Get local, get private, get secure, fight back.

bigfudge 1y ago

Will this apply to Azure OpenAI model APIs too?

merksittich 1y ago

Interesting detail from the court order [0]: When asked by the judge if they could anonymize chat logs instead of deleting them, OpenAI's response effectively dodged the "how" and focused on "privacy laws mandate deletion." This implicitly admits they don't have a reliable method to sufficiently anonymize data to satisfy those privacy concerns. This raises serious questions about the supposed "anonymization" of chat data used for training their new models, i.e. when users leave the "improve model for all users" toggle enabled in the settings (which is the default even for paying users). So, indeed, very bad for the current business model which appears to rely on present users (voluntarily) "feeding the machine" to improve it. [0] https://cdn.arstechnica.net/wp-content/uploads/2025/06/NYT-v-OpenAI-Preservation-Order-5-13-25.pdf https://cdn.arstechnica.net/wp-content/uploads/2025/06/NYT-v...

Kon-Peki 1y ago
Thank you for the link to the actual text! So, the NYT asked for this back in January and the court said no, but asked OpenAI if there was a way to accomplish the preservation goal in a privacy-preserving manner. OpenAI refused to engage for 5 f’ing months. The court said “fine, the NYT gets what they originally asked for”. Nice job guys.
- noworriesnate 1y ago
  Nice find! Maybe this is a ploy by OpenAI to use API requests for training while blaming the courts?
blackqueeriroh 1y ago
That’s not an implicit admission, it’s refusing to argue something they don’t want to do.

jameshart 1y ago

Thinking about the value of the dataset of Enron’s emails that was disclosed during their trials, imagine the value and cost to humanity of all OpenAI’s api logs even for a few months being entered into court record..