5 ms·

Software companies must be leaking code on a massive scale right now.

by gumballindie 4y ago

Bhilai 4y ago
Not only code, I have heard horror stories of employees uploading regulated customer data to Open AI.
- andy_ppp 4y ago
  There’s a ~3000 word prompt limit so while it’s bad whole user datasets are unlikely to have been uploaded… the issue will come when ChatGPT work out how to do live training from prompts and other user interactions, I suppose it will be filtered but nothing is 100%.
  - rst 4y ago
    Doesn't necessarily apply to data being processed with plugins. The "Code Interpreter" plugin lets you upload files up to 100 MB for use by AI-generated Python scripts; see https://www.oneusefulthing.org/p/it-is-starting-to-get-strange https://www.oneusefulthing.org/p/it-is-starting-to-get-stran... for examples (including useful output from vague prompts like "please characterize this dataset", at least if you get lucky).
- jeroenhd 4y ago
  There was a "Show HN" post on here that was planning on using this type of AI to help doctors, basically letting them pipe their patients' medical data into an online AI chat. The way basic privacy and secrecy is ignored in this space is staggering.
  - gumballindie 4y ago
    The whole space is built around it. The reason is simple. These machine learning systems need massive amounts of data to calibrate for accuracy. They are not intelligent to somehow learn on their own. So machine learning companies have figured they can do an aggressive marketing campaign to make people think these are intelligent and uncontrollable to mask their never ending hunger for our data, hoping everyone is ignorant enough not to catch on. What remains to be seen is whether we come back to our senses soon enough. A lot of “experts” working in ai are warning about dangers but what they omit telling is that it’s the humans who own ai that are dangerous.
    - hxugufjfjf 4y ago
      That’s just plain wrong. Any AI expert worth their salt will tell that the humans who own, control and operate the AI is the primary danger, at least the way it works today.
- ChatGTP 4y ago
  Whole city in Japan: https://www.japantimes.co.jp/news/2023/04/20/national/chatgpt-yokosuka-trial/ https://www.japantimes.co.jp/news/2023/04/20/national/chatgp... However, Samukawa offered assurances that Yokosuka intends to use the tool in line with OpenAI’s typical security policy. Ha
throwawaysleep 4y ago
Code and data. I know people putting company data in there.
speedgoose 4y ago
Some even upload all their conversations and confidential internal documents to Microsoft 365!
- belorn 4y ago
  Which works perfectly fine until someone leaks the fact that NSA have full access to those documents, and occasionally will provide help to a few US companies that are seen as important to the nation.
  - datadata 4y ago
    Do you have any example of your claim about providing help?
    - saiya-jin 4y ago
      This has been valid even decades before internet took over the world, I recall quite a few articles listed well known cases some years ago. Sure it wont push for some small startup but big corporations, sales of commercial planes, military equipment contracts and similar stuff for billions and more? CIA/NSA would be failing at its core real mission if they didnt help US interests when they see an opportunity. And with us-based cloud they dont need to hack anything remote, just fill another form and go again in. Makes me think, having strong privacy laws like ie in Swtzerland is a massive win for given country and its citizens in long term.
      - deleted 4y ago
        [deleted]
    - croes 4y ago
      They did in the past why not now? https://en.m.wikipedia.org/wiki/ECHELON#Concerns https://en.m.wikipedia.org/wiki/ECHELON#Concerns
    - shubb 4y ago
      Siemens and some other German majors claimed the NSA stole thier trade secrets related to wind turbines and gave them up some American companies. I think they claimed it was done via wire tap. I'm not sure if this claim was ever validated but it seems wild those fairly boring companies would come out with it over nothing, whereas it is related to of the US services stated mission. US government access to thier companies customers files has been a compliance issue in Europe for a while because it is hard to claim GDPR compliance if your vendor might be required to leak user data outside the judicial system. This is why privacy shield treaty was needed.
vsareto 4y ago
ChatGPT: “That’s not an abstraction, it’s an abomination”
npteljes 4y ago
Always have been. I've been working on healthcare project, they haven't even shared all the repos with us, for data leakage concern, but they were using free web tools to format jsons and things like that. Boggles the mind.

Software companies must be leaking code on a massive scale right now.

Bhilai 4y ago

Not only code, I have heard horror stories of employees uploading regulated customer data to Open AI.

andy_ppp 4y ago
There’s a ~3000 word prompt limit so while it’s bad whole user datasets are unlikely to have been uploaded… the issue will come when ChatGPT work out how to do live training from prompts and other user interactions, I suppose it will be filtered but nothing is 100%.
- rst 4y ago
  Doesn't necessarily apply to data being processed with plugins. The "Code Interpreter" plugin lets you upload files up to 100 MB for use by AI-generated Python scripts; see https://www.oneusefulthing.org/p/it-is-starting-to-get-strange https://www.oneusefulthing.org/p/it-is-starting-to-get-stran... for examples (including useful output from vague prompts like "please characterize this dataset", at least if you get lucky).
jeroenhd 4y ago
There was a "Show HN" post on here that was planning on using this type of AI to help doctors, basically letting them pipe their patients' medical data into an online AI chat. The way basic privacy and secrecy is ignored in this space is staggering.
- gumballindie 4y ago
  The whole space is built around it. The reason is simple. These machine learning systems need massive amounts of data to calibrate for accuracy. They are not intelligent to somehow learn on their own. So machine learning companies have figured they can do an aggressive marketing campaign to make people think these are intelligent and uncontrollable to mask their never ending hunger for our data, hoping everyone is ignorant enough not to catch on. What remains to be seen is whether we come back to our senses soon enough. A lot of “experts” working in ai are warning about dangers but what they omit telling is that it’s the humans who own ai that are dangerous.
  - hxugufjfjf 4y ago
    That’s just plain wrong. Any AI expert worth their salt will tell that the humans who own, control and operate the AI is the primary danger, at least the way it works today.
ChatGTP 4y ago
Whole city in Japan: https://www.japantimes.co.jp/news/2023/04/20/national/chatgpt-yokosuka-trial/ https://www.japantimes.co.jp/news/2023/04/20/national/chatgp... However, Samukawa offered assurances that Yokosuka intends to use the tool in line with OpenAI’s typical security policy. Ha

throwawaysleep 4y ago

Code and data. I know people putting company data in there.

speedgoose 4y ago

Some even upload all their conversations and confidential internal documents to Microsoft 365!

belorn 4y ago
Which works perfectly fine until someone leaks the fact that NSA have full access to those documents, and occasionally will provide help to a few US companies that are seen as important to the nation.
- datadata 4y ago
  Do you have any example of your claim about providing help?
  - saiya-jin 4y ago
    This has been valid even decades before internet took over the world, I recall quite a few articles listed well known cases some years ago. Sure it wont push for some small startup but big corporations, sales of commercial planes, military equipment contracts and similar stuff for billions and more? CIA/NSA would be failing at its core real mission if they didnt help US interests when they see an opportunity. And with us-based cloud they dont need to hack anything remote, just fill another form and go again in. Makes me think, having strong privacy laws like ie in Swtzerland is a massive win for given country and its citizens in long term.
    - deleted 4y ago
      [deleted]
  - croes 4y ago
    They did in the past why not now? https://en.m.wikipedia.org/wiki/ECHELON#Concerns https://en.m.wikipedia.org/wiki/ECHELON#Concerns
  - shubb 4y ago
    Siemens and some other German majors claimed the NSA stole thier trade secrets related to wind turbines and gave them up some American companies. I think they claimed it was done via wire tap. I'm not sure if this claim was ever validated but it seems wild those fairly boring companies would come out with it over nothing, whereas it is related to of the US services stated mission. US government access to thier companies customers files has been a compliance issue in Europe for a while because it is hard to claim GDPR compliance if your vendor might be required to leak user data outside the judicial system. This is why privacy shield treaty was needed.

vsareto 4y ago

ChatGPT: “That’s not an abstraction, it’s an abomination”

npteljes 4y ago

Always have been. I've been working on healthcare project, they haven't even shared all the repos with us, for data leakage concern, but they were using free web tools to format jsons and things like that. Boggles the mind.