13 ms·
We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: - there’s no way they’re manually scrubbing out sensitive
by cuuupid 4y ago
We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen:
- there’s no way they’re manually scrubbing out sensitive data so its bound to spill out from the training data when prompting the model
- OpenAI is openly storing all this data they’re collecting to the extent that they’ve had several leaks now where people can see others’ conversations and data. We are one step away if it hasn’t already happened from an exploit of their systems (that likely weren’t built with security as the top priority as opposed to scale and performance) that could leak a monumental amount of data from users.
In the most innocent case they could leak the personal info of naive users. But largely if Linkedin is any indication, the business world is filled with dopes who genuinely believe the AI is free thinking and better than their employees. For every org that restricts ChatGPT use, there are fifty others that don’t, most of which have at least one of said dopes who are ready to upload confidential data at a moments notice.
Wouldn’t even put it past military personnel putting S/TS information into it at this point. OpenAI should include more brazen warnings against providing this type of data if they want to keep up this facade of “we can’t release it because ethics” because cybersecurity is a much more real liability than a supervised LM turning into terminator.
- deleted 4y ago[deleted]
- Ensorceled 4y agoLet's also not discount that for every "dope" there is at least one "bad actor" who is willing to take the risk to get an edge in their workplace or appease their managers demands. The warnings will only deter the first group.
- comment_ran 4y agoIf your competitor use ChatGPT to compete with you and they're 10x productive than yours, are you still willing to insist? If the productive is 100x, will you?
- brobdingnagians 4y agoIt might be just as likely that ChapGPT will cause a mistake like Knight Capital because no one bothered to thoroughly verify the AI's looks-good-but-deeply-flawed answer, and the two aren't mutually exclusive possibilities.
- mlrtime 4y agoThe Knight meltdown was more of a disfunction of change management and trading system operations than it was of using a decommissioned feature flag. Source: worked there after the meltdown.
- causality0 4y agoRight. I've had ChatGPT completely fail at something as simple as writing a batch file to find and replace text in a text file.
- arcticfox 4y agoSure, but humans do that all the time as well
- remexre 4y agoHumans are a lot better at "I don't know how to do this; hey Alice, can you look this over if you've got a sec and tell me if I'm making a noob mistake"
- bolanyo 4y agoPerhaps the actual phenomenon is that humans are much better at saying "Alice wrote this code, she's pretty good at scripting but she might have made a noob mistake, better check it", or even "I wrote this code.." than they are at saying "ChatGPT wrote this code, but that application is not guaranteed to have correctly identified my problem, but may have just returned something that seems right both to the statistical model and to me, but which is actually deeply flawed, better check it".
- michaelteter 4y agoPossibly I don't know how this all works, but I think if the host of a ChatGPT interface were willing to provide their own API key (and pay), they could then provide a "service" to others (and collect all input). In that case, you wouldn't know to block them until it was too late. Ultimately either you must watch/block all outgoing traffic, or you must train your people so thoroughly that they become suspicious of everything. Sadly, being paranoid is probably the most economical attitude these days if IP and company secrets have any value.
- dragonwriter 4y ago> Possibly I don’t know how this all works, but I think if the host of a ChatGPT interface were willing to provide their own API key (and pay), they could then provide a “service” to others (and collect all input). Well, GP was referring to blocking ChatGPT as a federal contractor. I suspect that as a federal contractor, they are also vetting other people that they share data with, not just blocking ChatGPT as a one-off thing. I mean, generic federal data isn’t as tightly regulated as, say, HIPAA PHI (having spent quite a lot of time working for a place that handles both), but there are externally-imposed rules and consequences, unlike simple internal-proprietary data.
- michaelteter 4y agoBut it really seems like a cat and mouse game. For example, a very determined bad actor could infiltrate some lesser approved government contractor and provide an additional interface/API which would invite such information leaking, and possibly nobody would notice for a long time.
- Jensson 4y agoAnd then they could face death penalty for espionage if they leaked sensitive enough data. You would have to be really stupid to build such a service for government contractors unless you actually are a foreign spy.
- 4y ago
- yathrowaway01 4y agoSo you block internet access for all employees? Cos anything you think is being pasted into ChatGPT is being pasted everywhere, whether its Google, Slack, Chrome Plugins, public Wifi.
- toyg 4y agoOr all the ChatGPT clones that have sprung up and will continue to spring up every other day. It's a stupid and patronizing position, but corporate IT are sadly incentivised to be stupid and patronizing.
- kube-system 4y agoYes, these things are sometimes blocked in higher security workplaces… up to and including the public internet. Honestly airgapped systems are not all that uncommon anywhere that human life is at risk.
- probablynish 4y agoThis really depends on the cost/benefit tradeoff for the entity in question. If using ChatGPT makes you X% more productive (shipping faster / lowers labor costs / etc), but comes with Y% risk of data leakage, is that worth it in expectation or not? I would argue that there definitely exist companies for which it's worth the tradeoff. By the way, OpenAI says they wont use data submitted through its API for model training - https://techcrunch.com/2023/03/01/addressing-criticism-openai-will-no-longer-use-customer-data-to-train-its-models-by-default/ https://techcrunch.com/2023/03/01/addressing-criticism-opena...
- HeavyStorm 4y agoYou've certainly not worked with _real_ sensitive data. The kind that can bankrupt your business. I do and if it could be leaked through ChatGPT I would have it blocked. Risk isn't a single dimension, it's a combination of exposure (chance of happening) and impact (how much will you lose)
- KyeRussell 4y agoMate. You aren’t special. It’s the nature of the profession that most of us are in, that we end up dealing with the “sensitive” data that you’re describing, barring most people working in Big Companies with proper internal controls. Nothing you’ve said negates anything OP said. It’s simply an elaboration wrapped in elitism.
- heyyyouu 4y agoBut that's the API, not the Chat input or Playground. Companies can use Azure OpenAI Services to get around this -- there's data privacy, encryption, SLAs even. The problem is it's very hard to get access to (right now).
- eldaisfish 4y agothe #1 problem with corporations saying things is that many things they say are not regulated or are taken on good faith. What happens with OpenAI are acquired and the rules change? These comments are often entirely worthless.
- nostromo 4y agoDoes blocking ever work? People are smart and usually just work around them.
- mnd999 4y agoThe smart ones don’t paste in all their private data. And yes, if the bypassing the block is combined with disciplinary action, it does work. It’s not worth getting fired over. This is likely what heavily regulated industries like financial services and defense are doing.
- tedunangst 4y agoBlocks are effective reminders of policies.
- acomjean 4y agoI remember someone trying to look up winning lottery numbers at work. The site came up "Blocked: Gambling". It was a little reminder that they're watching our web browsing at work..
- TurkishPoptart 4y agoWell, a firewall rule based on a cloud-populated access control list interrupted traffic. More likely vendor-related than employer-related.
- spprashant 4y agoThose are pre-configured firewall rules. These firewalls can go deep packet inspection and block traffic. It's a fairly standard practice. I wouldn't associate it with overreaching surveillance.
- PeterisP 4y agoIt works in the sense that it does add an extra "reminder" and requires specific intent. I mean, in this scenario all the people already have been informed that they're absolutely not allowed to do things like that, but if someone has forgotten that, or simply is careless and just wants to "try something out" then if it's unblocked they might actually do it, but if they need to work around a restriction, that forces them to acknowledge that there is a restriction and they shouldn't try to work around it even if they can.
- m000 4y agoDoes US intelligence have access to OpenAI data? Private organizations is one thing. But with all the dopes in government positions around the world, OpenAI logs would probably be a treasure trove for intelligence gathering.
- grepfru_it 4y agoThey are just one national security letter away from all US-held data.
- px43 4y agoMicrosoft is well known for piping data to US intelligence as a service. It's almost certainly why they bought Skype, then removed all the end to end encryption.
- speedgoose 4y agoUSA has the patriot act and the cloud act to request the data from any USA company. Like AWS, Microsoft 365, Google…
- SrslyJosh 4y ago> Wouldn’t even put it past military personnel putting S/TS information into it at this point. Hey, they need someone to proofread their War Thunder forum posts to make sure they're using correct spelling and grammar when leaking classified info. ;-) (Ref if you don't get the joke: https://taskandpurpose.com/news/war-thunder-forum-military-technology-leaks/ https://taskandpurpose.com/news/war-thunder-forum-military-t...)
- RF_Savage 4y agoNot only that, but the European theater nuclear forces leaking security arragements and even door PIN-codes for nuclear weapons bunkers via online flash card sites might be a better example. As the leaks ware more inadvertent.
- nomilk 4y ago> there’s no way they’re manually scrubbing out sensitive data I was under the impression OpenAI weren't using questions as training data for future models. I recall Sam Altman saying they delete questions after 1 month, but I can't locate the source for that.
- whitemary 4y agoEven if they do today, they could change their mind tomorrow, or even tonight.
- Schroedingersat 4y agoRight. They also haven't been caught lying about everything else from cuttoff dates to live functionality.
- wkat4242 4y agoEven without training the model, it will still end up in logs. For example you can now see your previous questions in the UI and it'll probably be stored in other places of their backend too. This is still a serious data loss risk.
- capybara_2020 4y agoI am curious, do you block MS Edge? It has a grammar check for all input boxes that sends data to MS servers to check. Similar to what Grammarly does. MS also "helpfully" asks you if you want to use that enhanced grammar check in MS Word(as far as I have seen, might be there in other office products too). I cannot imagine sending all my documents to MS. But I am not sure most users will realize what is happening. All these companies offer helpful services but are hovering up data and no one knows the consequences yet. It feels like ChatGPT is just one symptom of a bigger problem.
- lazyasciiart 4y agoYou can disable the feature entirely in group policies, I imagine organizations with a decent IT org will do so before deploying the update.
- cuuupid 4y agoWe don't but the grammar check is disabled. In general anything cloud-based services are vetted before being allowed. I think MS Edge is getting even worse about this, with the big fucking Bing icon in the corner and making it impossibly hard to get rid of it.
- nukeman 4y agoSome military folks put nuclear weapons storage training materials onto Quizlet, so I don’t doubt for a second people would try to put ChatGPT onto a classified computer system.
- check_muk 4y agoBy using Azure, you can access ChatGPT and GPT, which come with enterprise-grade security and established data agreements, setting them apart from OpenAI. I'm not entirely sure of the technical details, but you can explore this option.
- ozim 4y agoProblem is you will have to heavily advertise it in your org because people will not understand why, where etc. They will go to OpenAI directly and do stuff because "they want it now" and they don't understand why not. Microsoft is already building it into Office 365 with the same enterprise-grade agreements so it might get easier that way.
- tempestn 4y ago> they’ve had several leaks now where people can see others’ conversations and data Do you have a source for this? I know some people have claimed to see others' data, but I haven't seen any evidence that that's what's actually being seen, vs LLM hallucinations. OpenAI claims, and I can't imagine they're lying, that the training data is fixed and ends in 2021, so I don't see how it would be possible for user prompts to be leaking into output, absent a massive and very unlikely bug (compared to the much more likely AI hallucination explanation).
- nfm 4y agoYep: https://openai.com/blog/march-20-chatgpt-outage https://openai.com/blog/march-20-chatgpt-outage Some kind of concurrency bug in a library they were using to retrieve cached data from Redis led to this leak. > We took ChatGPT offline earlier this week due to a bug in an open-source library which allowed some users to see titles from another active user’s chat history. It’s also possible that the first message of a newly-created conversation was visible in someone else’s chat history if both users were active around the same time.
- PUSH_AX 4y agoI think they’re referencing this: https://openai.com/blog/march-20-chatgpt-outage https://openai.com/blog/march-20-chatgpt-outage
- hannob 4y ago> We block ChatGPT, as do most federal contractors. I think it’s a horrible exploit waiting to happen: Do you also block pastebin? Anything else that has a web form? How is ChatGPT special compared to any other service on the Internet where people can paste data in a form? I mean... I see the problem, but I think one needs to realize that it's a far more generic problem that has basically nothing to do with ChatGPT and AI. If people paste confidential data into random webpages that's of course bad. But if you block ChatGPT because you fear that, it means you expect that people might do that. And then your problem is not ChatGPT, but lack of awareness what is confidential data and what to do with it.
- detaro 4y agoBecause "awareness only" has such a great track record when it comes to security-adjacent issues, and totally satisfies auditors/customers/regulators/...?
- hannob 4y agoI don't think awareness only has any reasonable track record and I would always prefer a technical control if there is one. But I have a hard time seeing any alternative here. I don't think the idea that you can give people access to the www and at the same time preventing them from putting things in forms can be done. That's simply not how it works. And if you're blocking access to a few services where they might do that, well, they have a million others, and you're deceiving yourself that you've done something.
- gpderetta 4y ago> Do you also block pastebin? Anything else that has a web form? pastebin and indeed most things that has some sort of public webform is blocked in all the companies I have worked with. It is probably a losing battle though, as it is very hard to block everything without default deny. Paradoxically, maybe GPT could be used to veto websites on first access :)
- TeMPOraL 4y ago
- black_13 4y ago[dead]