4 ms·

I have very little confidence that OpenAI has the technical expertise or company culture required to deal with APT (advanced persistent threats).

by dekhn 2y ago

ActionHank 2y ago
Also, the article doesn't state it outright, but the implication is that they only stopped bad actors after they had already been using their services, with no indication of how long they were using them for.
baxtr 2y ago
I recommend anyone interested in the subject to study Meta’s Adversial Threat Reports. They started their counter operations a while back and regularly draft extensive reports [1]. Reading through those reports it becomes obvious how much effort this requires. I’m not sure how much effort OpenAI can put into this at all. [1] https://transparency.meta.com/de-de/metasecurity/threat-reporting/ https://transparency.meta.com/de-de/metasecurity/threat-repo...
- lolinder 2y ago
  This is a slight tangent, but I would love to see a sentiment analysis graph of HN perceptions of Meta/Facebook over time for the past two years or so. Subjectively, my sense is that releasing Llama 2/3 gained them a lot of good will in this community and attitudes have generally become much more positive towards them as OpenAI's star has fallen. In terms of developer goodwill, they may be the real winners of this whole gold rush.
  - baxtr 2y ago
    I had a similar thought recently. They really do a good job on many fronts right now. Additionally the whole TikTok debate will help them a lot too.
  - ben_jones 2y ago
    Facebook, I mean Meta, has benefitted from the enshittification of the tech industry. Next to TikTok they’re better moralistically. Next to Sam Altman, I mean OpenAI, they’re better moralistically. Next to other companies doing layoffs they’re better moralistically (biggest exit packages?)
  - brokenmachine 2y ago
    I'm not a fan of them using our data to train their AI, with very obscure opt-out procedures. Not unexpected but just what you'd expect. But I haven't used Facebook for about a decade, so ¯\_(ツ)_/¯
summarity 2y ago
Looking at open roles, it seems like they're fleshing out their security research program: https://openai.com/careers/research-program-manager-security/ https://openai.com/careers/research-program-manager-security...
rozap 2y ago
I think if the mossad is included in your threat model, you may as well just pack up and go home.
- rurp 2y ago
  Maybe in some cases but certainly not for systemically important platforms, as OpenAI aspires to be. Google and Microsoft don't just throw up their hands at the threat of nation-state actors; doing so would be hugely irresponsible.
  - rozap 2y ago
    All technologies live on some scale of exploitability. Mobile phones, for example, are somewhere on that spectrum, and google and apple try their best to secure them. But there are still state actors that have really sophisticated attacks that worm their way into devices. To be clear, I think the security teams at google and apple shouldn't throw their hands up and give up, but I think you need to be honest with yourself when building capability, to accept that it'll get misused. If the misuse doesn't outweigh the benefit to humanity, then it's pretty easy to sleep at night. If it does, then I think you have to wrestle with some awkward questions. On the scale of exploitability, I think some of the LLMs we have now are pretty high up there, beyond mobile phones - harder to secure and with bigger consequences when misused. The intelligence agencies of pariah states like Russia, Iran and Israel will absolutely use this stuff to sway public opinion and get large numbers of people killed. I think employees of OpenAI need to be honest with themselves about this, rather than deluding themselves and saying "oh we'll just do security/trust/safety" better than the a nation state. I think there are a number of ways out of this which are consistent. You could convince yourself that AI's benefit to humanity as a whole over time will outweigh these issues now [citation needed], or you could take a nihilistic approach and say that you just wanna print stacks of money and go relax on the beach. Those are at least arguments you can make. But I think it's intellectually dishonest to say "Bad stuff won't happen with our tech because our team will out compete the mossad".

I have very little confidence that OpenAI has the technical expertise or company culture required to deal with APT (advanced persistent threats).

by dekhn 2y ago

I have very little confidence that OpenAI has the technical expertise or company culture required to deal with APT (advanced persistent threats).

ActionHank 2y ago

Also, the article doesn't state it outright, but the implication is that they only stopped bad actors after they had already been using their services, with no indication of how long they were using them for.

baxtr 2y ago

I recommend anyone interested in the subject to study Meta’s Adversial Threat Reports. They started their counter operations a while back and regularly draft extensive reports [1]. Reading through those reports it becomes obvious how much effort this requires. I’m not sure how much effort OpenAI can put into this at all. [1] https://transparency.meta.com/de-de/metasecurity/threat-reporting/ https://transparency.meta.com/de-de/metasecurity/threat-repo...

lolinder 2y ago
This is a slight tangent, but I would love to see a sentiment analysis graph of HN perceptions of Meta/Facebook over time for the past two years or so. Subjectively, my sense is that releasing Llama 2/3 gained them a lot of good will in this community and attitudes have generally become much more positive towards them as OpenAI's star has fallen. In terms of developer goodwill, they may be the real winners of this whole gold rush.
- baxtr 2y ago
  I had a similar thought recently. They really do a good job on many fronts right now. Additionally the whole TikTok debate will help them a lot too.
- ben_jones 2y ago
  Facebook, I mean Meta, has benefitted from the enshittification of the tech industry. Next to TikTok they’re better moralistically. Next to Sam Altman, I mean OpenAI, they’re better moralistically. Next to other companies doing layoffs they’re better moralistically (biggest exit packages?)
- brokenmachine 2y ago
  I'm not a fan of them using our data to train their AI, with very obscure opt-out procedures. Not unexpected but just what you'd expect. But I haven't used Facebook for about a decade, so ¯\_(ツ)_/¯

summarity 2y ago

Looking at open roles, it seems like they're fleshing out their security research program: https://openai.com/careers/research-program-manager-security/ https://openai.com/careers/research-program-manager-security...

rozap 2y ago

I think if the mossad is included in your threat model, you may as well just pack up and go home.

rurp 2y ago
Maybe in some cases but certainly not for systemically important platforms, as OpenAI aspires to be. Google and Microsoft don't just throw up their hands at the threat of nation-state actors; doing so would be hugely irresponsible.
- rozap 2y ago
  All technologies live on some scale of exploitability. Mobile phones, for example, are somewhere on that spectrum, and google and apple try their best to secure them. But there are still state actors that have really sophisticated attacks that worm their way into devices. To be clear, I think the security teams at google and apple shouldn't throw their hands up and give up, but I think you need to be honest with yourself when building capability, to accept that it'll get misused. If the misuse doesn't outweigh the benefit to humanity, then it's pretty easy to sleep at night. If it does, then I think you have to wrestle with some awkward questions. On the scale of exploitability, I think some of the LLMs we have now are pretty high up there, beyond mobile phones - harder to secure and with bigger consequences when misused. The intelligence agencies of pariah states like Russia, Iran and Israel will absolutely use this stuff to sway public opinion and get large numbers of people killed. I think employees of OpenAI need to be honest with themselves about this, rather than deluding themselves and saying "oh we'll just do security/trust/safety" better than the a nation state. I think there are a number of ways out of this which are consistent. You could convince yourself that AI's benefit to humanity as a whole over time will outweigh these issues now [citation needed], or you could take a nihilistic approach and say that you just wanna print stacks of money and go relax on the beach. Those are at least arguments you can make. But I think it's intellectually dishonest to say "Bad stuff won't happen with our tech because our team will out compete the mossad".