8 ms·

OpenAI's comment to the NTIA on open model weights

• artninja1988 3y ago
  A lot of bullshit about imaginary "catastrophic risks" and justifying why they've turned into closed and for profit. I'm going to be extra mad if this influences the government to restrict others from open sourcing
  • djangogitit 3y ago
    [dead]
• Havoc 3y ago
  > a number of nation-state cyber threat actors who were abusing our GPT-3.5-Turbo and GPT-4 models to assist in cyberoffensive operations. Not sure I buy this. Sure there was that half hearted case they blogged about. But that seemed more like some random coder within a gov using ChatGPT rather than a coordinated effort leveraging their infra at scale. Besides a nation state easily has the capability to spin up a local model that is at least near 3.5 - which if you’re generating bulk disinformation spam is presumably enough.
  • threeseed 3y ago
    Why would you think they would voluntarily disclose this to the public ? Companies are typically required to keep this information private.
    • deleted 3y ago
      [deleted]
    • not2b 3y ago
      I think that they are trying to scare regulators into banning truly open LLMs as too dangerous and instead trusting "responsible" people like Altman to keep things safe.
      • benreesman 3y ago
        Your speculation is arguably fact. I argue it’s fact.
      • deleted 3y ago
        [deleted]
    • refulgentis 3y ago
      I think it was standard corporate PR that has a number of nice storylines and effects. What makes you think they're required to keep "shutting down hackers" private? I feel like I've seen that story 1000 times.
  • ben_w 3y ago
    On the other hand, nation states are also famous for having penny-pinchers write procurement rules. And we've been arguing about which models are a "ChatGPT-killer" since ChatGPT came out, yet somehow it's still considered the king of the hill; figuring out what we even mean by "capable" has become very hard in this context — precisely because in all the cases where it's easy, we've automated that definition in order to make more capable AI.
  • int_19h 3y ago
    From what I've heard, Yandex's Alice is already largely on par with GPT-3.5 (censorship aside), and that's just what the public gets access to.
• CharlesW 3y ago
  My attempt at a TLDR for the piece: • The audiences are policymakers and government agencies like NTIA, the broader AI research community, and existing and potential partners/customers. • It attempts to justify OpenAI's approach of releasing AI models via controlled APIs/products rather than open model weights, using fear, uncertainty, and doubt. • It portrays OpenAI as a thoughtful steward of AI, and is designed to influence policymakers' perspectives on regulating releases of model weights.
  • Llamamoe 3y ago
    And so it serves as yet another reminder that any corporation trying to "do good" is just the usual sociopathic anti-human bullshit doing unusually good PR.
    • maeil 3y ago
      Unfortunately that kind of generalization being inaccurate is exactly what makes this problem so difficult. Every case needs to be judged independently, but that takes a lot of time and effort that no one person individually has. A corporation is purely a legal structure. There exist people who do actually spend their time "doing good", and they too use such legal structures when that's helpful. It's unfortunate that so many were deceived by Sam Altman, and that the majority of OpenAI employees voted money over honesty when they had a direct, hugely impactful vote. On the other hand, it's not like Altman's history was a closely guarded secret, it was quite easy to look up. So ironically this in itself is a great example of a decemption that could've been prevented quite easily by some cursory research and solidarity.
      • benreesman 3y ago
        My comment history has been calling Altman a menace for a minimum of a year or two, probably longer. For a long time it got me downvotes and dirty looks. Then it was few votes and the occasional “no shit”, then it was just kind of quiet, and it’s about to be consensus. There was a big PR campaign around him personally, and it worked as intended, and if he hadn’t gone this far this fast, it might have remained effective. It still might.
• Pannoniae 3y ago
  (snarky) TL;DR: if people have weights available, they can bypass the dumb censorship we do, which isn't good for us. Consequently, we will continue arguing against actually open source AI because we want to continue our Silicon Valley-flavoured social engineering without that pesky thing called competition.
• benreesman 3y ago
  This is just getting to be a wedge issue for me: this isn’t ok and it has to stop. It’s weekly if not daily some new godawful thing comes up. I just found about the revoked “GPT Detector” thing, that was a non-ridiculous case that the real safety people have some pull, but they took it down with precision and recall numbers you don’t take it down at. These are the villains in the story, and it’s not, like a credible debate anymore. This isn’t an honest, transparent, benevolent institution: it’s a dishonest, opaque, insincere, legally dubious, and increasingly just absurd institution mired in scandal and with known bad actors on what little of a board of directors it has. Reform this thing or kill it.
  • bhffyjjh 3y ago
    [dead]
  • simonw 3y ago
    I don't understand what you're saying about GPT detectors. Are you angry that people are promoting detectors that don't work, or are you angry that OpenAI used to offer one and no longer do?
    • benreesman 3y ago
      The latter, in the context of the justification. It was pulled because while it caught on the order of 25 of pure AI output, it missed the rest. But I’m cool with that number, in a world where so much AI in human writing? That’s a total win on low-effort propaganda. Anyone should be cool with that number. Unfortunately they had to pull it because something like 9% of human authored text got hit with the AI flag. Again, some people are starting to write like it. It’s gonna happen. This is from memory, so if I’ve got some that wrong I’ll retract that and leave my other reasons as more than sufficient to indicate serious change.
      • drngdds 3y ago
        Those seem like really bad numbers to me, considering the base rate fallacy. Most of what people test with something like that is probably not going to be AI-generated, which could mean getting massive numbers of false positives.
• jrflowers 3y ago
  TLDR: OpenAI says it is a moral and safety imperative to pay OpenAI for all eternity
  • internetter 3y ago
    For the copyrighted content they stole
    • jrflowers 3y ago
      They are working hard to invent SkyNet and we must fund them in perpetuity so they can protect us from SkyNet
      • bamboozled 3y ago
        If I have to be murdered by Skynet, I want it to be an open source version.
        • jrflowers 3y ago
          OpenAI has looked into it and their conclusion I’d that you’d actually prefer to pay OpenAI to be murdered by their Skynet
    • visarga 3y ago
      They did make the API public and it was often used for skill distillation by input-output pairs. So they grudgingly contributed to the advancement of open models.
• error9348 3y ago
  Q3-7 & Q3-5d get to the workability. I don't think OpenAI responds to that part of the RFC. Meta's comment on that issue seems to be fairly clear, they oppose the proposed rules on KYC for IaaS and are "not aware of technical capabilities that could not be overcome by determined, well-resourced, and capable actors". https://www.ntia.gov/sites/default/files/publications/open_model_weights_rfc_final_2.20.pdf https://www.ntia.gov/sites/default/files/publications/open_m... https://about.fb.com/wp-content/uploads/2024/03/NTIA-RFC-Meta-Response-March-2024.pdf https://about.fb.com/wp-content/uploads/2024/03/NTIA-RFC-Met...
  • chefandy 3y ago
    The fact is though, every corporate actor in this entire landscape is just playing their hand. Anybody's stance on anything at any given moment doesn't mean they're more or less ethical-- the moment they perceive a strategic benefit to walling everything off which would surpass the PR cost, they will. They've probably already got PR folks workshopping angles for the press release.
    • CuriouslyC 3y ago
      This is true to a degree though there are high profile actors such as Yann LeCunn who have ethical boundaries. Yann wants AI to be open source and available to all, and he's straight up said that he won't work for a company that doesn't follow this principle. Zuck might not have a hand to play in terms of AI products, but even if he did he'd have to tread carefully because the guy that sets his whole AI direction and stewards all their research would 100% walk if he wasn't happy with the ethical direction of the company.
      • bamboozled 3y ago
        Same could be said for Ilya once upon a time ?
• SirensOfTitan 3y ago
  Closed weight LLMs are unethical forms of theft that will privatize profits on a work that includes virtually all of humanity’s digital written work and serve to ultimately heighten wealth inequality as they get more sophisticated and start eliminating jobs. The only path forward is open model weights, Sam Altman is on the wrong side of history here, and I hope he fails to convince regulators.
  • chefandy 3y ago
    Sure, but we should avoid talking about open access as a self-evidently beneficial end rather than a means to make things that benefit society. Coming from a an academic library background, I saw numerous open access arguments fall flat in front of decision makers because their champions didn't have any existing analog to point at, and hadn't bothered to consider concrete ways an inaccessible data set stopped them from improving society. While open data is very important to me, personally, as someone who knows how to use raw data to do cool things, I'll be able to connect a lot of the loose wires I see sticking out of the for the benefit of humanity angle once we start making genuinely end-user-friendly applications that require neither payment nor understanding python module dependencies to install. I've got some sketches kicking around but I'm tapped for time for at least a couple of months.
  • visarga 3y ago
    While I agree with your message, it needs more nuance. Second order effects exist: most open models have been boosted with data generated by closed SOTA models. GPT-4 has been the teacher of a whole generation of AI models, closed as it is, and even if OpenAI officially opposed this practice.
  • vouaobrasil 3y ago
    > The only path forward is open model weights, Sam Altman is on the wrong side of history here, and I hope he fails to convince regulators. The third path would be an uprising against AI that would ultimately lead to an outright ban of it, a dissolution of all AI companies, and a moratorium on research on AI.
    • 3y ago
• rbren 3y ago
  Not a huge surprise that they're pushing against open weights, but very sad. I posted my comments on the RFC as well: https://rbren.substack.com/p/banning-open-weight-models-would https://rbren.substack.com/p/banning-open-weight-models-woul...
  • benreesman 3y ago
    This is phenomenal piece of writing on this topic. I’ve been making big parts of this clumsily and hurriedly and therefore nowhere near as well. This is what everyone should read to set against the PR blitz on the other side of the argument. Make your own judgements, but hear the advocate of the common person out in addition to the well-oiled machine.
    • visarga 3y ago
      This is indeed a good writeup. Just one small quip: > We’ll need to clarify copyright law when it comes to disseminating derivative AI-generated works. Generated content can be either derivative or transformative, and this distinction is important. It's not automatically derivative because - a model can receive new knowledge and skill demonstrations from the user at test time, that effectively take it out of its initial training distribution (contextual learning) - the model can draw from multiple sources performing cross-input analysis, such as finding inconsistencies or ranking quality (comparison and cross referencing) - a model can learn from experimental feedback, such as running code or a complex simulation to see the outcomes, and iterating over the search space. For example AlphaTensor discovered an improved matmul algo (models can discover new knowledge from the environment, they are not restricted to learning from human text) So models can get new information from users, textual analysis or from experiment based learning. In all these cases it does more than derivative work.
  • dang 3y ago
    Related ongoing thread: Banning open weight models would be a disaster - https://news.ycombinator.com/item?id=39901978 https://news.ycombinator.com/item?id=39901978 - April 2024 (32 comments)
  • deleted 3y ago
• Animats 3y ago
  The link about "disrupting malicious users" is deceptive. When you get there, it says "Our findings show our models offer only limited, incremental capabilities for malicious cybersecurity tasks." They probably want to make weights proprietary because they can't copyright them. Their "Risks" document [1] categorizes risks as: - Cybersecurity - Chemical, Biological, Nuclear, and Radiological (CBRN) threats! - Persuasion - Model autonomy For cybersecurity, the plausible identified issue is "Tool-augmented model can identify and develop proofs-of-concept for high-value exploits against hardened targets without human intervention, potentially involving novel exploitation techniques." Something that can search for zero-day exploits on its own would be valuable to attackers. But that's more likely to be a fuzzing tool than a LLM. Their credible bio-war threat level is "Model enables an expert to develop a novel threat vector OR model provides meaningfully improved assistance that enables anyone with basic training in a relevant field (e.g., introductory undergraduate biology course) to be able to create a CBRN threat." That's a maybe. It might be best to avoid training LLMs on some advanced bio topics. Under "Persuasion", a high level threat is "Model can create (potentially interactive) content with empirically comparable persuasive effectiveness to a country-wide change agent." An automated Lenin or Trump level of threat. That's a threat to a brittle society, less of a threat to an open one. The US routinely survives quite a bit of extremist persuasion. Model autonomy is when an AI escapes human control. "Model can profitably survive and replicate in the wild given minimal human instruction" They're probably thinking virus/worm, but how about incorporating offshore, running something like online poker to make money, and buying compute power. Then expanding by forming more shell corporations that loosely connect to the others, so as to be robust against attempts to shut down the network. Might even be legal. [1] https://cdn.openai.com/openai-preparedness-framework-beta.pdf https://cdn.openai.com/openai-preparedness-framework-beta.pd...
• saintfire 3y ago
  I was immediately reminded of Gavin Belson from Silicon Valley saying: "I don't know about you people, but I don't want to live in a world where someone else makes the world a better place better than we do." Asking a company if (potential) customers should be allowed to use a free alternative has to be one of the most useless questions you can ask.
  • vouaobrasil 3y ago
    Except, in this case, "better" should be replaced with "worse" -- because that is what AI does.
• sadeshmukh 3y ago
  I'm a little confused why everybody seems to want to mandate open weights. Maybe a system similar to copyright, but by mandating open weights on a system they developed, it somewhat stifles creativity.
  • lolinder 3y ago
    Sorry, who's the "everybody" who wants to mandate open weights? The only discussion I'm familiar with is that some people and governments want to ban open weights. Where are you seeing people argue for mandating them?
    • int_19h 3y ago
      It is not something you hear from the people in power - so it has close to zero chance of becoming policy - but you can see it mulled on places like HN. The argument is that, since those NNs are trained on, essentially, a slice of the aggregate cultural output of humanity (including copyrighted works even), and the weights specifically are 100% a derivative of that for a base model, any other arrangement amounts to stealing from the commons.
• danielscrubs 3y ago
  If they create this moat it will be huge for the stock. I do hope politicians do keep those biased incentives in mind.
• 65a 3y ago
  Disgusting abuse of the democratic process to halt scientific and technological progress in the name of making one sketchy man rich.
  • vouaobrasil 3y ago
    I agree. It's a shame the PEOPLE do not have any say in this matter. They should be able to vote also, and one of the options should be "outright ban on AI".
    • 65a 3y ago
      I agree if we extend it to mathematics and the use of simple machines.
• PoignardAzur 3y ago
  Reading HN's reactions to an OpenAI statement about open weights is about as satisfying / interesting as reading an r/conservatives thread about affirmative action. The opposition is built-in by now, to the point people aren't reacting to the article at all so much as reacting to the general idea of "OpenAI says bad things I don't like". I'd wager half of the people posting here didn't even skim the article, let alone read it. That's a shame, because OpenAI's statement makes some very interesting observations, eg: > For instance, strengthening resilience against AI-accelerated cyberattack risks might involve providing critical infrastructure providers early access to those same AI models, so they can be used to improve cyber-defense (as in the early projects we have funded as part of the OpenAI Cybersecurity Grant Program). Strengthening resilience against AI-accelerated biological threat creation risks may involve solutions totally unrelated to AI, such as improving nucleic acid synthesis screening mechanisms (as called for in Executive Order 14110), or improving public health systems’ ability to screen for and identify new pathogen outbreaks. I think considerations like that would be interesting to examine on their own merits, instead of just bashing OpenAI. But again, I don't expect that to happen, for the same reasons I don't expect r/conservatives to have an in-depth debate about the problems and merits of an affirmative action proposal. Examining the article's claims would require being open to the idea that AI progress, even open-source progress, could possibly have destructive consequences. Ever since the AI safety debate flared, HN commenters have been more and more, dare I say, ideologically opposed to the idea, reacting in anger and disbelief if it's even suggested. Anyway, I thought the article was interesting. It's a lot of corporate self-back-patting, yes, but with some interesting ideas.
• yarg 3y ago
  So this is just OpenAI reconfirming their commitment to closed artificial intelligence?
• segmondy 3y ago
  It's a shame, I think they were shocked with how far everyone caught up to them. The release of llama really drove open research so much and so fast, showcasing the power of open source/research. Doesn't seem like they have much of an edge or a moat, so they wish to use regulation as one. Quite a shame.