7 ms·

Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.

sjwhevvvvvsj 3y ago
I also think it’s ludicrous to the point of hilarity; but it’s also harmful as people who can make laws and big decisions are buying this horse shit.
- RajT88 3y ago
  You do have to appreciate the Machiavellian cleverness of this approach to marketing.
  - sjwhevvvvvsj 3y ago
    Nah, a true Machiavellian would fool smart people too - this has the sophistication of jangling keys in front of an infant. I’m a bit embarrassed for them.
    - danielmarkbruce 3y ago
      They are fooling lots of smart but not technical people. You may not be one of them, but there are many.
      - sjwhevvvvvsj 3y ago
        Fair point.
    - kjkjadksj 3y ago
      Hard to be embarrassed at the player when jingling keys in front of an infant is often literally the game.
danielmarkbruce 3y ago
Are they? Or is this marketing genius in several ways?
- therein 3y ago
  Alienating the developers is always a good idea, yeah. Google does it all the time, works great.
  - danielmarkbruce 3y ago
    How is this alienating developers?
    - solarpunk 3y ago
      how likely are you to start work on a project depending on an ecosystem of wildly overstated capabilities?
      - danielmarkbruce 3y ago
        100% likely. Several projects, right now. GPT-4 is the best model currently available. There are reasons why it's better to control a model and host yourself etc etc, but there are also reasons to use the best model available.
        solarpunk 3y ago
        ... I'm not trying to be rude, but do you think maybe you have bought into the purposely exaggerated marketing?
        tempusalaria 3y ago
        GPT-4 is the best model though… the gap has closed a lot but it’s still the best I despise openai but I can’t really argue with that
        3y ago
deleted 3y ago
[deleted]
valine 3y ago
This will likely be used as evidence to justify regulating open weight models. It doesn’t matter if the models are actually dangerous, the messaging is a means to an end.
- api 3y ago
  Yep, the strategy seems to be to legally require AI to be closed SaaS. Otherwise OpenAI doesn't actually have much of a moat. Chips capable of running local AI models are only going to get cheaper, especially as every chip maker is now going in that direction to chase Nvidia.
  - devsda 3y ago
    Not just a closed SaaS. If governments decide to set whatever 'safeguards' open AI comes up with as the safety baseline for general AI, it increases compliance costs for its competitors(both open and closed).
agnokapathetic 3y ago
They're pretty clear about it though erring on the conservative side. > While none of the above results were statistically significant, we interpret our results to indicate that access to (research-only) GPT-4 may increase experts’ ability to access information about biological threats, particularly for accuracy and completeness of tasks. This access to research-only GPT-4, along with our larger sample size, different scoring rubric, and different task design (e.g., individuals instead of teams, and significantly shorter duration) may also help explain the difference between our conclusions and those of Mouton et al. 2024, who concluded that LLMs do not increase information access at this time.
wg0 3y ago
Absolutely. In reality, it often fails to do Type gymnastics in Typescript let alone virology.
deleted 3y ago
[deleted]
aftbit 3y ago
Well ironically this study shows that GPT-4 isn't actually very good: >However, the obtained effect sizes were not large enough to be statistically significant, and our study highlighted the need for more research around what performance thresholds indicate a meaningful increase in risk. A "mild uplift" in capabilities that isn't statistically significant doesn't really sound like overestimation.
- digging 3y ago
  That's because they're not overestimating their product, they're trying to gauge what risk looks like.
  - ctrw 3y ago
    They are going for regulatory capture because they have no moat and open source Ai models are eating their lunch.
digging 3y ago
It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending humanity once it has the capabilities is completely unsolved. We don't even know if it's solvable, let alone what approach to take. Don't you actually believe it is not essential to practice on weaker AI before we get to that point? Would you walk through a combat zone without thinking about how to protect yourself until after you hear a gunshot? I expect many replies about ChatGPT being "too stupid" to end the world. Please hold those replies, as they completely miss the point. If you consider yourself an intelligent and technical person, and you think it's not worth thinking about existential risks posed by future AI, I would like to know when you think it will be time for AI researchers (not you personally) to start preparing for those risks.
- solarpunk 3y ago
  what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes? if that's the case, how much resources do you think should be dedicated to regulating it? more or less than currently identified existential risks? which entities should be paying for the regulatory controls? what's the proposal here? it's odd because only this one single company that is hedging it's entire existence on "oh boy what if this thing is dangerous some time in the near future" is doing silly stunts like this. why aren't they demanding nvidia start building DRM enabled thermite charges into A100s?
  - digging 3y ago
    > what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes? It certainly could. More likely, if an LLM is used, it will be as a piece integrating various specialized agents. I, not an expert in AI interpretability or alignment research, can't say if what they're doing is worthwhile or not in addressing existential risk. But I also don't know if actual experts can say that either. > how much resources do you think should be dedicated to regulating it? Definitely not a lower amount than we currently are allocating. > what's the proposal here? That the smart people here stop looking for any excuse to deny and ridicule the existential threat posed by future AI. Every thread involving OpenAI (a company I personally dislike and don't trust) doesn't need to just turn into series of glib, myopic jokes.
dylan604 3y ago
Open AI is clearly overestimating the capabilities of its current product I'm sure they recognize this, and have decided that anything that comes out now would be much more favorable for them based on their current capabilities

I also think it’s ludicrous to the point of hilarity; but it’s also harmful as people who can make laws and big decisions are buying this horse shit.

RajT88 3y ago
You do have to appreciate the Machiavellian cleverness of this approach to marketing.
- sjwhevvvvvsj 3y ago
  Nah, a true Machiavellian would fool smart people too - this has the sophistication of jangling keys in front of an infant. I’m a bit embarrassed for them.
  - danielmarkbruce 3y ago
    They are fooling lots of smart but not technical people. You may not be one of them, but there are many.
    - sjwhevvvvvsj 3y ago
      Fair point.
  - kjkjadksj 3y ago
    Hard to be embarrassed at the player when jingling keys in front of an infant is often literally the game.

danielmarkbruce 3y ago

Are they? Or is this marketing genius in several ways?

therein 3y ago
Alienating the developers is always a good idea, yeah. Google does it all the time, works great.
- danielmarkbruce 3y ago
  How is this alienating developers?
  - solarpunk 3y ago
    how likely are you to start work on a project depending on an ecosystem of wildly overstated capabilities?
    - danielmarkbruce 3y ago
      100% likely. Several projects, right now. GPT-4 is the best model currently available. There are reasons why it's better to control a model and host yourself etc etc, but there are also reasons to use the best model available.
      - solarpunk 3y ago
        ... I'm not trying to be rude, but do you think maybe you have bought into the purposely exaggerated marketing?
        tempusalaria 3y ago
        GPT-4 is the best model though… the gap has closed a lot but it’s still the best I despise openai but I can’t really argue with that
        3y ago

deleted 3y ago

[deleted]

valine 3y ago

This will likely be used as evidence to justify regulating open weight models. It doesn’t matter if the models are actually dangerous, the messaging is a means to an end.

api 3y ago
Yep, the strategy seems to be to legally require AI to be closed SaaS. Otherwise OpenAI doesn't actually have much of a moat. Chips capable of running local AI models are only going to get cheaper, especially as every chip maker is now going in that direction to chase Nvidia.
- devsda 3y ago
  Not just a closed SaaS. If governments decide to set whatever 'safeguards' open AI comes up with as the safety baseline for general AI, it increases compliance costs for its competitors(both open and closed).

agnokapathetic 3y ago

They're pretty clear about it though erring on the conservative side. > While none of the above results were statistically significant, we interpret our results to indicate that access to (research-only) GPT-4 may increase experts’ ability to access information about biological threats, particularly for accuracy and completeness of tasks. This access to research-only GPT-4, along with our larger sample size, different scoring rubric, and different task design (e.g., individuals instead of teams, and significantly shorter duration) may also help explain the difference between our conclusions and those of Mouton et al. 2024, who concluded that LLMs do not increase information access at this time.

wg0 3y ago

Absolutely. In reality, it often fails to do Type gymnastics in Typescript let alone virology.

deleted 3y ago

[deleted]

aftbit 3y ago

Well ironically this study shows that GPT-4 isn't actually very good: >However, the obtained effect sizes were not large enough to be statistically significant, and our study highlighted the need for more research around what performance thresholds indicate a meaningful increase in risk. A "mild uplift" in capabilities that isn't statistically significant doesn't really sound like overestimation.

digging 3y ago
That's because they're not overestimating their product, they're trying to gauge what risk looks like.
- ctrw 3y ago
  They are going for regulatory capture because they have no moat and open source Ai models are eating their lunch.

digging 3y ago

It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending humanity once it has the capabilities is completely unsolved. We don't even know if it's solvable, let alone what approach to take. Don't you actually believe it is not essential to practice on weaker AI before we get to that point? Would you walk through a combat zone without thinking about how to protect yourself until after you hear a gunshot? I expect many replies about ChatGPT being "too stupid" to end the world. Please hold those replies, as they completely miss the point. If you consider yourself an intelligent and technical person, and you think it's not worth thinking about existential risks posed by future AI, I would like to know when you think it will be time for AI researchers (not you personally) to start preparing for those risks.

solarpunk 3y ago
what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes? if that's the case, how much resources do you think should be dedicated to regulating it? more or less than currently identified existential risks? which entities should be paying for the regulatory controls? what's the proposal here? it's odd because only this one single company that is hedging it's entire existence on "oh boy what if this thing is dangerous some time in the near future" is doing silly stunts like this. why aren't they demanding nvidia start building DRM enabled thermite charges into A100s?
- digging 3y ago
  > what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes? It certainly could. More likely, if an LLM is used, it will be as a piece integrating various specialized agents. I, not an expert in AI interpretability or alignment research, can't say if what they're doing is worthwhile or not in addressing existential risk. But I also don't know if actual experts can say that either. > how much resources do you think should be dedicated to regulating it? Definitely not a lower amount than we currently are allocating. > what's the proposal here? That the smart people here stop looking for any excuse to deny and ridicule the existential threat posed by future AI. Every thread involving OpenAI (a company I personally dislike and don't trust) doesn't need to just turn into series of glib, myopic jokes.

dylan604 3y ago

Open AI is clearly overestimating the capabilities of its current product I'm sure they recognize this, and have decided that anything that comes out now would be much more favorable for them based on their current capabilities