6 ms·

I don’t understand this. Won’t that hurt their progress on GPT-5?

by wrasee 3y ago

ben_w 3y ago
The public position (as opposed to the rumour mills) is that they're not working on a 5, and don't intend to at least until they can figure out how to do it safely.
- ethanbond 3y ago
  [or until someone else starts catching up] The same dynamic that incentivized a mass rollout of these unaligned systems will be perfectly sufficient to incentivize mass rollout of stronger, also unaligned systems.
  - ben_w 3y ago
    Yeah, that's one of Yudkowsky's fears. I'm not going to take his fears as gospel, as he's spent so long focussing on his fear there's a danger of availability heuristic/attention biases having him take the worst case for everything. I'm still going to promote caution, as the more potent a tech the greater the downside of being wrong, but it currently looks like we can cooperate with each other in this IRL iterated prisoner's dilemma.
- namaria 3y ago
  And safely here means non disruptive to established businesses. If they create a tool so powerful that it can replace entire professional classes, that would just mean people can bypass the employers and get their value directly from an API. OpenAI needs to make sure they have a business model where they can charge enterprise fees and provide value to corporations, not directly to people. ChatGPT was merely a publicity stunt. I do believe they got scared of how much value people could derive independently from it tho, hence the "regulate this" pressure.
  - ben_w 3y ago
    > And safely here means non disruptive to established businesses Why would OpenAI care about that? No, it means safety, as in not giving out dangerous answers that get people killed.
    - namaria 3y ago
      Because they belong to Microsoft. Because they are funded by rich people who own established businesses. Because they are the privileged .01% who are invested in the status quo. Because their whole networks consist of people with entrenched business interests. Pick one or more. Why would you think rich American engineers are in general any more worried about the overall safety of the world then their own self interest? That's what they show day in day out with their decisions.
      - ben_w 3y ago
        > Because they belong to Microsoft They don't — 49%, and they get to buy themselves out of even that, and if they do that by eating MS's lunch? Then it would suck to be an MS shareholder. And they've been cautious since before the investment, with the cautious announcement of GPT-2 a month before the change in corporate structure to allow investors. But even if OpenAI were owned completely by MS, why should MS care? Disrupting all employment sounds like the raison d'être for most if not all of silicon valley. At the very least, imagine all the employees MS could let go if GPT-4 was even 80% as good as the advertisements in the form of "buy my guide for how to use GPT to make a fortune with best selling books/award winning advertisements/fantastic business proposals" keep claiming. > Why would you think rich American engineers are in general any more worried about the overall safety of the world then their own self interest? One is a subset of the other. People regularly disagree about how risky things are, but they don't generally pick "destroy all things" over "destroy that which is close to me" when they actually believe that's the real choice and not, say, a panicked Chicken Little (which is how I think LeCun sees Yudkowsky).
        namaria 3y ago
- captainbland 3y ago
  I think the reality is that training a "GPT-5" which is as much of a bump as GPT-4 was over GPT-3 or that was over GPT-2 would cost in the billions of dollars using today's hardware so they're choosing to wait it out until the risk of them screwing it up doesn't cost as much as a state of the art microfabrication plant.
- ilaksh 3y ago
  I believe that the superalignment thing will basically be GPT-5.
  - ben_w 3y ago
    I think it's a necessary precursor to any future model. Would you want to spend ten times as much (or whatever) red-teaming the next increase in capabilities? And what would make you more confident in advance that the red team would spot everything?
- raydev 3y ago
  More cynically, they think they've hit a wall and a projected GPT-5 won't be a huge or meaningful jump in performance, so they'd like everyone else to slow down too.

I don’t understand this. Won’t that hurt their progress on GPT-5?

by wrasee 3y ago

I don’t understand this. Won’t that hurt their progress on GPT-5?

The public position (as opposed to the rumour mills) is that they're not working on a 5, and don't intend to at least until they can figure out how to do it safely.

ethanbond 3y ago
[or until someone else starts catching up] The same dynamic that incentivized a mass rollout of these unaligned systems will be perfectly sufficient to incentivize mass rollout of stronger, also unaligned systems.
- ben_w 3y ago
  Yeah, that's one of Yudkowsky's fears. I'm not going to take his fears as gospel, as he's spent so long focussing on his fear there's a danger of availability heuristic/attention biases having him take the worst case for everything. I'm still going to promote caution, as the more potent a tech the greater the downside of being wrong, but it currently looks like we can cooperate with each other in this IRL iterated prisoner's dilemma.
namaria 3y ago
And safely here means non disruptive to established businesses. If they create a tool so powerful that it can replace entire professional classes, that would just mean people can bypass the employers and get their value directly from an API. OpenAI needs to make sure they have a business model where they can charge enterprise fees and provide value to corporations, not directly to people. ChatGPT was merely a publicity stunt. I do believe they got scared of how much value people could derive independently from it tho, hence the "regulate this" pressure.
- ben_w 3y ago
  > And safely here means non disruptive to established businesses Why would OpenAI care about that? No, it means safety, as in not giving out dangerous answers that get people killed.
  - namaria 3y ago
    Because they belong to Microsoft. Because they are funded by rich people who own established businesses. Because they are the privileged .01% who are invested in the status quo. Because their whole networks consist of people with entrenched business interests. Pick one or more. Why would you think rich American engineers are in general any more worried about the overall safety of the world then their own self interest? That's what they show day in day out with their decisions.
    - ben_w 3y ago
      > Because they belong to Microsoft They don't — 49%, and they get to buy themselves out of even that, and if they do that by eating MS's lunch? Then it would suck to be an MS shareholder. And they've been cautious since before the investment, with the cautious announcement of GPT-2 a month before the change in corporate structure to allow investors. But even if OpenAI were owned completely by MS, why should MS care? Disrupting all employment sounds like the raison d'être for most if not all of silicon valley. At the very least, imagine all the employees MS could let go if GPT-4 was even 80% as good as the advertisements in the form of "buy my guide for how to use GPT to make a fortune with best selling books/award winning advertisements/fantastic business proposals" keep claiming. > Why would you think rich American engineers are in general any more worried about the overall safety of the world then their own self interest? One is a subset of the other. People regularly disagree about how risky things are, but they don't generally pick "destroy all things" over "destroy that which is close to me" when they actually believe that's the real choice and not, say, a panicked Chicken Little (which is how I think LeCun sees Yudkowsky).
      - namaria 3y ago
captainbland 3y ago
I think the reality is that training a "GPT-5" which is as much of a bump as GPT-4 was over GPT-3 or that was over GPT-2 would cost in the billions of dollars using today's hardware so they're choosing to wait it out until the risk of them screwing it up doesn't cost as much as a state of the art microfabrication plant.
ilaksh 3y ago
I believe that the superalignment thing will basically be GPT-5.
- ben_w 3y ago
  I think it's a necessary precursor to any future model. Would you want to spend ten times as much (or whatever) red-teaming the next increase in capabilities? And what would make you more confident in advance that the red team would spot everything?
raydev 3y ago
More cynically, they think they've hit a wall and a projected GPT-5 won't be a huge or meaningful jump in performance, so they'd like everyone else to slow down too.