7 ms·

A deal-making CEO who can carry rapport with the right people, make clever deals, and earn public trust can genuinely make a huge difference to a profit-seeking

A deal-making CEO who can carry rapport with the right people, make clever deals, and earn public trust can genuinely make a huge difference to a profit-seeking product company's trajectory. But when your profit-seeking company is owned by a non-profit with a public mission, that trajectory might end up pointed the wrong way. The Dev Day announcements, and especially the marketplace, can be seen as suggesting that's exactly what was happening at OpenAI. I don't think everyone there wants them to be selling cool LLM toys, especially not on a "make fast and break things" approach and with an ecosystem of startup hackers operationalizing it. (Wisely or not) I think they want to be shepherding responsible AGI before someone else does so irresponsibly.

jddj 3y ago
This is where I've ended up as well for now. I'm as distant from it all as anyone else, but I can easily believe the narrative that Ilya (et al.) didn't sign up there just to run through a tired page from the tech playbook where they make a better Amazon Alexa with an app store and gift cards and probably Black Friday sales.
- gardenhedge 3y ago
  And that is fine.. but why the immediate firing, why the controversy?
  - justaj 3y ago
    My guess is that if Sam would have found this out before being fired, he would have done his best not to be fired. As such, it would have been much more of a challenge to shift OpenAI's supposed over-focus on commerce towards a supposed non-profit focus.
  - eganist 3y ago
    The immediate firing is from our perspective. Who's to say everything else wasn't already tried in private?
    - jacquesm 3y ago
      That may be so but then they should have done it well before Altman's last week at OpenAI where they allowed him to become that much more tied to their brand as the 'face' of the operation.
      - eganist 3y ago
        For all we know, the dev day announcements were the final straw and trigger for the decision that was probably months in the making. He was already the brand, and there likely wouldn't have been a convenient time to remove him from their perspective.
        jacquesm 3y ago
        That may well be true. But that would prove that the board was out of touch with what the company was doing. If the board sees anything new on 'dev day' that means they haven't been doing their job in the first place.
        cthalupa 3y ago
        Unless seeing something new on dev day is exactly what they meant by Altman not being consistently candid.
        jacquesm 3y ago
        If Altman was doing something that ran directly against the mission of OpenAI in a way that all of the other stuff that OpenAI has been doing so far did not then I haven't seen it. OpenAI has been off-script for a long time now (compared to what they originally said) and outwardly it seemed the board was A-Ok with that. Now we either see a belated - and somewhat erratic - response to all that went before or there is some smoking gun. If there isn't they have just done themselves an immense disservice. Maybe they think they can live without donations now that the commercial ball is rolling downhill fast enough but that only works if you don't damage your brand.
cmrdporcupine 3y ago
I agree with what you've written here but would add the caveat that it's also rather terrible to be in a position where somehow "shepherding responsible AGI" is falling to these self-appointed arbiters. They strike me as woefully biased and ideological and I do not trust them. While I trust Altman even less, there's nothing I've read about Sutskever that makes me think I want him or the people who think like him around him having this kind of power. But this is where we've come to as a society. I don't think it's a good place.
- nick222226 3y ago
  I mean, aren't they self appointed because they got there first?
  - cmrdporcupine 3y ago
    No. Knew the right people, had the right funds, and said and did and thought the things compatible with getting investment from people with even more influence than them. Unless you're saying my only option is to pick and choose between different sets of people like that?
    - 38321003thrw 3y ago
      There is a political economy as well as a technical aspect to this that present inherent issues. Even if we can address the former by say regime change, the latter issue remains: the domain is technical and cognitively demanding. Thus the practitioners will generally sound sane and rational (they are smart people but that is no guarantee of anything other than technical abilities) and non-technical policy types (like most of the remaining board members at openAI) are practically compelled to take policy positions based either on ‘abstract models’ (which may be incorrect) or as after the fact reaction to observation of the mechanisms (which may be too late). The thought occurs that it is quite possible that just like humanity is really not ready (we remain concerned) to live with WMD technologies, it is possible that we have again stumbled on another technology that taxes our ethical, moral, educational, political, and economic understanding. We would be far less concerned if we were part of a civilization of generally thoughtful and responsible specimens but we’re not. This is a cynical appraisal of the situation, I realize, but tldr is “it is a systemic problem”.
beachy 3y ago
> I think they want to be shepherding responsible AGI before someone else does so irresponsibly. Is this a thing? This would be like Switzerland in WWII doing nuclear weapons research to try and get there before the Nazis. Would that make any difference whatsoever to the Nazis timeframe? No. I fail to see how the presence of "ethical" AI researchers would slow down in the slightest the bad actors who are certainly out there.
- deleted 3y ago
  [deleted]
- FormerBandmate 3y ago
  America did nuclear weapons research to get there before the Nazis and Japan and we were able to use them to stop Japan
  - vanrysss 3y ago
    So the first AGI is going to be used to kill other AGIs in the cradle ?
    - macintux 3y ago
      Which reminds me, I really need to finish Person of Interest someday.
    - kbenson 3y ago
      Or contain, or counter, or be used as a deterrent. At least, I think that's the idea being espoused here (in general, if not in the GP comment). I think U. S. VS Japan is not.necessarily the right model to be thinking here, but U.S. VS U.S.S.R., where we'd like to believe that neither nation would actually launch against the other, but both having the weapon meant they couldn't without risking severe damage in response making it a losing proposition. That said, I'm sure anyone with an AGI in their pocket/on their side will attempt to use it as a big stick against those that don't, in the Teddy Roosevelt meaning.
    - T-A 3y ago
nradov 3y ago
There is no particular reason to expect that OpenAI will be the first to build a true AGI, responsible or otherwise. So far they haven't made any demonstrable progress towards that goal. ChatGPT is an amazing accomplishment and very useful, but probably tangential to the ultimate goal. When a real AGI is eventually built it may be the result of a breakthrough from some totally unexpected source.
kelipso 3y ago
I'm guessing Altman had a bunch of experienced ML researchers writing CRUD apps and LLM toys instead of actual AI research and they weren't too happy. Personally I would be pissed as a researcher if the company took a turn and started in on improved marketing blurbs LLMs or whatever.
- Solvency 3y ago
  If every jackass brain dead move Elon Musk has ever made hasnt gotten him fired yet, then allocating too many teams to side projects instead of AI research should not be a fireable offense.
  - Mountain_Skies 3y ago
    Musk was fired as CEO of X/PayPal.
    - dragonwriter 3y ago
      Fired as the CEO of X twice, the last time right before it became PayPal.
- bertil 3y ago
  I would be shocked if that was the case. There are plenty of people I know from FAANG, now at OpenAI, where they do product design, operation, and DevOps at scale —all complicated, valuable, and worthwhile endeavors in their own right— that don’t need to get in the way of research. They are just the kind of talent that can operate a business with 90% margins to pay for that research. Could there be requests or internal projects that are less exciting for some people? Sure, but it’s not very hard to set up Chinese Walls, priorities, etc. Every one of those people had to deal with similar concerns at previous companies and would know how to apply the right principles.
jes5199 3y ago
okay but I personally do want new LLM toys. who is going to provide them, now?
- quickthrower2 3y ago
  Various camelid inspired models and open source code.
- MVissers 3y ago
  [dead]
bertil 3y ago
I think the research ambition is worthwhile, but it has raised pressing questions about financing. If shepherding responsible AGI can be done without a $10B budget in H100, sure… but it seems that scale matters. Having some people in the company sell state-of-the-art solutions to pay for the rest doing cutting-edge, expensive, necessary research isn’t a bad model. If those separations needed to be re-affirmed, the research formally separated, a board decision approved to share any model from the research arm before it’s commercialized, etc., all that could be implemented within the mission of the entity. Microsoft Research, before them Bell Labs, and many others, have worked like that.
blazespin 3y ago
If they didn't want to be responsible for running and paying for an actual service like ChatGPT, they should have spun it off to someone who did. The notion they could just build it and they will come is ludicrous. Sam understood that and was trying to figure out a model that could pay for itself. Obviously there will be mistakes made along the way. That's how it goes. Don't forget. ChatGPT has competitors. A lot of them and they're getting pretty good.

jddj 3y ago

This is where I've ended up as well for now. I'm as distant from it all as anyone else, but I can easily believe the narrative that Ilya (et al.) didn't sign up there just to run through a tired page from the tech playbook where they make a better Amazon Alexa with an app store and gift cards and probably Black Friday sales.

gardenhedge 3y ago
And that is fine.. but why the immediate firing, why the controversy?
- justaj 3y ago
  My guess is that if Sam would have found this out before being fired, he would have done his best not to be fired. As such, it would have been much more of a challenge to shift OpenAI's supposed over-focus on commerce towards a supposed non-profit focus.
- eganist 3y ago
  The immediate firing is from our perspective. Who's to say everything else wasn't already tried in private?
  - jacquesm 3y ago
    That may be so but then they should have done it well before Altman's last week at OpenAI where they allowed him to become that much more tied to their brand as the 'face' of the operation.
    - eganist 3y ago
      For all we know, the dev day announcements were the final straw and trigger for the decision that was probably months in the making. He was already the brand, and there likely wouldn't have been a convenient time to remove him from their perspective.
      - jacquesm 3y ago
        That may well be true. But that would prove that the board was out of touch with what the company was doing. If the board sees anything new on 'dev day' that means they haven't been doing their job in the first place.
        cthalupa 3y ago
        Unless seeing something new on dev day is exactly what they meant by Altman not being consistently candid.
        jacquesm 3y ago
        If Altman was doing something that ran directly against the mission of OpenAI in a way that all of the other stuff that OpenAI has been doing so far did not then I haven't seen it. OpenAI has been off-script for a long time now (compared to what they originally said) and outwardly it seemed the board was A-Ok with that. Now we either see a belated - and somewhat erratic - response to all that went before or there is some smoking gun. If there isn't they have just done themselves an immense disservice. Maybe they think they can live without donations now that the commercial ball is rolling downhill fast enough but that only works if you don't damage your brand.

cmrdporcupine 3y ago

I agree with what you've written here but would add the caveat that it's also rather terrible to be in a position where somehow "shepherding responsible AGI" is falling to these self-appointed arbiters. They strike me as woefully biased and ideological and I do not trust them. While I trust Altman even less, there's nothing I've read about Sutskever that makes me think I want him or the people who think like him around him having this kind of power. But this is where we've come to as a society. I don't think it's a good place.

nick222226 3y ago
I mean, aren't they self appointed because they got there first?
- cmrdporcupine 3y ago
  No. Knew the right people, had the right funds, and said and did and thought the things compatible with getting investment from people with even more influence than them. Unless you're saying my only option is to pick and choose between different sets of people like that?
  - 38321003thrw 3y ago
    There is a political economy as well as a technical aspect to this that present inherent issues. Even if we can address the former by say regime change, the latter issue remains: the domain is technical and cognitively demanding. Thus the practitioners will generally sound sane and rational (they are smart people but that is no guarantee of anything other than technical abilities) and non-technical policy types (like most of the remaining board members at openAI) are practically compelled to take policy positions based either on ‘abstract models’ (which may be incorrect) or as after the fact reaction to observation of the mechanisms (which may be too late). The thought occurs that it is quite possible that just like humanity is really not ready (we remain concerned) to live with WMD technologies, it is possible that we have again stumbled on another technology that taxes our ethical, moral, educational, political, and economic understanding. We would be far less concerned if we were part of a civilization of generally thoughtful and responsible specimens but we’re not. This is a cynical appraisal of the situation, I realize, but tldr is “it is a systemic problem”.

beachy 3y ago

> I think they want to be shepherding responsible AGI before someone else does so irresponsibly. Is this a thing? This would be like Switzerland in WWII doing nuclear weapons research to try and get there before the Nazis. Would that make any difference whatsoever to the Nazis timeframe? No. I fail to see how the presence of "ethical" AI researchers would slow down in the slightest the bad actors who are certainly out there.

deleted 3y ago
[deleted]
FormerBandmate 3y ago
America did nuclear weapons research to get there before the Nazis and Japan and we were able to use them to stop Japan
- vanrysss 3y ago
  So the first AGI is going to be used to kill other AGIs in the cradle ?
  - macintux 3y ago
    Which reminds me, I really need to finish Person of Interest someday.
  - kbenson 3y ago
    Or contain, or counter, or be used as a deterrent. At least, I think that's the idea being espoused here (in general, if not in the GP comment). I think U. S. VS Japan is not.necessarily the right model to be thinking here, but U.S. VS U.S.S.R., where we'd like to believe that neither nation would actually launch against the other, but both having the weapon meant they couldn't without risking severe damage in response making it a losing proposition. That said, I'm sure anyone with an AGI in their pocket/on their side will attempt to use it as a big stick against those that don't, in the Teddy Roosevelt meaning.
  - T-A 3y ago

nradov 3y ago

There is no particular reason to expect that OpenAI will be the first to build a true AGI, responsible or otherwise. So far they haven't made any demonstrable progress towards that goal. ChatGPT is an amazing accomplishment and very useful, but probably tangential to the ultimate goal. When a real AGI is eventually built it may be the result of a breakthrough from some totally unexpected source.

kelipso 3y ago

I'm guessing Altman had a bunch of experienced ML researchers writing CRUD apps and LLM toys instead of actual AI research and they weren't too happy. Personally I would be pissed as a researcher if the company took a turn and started in on improved marketing blurbs LLMs or whatever.

Solvency 3y ago
If every jackass brain dead move Elon Musk has ever made hasnt gotten him fired yet, then allocating too many teams to side projects instead of AI research should not be a fireable offense.
- Mountain_Skies 3y ago
  Musk was fired as CEO of X/PayPal.
  - dragonwriter 3y ago
    Fired as the CEO of X twice, the last time right before it became PayPal.
bertil 3y ago
I would be shocked if that was the case. There are plenty of people I know from FAANG, now at OpenAI, where they do product design, operation, and DevOps at scale —all complicated, valuable, and worthwhile endeavors in their own right— that don’t need to get in the way of research. They are just the kind of talent that can operate a business with 90% margins to pay for that research. Could there be requests or internal projects that are less exciting for some people? Sure, but it’s not very hard to set up Chinese Walls, priorities, etc. Every one of those people had to deal with similar concerns at previous companies and would know how to apply the right principles.

jes5199 3y ago

okay but I personally do want new LLM toys. who is going to provide them, now?

quickthrower2 3y ago
Various camelid inspired models and open source code.
MVissers 3y ago
[dead]

bertil 3y ago

I think the research ambition is worthwhile, but it has raised pressing questions about financing. If shepherding responsible AGI can be done without a $10B budget in H100, sure… but it seems that scale matters. Having some people in the company sell state-of-the-art solutions to pay for the rest doing cutting-edge, expensive, necessary research isn’t a bad model. If those separations needed to be re-affirmed, the research formally separated, a board decision approved to share any model from the research arm before it’s commercialized, etc., all that could be implemented within the mission of the entity. Microsoft Research, before them Bell Labs, and many others, have worked like that.

blazespin 3y ago

If they didn't want to be responsible for running and paying for an actual service like ChatGPT, they should have spun it off to someone who did. The notion they could just build it and they will come is ludicrous. Sam understood that and was trying to figure out a model that could pay for itself. Obviously there will be mistakes made along the way. That's how it goes. Don't forget. ChatGPT has competitors. A lot of them and they're getting pretty good.