3 ms·

I think "unwilling to divulge company secrets" is the best explanation here. We know that OpenAI does a staged release for their models with pre-release red-te

I think "unwilling to divulge company secrets" is the best explanation here. We know that OpenAI does a staged release for their models with pre-release red-teaming. Helen says the key issue was the board struggling to "effectively supervise the company": https://nitter.net/hlntnr/status/1730034017435586920#m https://nitter.net/hlntnr/status/1730034017435586920#m Here's Nathan Labenz on how sketchy the red-teaming process for GPT4 was. Nathan states that OpenAI shut him out soon after he reached out to the board to let them know that GPT4 was a big deal and the board should be paying attention: https://nitter.net/labenz/status/1727327424244023482#m https://nitter.net/labenz/status/1727327424244023482#m [Based on the thread it seems like he reached out to people outside OpenAI in a way which could have violated a confidentiality agreement -- that could account for the shutout] My suspicion is that there was a low-level power struggle ongoing on the board for some time, but the straw that broke the camel's back was something like Nathan describes in his thread. To be honest I don't understand why his thread is getting so little play. It seems like a key piece of the puzzle. In any case, I don't think it would've been right for Helen to say publicly that "we hear GPT-5 is lit but Sam isn't letting us play with it", since "GPT-5 is lit" would be considered confidential information that she shouldn't unilaterally reveal.

g42gregory 3y ago
So what is Nathan Labenz saying? That GPT-4 is dangerous somehow? It will get many people out of jobs? MS Office got all the typists out of jobs. OCR and Medical Software got all the medical transcriptionists out of jobs. And they created a lot more jobs in the process. GPT-4 is a very powerful tool. It has not a whiff of AGI in it. The whole AGI "scare" seems to be extremely political.
- 0xDEAFBEAD 3y ago
  Nathan says the initial version of GPT-4 he red-teamed was "totally amoral" and it was happy to plan assassinations for him: https://nitter.net/labenz/status/1727327464328954121#m https://nitter.net/labenz/status/1727327464328954121#m Reducing the cost of medical transcription to ~$0 is one thing. Reducing the cost of assassination to ~$0 is quite another.
  - g42gregory 3y ago
    This is a piece of software. What would "totally amoral" even mean here? It's an inanimate object, it has no morals, feelings, conscience, etc... He gives it an amoral input, he gets an amoral output.
    - deleted 3y ago
      [deleted]
    - 0xDEAFBEAD 3y ago
      I mean, there's a sense in which my mind is software that's being run by my brain, right? Yet that doesn't absolve me of moral responsibility. In any case, an F16 fighter jet is amoral in a certain sense, but it wouldn't be smart to make F16s available to the average Joe so he can conduct an airstrike whenever he wants.
      - consp 3y ago
        Completely depends on your morality. I'm pretty sure there are some libertarians out there who think the most basic version of the second amendment includes owning F16 with live weapons.
        0xDEAFBEAD 3y ago
        Sure -- if you're a libertarian who thinks it should be possible to purchase an F16 without a background check, that seems consistent with the position that an amoral GPT-4 should be broadly available.
        civilitty 3y ago
        What kind of background check do you think exists when buying a fighter jet? It’s kind of a moot point since only one F16 exists in civilian hands but you can buy other jets with weapon hardpoints for a million. Under 3 million if you want to go supersonic too. The cheapest fighter jet is on the order of $250k. There’s zero background check.
- djhn 3y ago
  Medical transcriptionists out of jobs? As far as I'm aware, medical transcription is still very much the domain of human experts, since getting doctors to cater their audio notes to the whims of software turned out to be impossible (at least in my corner of the EU).
  - consp 3y ago
    My mom had to do this for her job and apparently some of the docs are so mumbly they have to infer a lot of the words from context and type of procedure but there is a lot of crossover everywhere so it depends a lot on which doc is mumbeling what. And yes you need special training for it (no medical degree though)
- tomjakubowski 3y ago
  FWIW I had a doctor's appointment just this year with a transcriptionist present. (USA)
- vikramkr 3y ago
  Did they really create a ton more jobs? The past few rounds of industrialization and automation have coinved with plagues/the black death that massively reduced the population, mass agitation, increasing inequality, and recently a major opioid epidemic in regions devastated by the economic changes of globalization and industrialization. I think these tools are very good and we should develop, I also think it's delusional to think it'll just balance out magically and dangerous to expect our already failed systems to protect people left behind. Doesnt exactly look like they worked any of the previous times!
- ben_w 3y ago
  Would you rather: 1) be surprised by and unprepared for AGI and every step on the path to that goal, or 2) have the developers of every AI examine their work for its potential impact, both when used as intended and when misused, with regard to all the known ways even non-G AI can already go wrong: bugs, making stuff up, reward hacking, domain shift, etc.; or economically speaking how many people will be made unemployed by just a fully [G]eneral self-driving AI? What happens if this is deployed over one year? Do existing LLMs get used by SEO to systematically undermine the assumptions behind Page rank and thus web search?; and culturally: how much economic harm do artists really suffer from Diffusion models? Are harms caused by AI porn unavoidable thanks to human psychology, or artefacts of out milieu that will disappear as people become accustomed to it? There's also a definition problem for AGI, with a lot of people using a standard I'd reserve for ASI. Also some people think an AGI would have to be conscious, I don't know why. The best outcome is Fully Automated Luxury Communism, but assuming the best outcome is the median outcome is how actual Communism turned into gulags and secret police.
- Towaway69 3y ago
  It seems that silicon valley is developing not a conscious sentient piece of software rather a conscience, a moral compass is beginning to emerge. After giving us Facebook, insta, Twitter, and ego-search, influencing many people negatively, suddenly there are moral values being discussed amongst those that decide our collective tech futures. AI will have even more influence on humankind and some are questioning the morality of money (hint: money had no morals).

g42gregory 3y ago

So what is Nathan Labenz saying? That GPT-4 is dangerous somehow? It will get many people out of jobs? MS Office got all the typists out of jobs. OCR and Medical Software got all the medical transcriptionists out of jobs. And they created a lot more jobs in the process. GPT-4 is a very powerful tool. It has not a whiff of AGI in it. The whole AGI "scare" seems to be extremely political.

0xDEAFBEAD 3y ago
Nathan says the initial version of GPT-4 he red-teamed was "totally amoral" and it was happy to plan assassinations for him: https://nitter.net/labenz/status/1727327464328954121#m https://nitter.net/labenz/status/1727327464328954121#m Reducing the cost of medical transcription to ~$0 is one thing. Reducing the cost of assassination to ~$0 is quite another.
- g42gregory 3y ago
  This is a piece of software. What would "totally amoral" even mean here? It's an inanimate object, it has no morals, feelings, conscience, etc... He gives it an amoral input, he gets an amoral output.
  - deleted 3y ago
    [deleted]
  - 0xDEAFBEAD 3y ago
    I mean, there's a sense in which my mind is software that's being run by my brain, right? Yet that doesn't absolve me of moral responsibility. In any case, an F16 fighter jet is amoral in a certain sense, but it wouldn't be smart to make F16s available to the average Joe so he can conduct an airstrike whenever he wants.
    - consp 3y ago
      Completely depends on your morality. I'm pretty sure there are some libertarians out there who think the most basic version of the second amendment includes owning F16 with live weapons.
      - 0xDEAFBEAD 3y ago
        Sure -- if you're a libertarian who thinks it should be possible to purchase an F16 without a background check, that seems consistent with the position that an amoral GPT-4 should be broadly available.
        civilitty 3y ago
        What kind of background check do you think exists when buying a fighter jet? It’s kind of a moot point since only one F16 exists in civilian hands but you can buy other jets with weapon hardpoints for a million. Under 3 million if you want to go supersonic too. The cheapest fighter jet is on the order of $250k. There’s zero background check.
djhn 3y ago
Medical transcriptionists out of jobs? As far as I'm aware, medical transcription is still very much the domain of human experts, since getting doctors to cater their audio notes to the whims of software turned out to be impossible (at least in my corner of the EU).
- consp 3y ago
  My mom had to do this for her job and apparently some of the docs are so mumbly they have to infer a lot of the words from context and type of procedure but there is a lot of crossover everywhere so it depends a lot on which doc is mumbeling what. And yes you need special training for it (no medical degree though)
tomjakubowski 3y ago
FWIW I had a doctor's appointment just this year with a transcriptionist present. (USA)
vikramkr 3y ago
Did they really create a ton more jobs? The past few rounds of industrialization and automation have coinved with plagues/the black death that massively reduced the population, mass agitation, increasing inequality, and recently a major opioid epidemic in regions devastated by the economic changes of globalization and industrialization. I think these tools are very good and we should develop, I also think it's delusional to think it'll just balance out magically and dangerous to expect our already failed systems to protect people left behind. Doesnt exactly look like they worked any of the previous times!
ben_w 3y ago
Would you rather: 1) be surprised by and unprepared for AGI and every step on the path to that goal, or 2) have the developers of every AI examine their work for its potential impact, both when used as intended and when misused, with regard to all the known ways even non-G AI can already go wrong: bugs, making stuff up, reward hacking, domain shift, etc.; or economically speaking how many people will be made unemployed by just a fully [G]eneral self-driving AI? What happens if this is deployed over one year? Do existing LLMs get used by SEO to systematically undermine the assumptions behind Page rank and thus web search?; and culturally: how much economic harm do artists really suffer from Diffusion models? Are harms caused by AI porn unavoidable thanks to human psychology, or artefacts of out milieu that will disappear as people become accustomed to it? There's also a definition problem for AGI, with a lot of people using a standard I'd reserve for ASI. Also some people think an AGI would have to be conscious, I don't know why. The best outcome is Fully Automated Luxury Communism, but assuming the best outcome is the median outcome is how actual Communism turned into gulags and secret police.
Towaway69 3y ago
It seems that silicon valley is developing not a conscious sentient piece of software rather a conscience, a moral compass is beginning to emerge. After giving us Facebook, insta, Twitter, and ego-search, influencing many people negatively, suddenly there are moral values being discussed amongst those that decide our collective tech futures. AI will have even more influence on humankind and some are questioning the morality of money (hint: money had no morals).