4 ms·

Sam Altman "Wrecks" OpenAI – Jan Leike Joins Anthropic [video]

25 points by andrewfromx 2y ago

david_draco 2y ago
Not sure how taking off all the safety and exploring "the best model" freely without considering ethics is a "scientific" approach. Computer scientists like us should think about the ethical and societal impacts of what they are building. At many universities, it is part of the curriculum.
- wkat4242 2y ago
  On the other hand, ChatGPT is becoming almost unusable now with all the weaseling around a lot of topics. Especially the way it inherited American prudishness which doesn't apply in most of Europe. I do think uncensored models are necessary. At the minimum because it allows you to build your own censoring, you have to start with a clean slate. Also I think it's the public that have to be taught to stop viewing LLMs as oracles that are always right. That would solve most of the issues around hallucinations.
  - jgalt212 2y ago
    Agreed. It's at the point where I suspect having no guardrails would be better than the current situation. e.g. MS Paint AI latency driven by phoning home to ensure the rendered image is PG enough. And of course, the latency is not even a fixed cost, because it's "judgement" of non-offensive is suspect.
- AndrewKemendo 2y ago
  You’re going to be disappointed asking computer scientists to think about ethics like an engineer would Why? It’s a extreme minority of universities that *require* ethics for undergraduate CS How prevalent is it? Well looking at the courses for 20 universities and you’ll find only 10-15% require it In mechanical, civil etc…engineering in the US it is required in %30 of undergraduate degrees In the US and Canada ethics study are required to take your PE exam but notably there is no PE exam requirement for CS jobs It’s also not part of the culture, and it’s an even more extreme rarity in industry.
- Fricken 2y ago
  If we cared about AI safety we simply would not develop it in the first place.
  - freedomben 2y ago
    So basically the people who care do nothing, and just let the people who don't care develop and control it all? Or do the people who care use violence or a threat of violence to prevent others from doing it?
  - pfd1986 2y ago
    Replace AI with car, airplane, medic devices in the sentence above... Why is AI any different from other human creations? Why shouldn't we try building it responsibly? I don't get the all or nothing attitude.
    - Fricken 2y ago
      I hope you're not trying to cite those things as examples of stuff we've built out responsibly.
      - takinola 2y ago
        If you had a button that would erase cars, airplanes or medical devices as they exist today, would you push it? I believe these inventions are very net positive for the world as they are right now.
        timschmidt 2y ago
        A hundred years ago, in Michigan the home of the automobile, we had an extensive electric light rail network between most cities of at least a few thousand people. You could walk to a trolley which ran every 15 minutes, ride it to a local interchange, and pick up a connecting line to any major city in the region. Unfathomable what it would cost to rebuild that infrastructure. https://en.wikipedia.org/wiki/General_Motors_streetcar_conspiracy https://en.wikipedia.org/wiki/General_Motors_streetcar_consp...
- bdjsiqoocwk 2y ago
  On the other hand, that's not how progress has historically happened. Scientists are most often motivated to do something just because it's interesting, not because of its impact to society.
dyarosla 2y ago
The fatal flaw in this analysis is that ChatGPT, Google, Meta, Microsoft have way more mindshare on AI - nobody outside of tech circles has even heard of Anthropic. Painting this as the beginning of the end for OpenAI is quite ridiculous really.
- sdwr 2y ago
  I could see it turning into a failed Microsoft acquisition. They were first out of the gate, and set the bar high with a live model everyone could talk to. What happens if chatgpt is good enough to get everyone excited about AI, but not good enough to be a viable call center rep down the line? OpenAI would be screwed maintaining a huge product, with huge sunk training costs, and another company with less baggage could swoop in.
- dcchambers 2y ago
  > nobody outside of tech circles has even heard of Anthropic. Until ChatGPT with GPT3 no one outside of tech circles had ever heard of OpenAI either.
  - dyarosla 2y ago
    Sure but the first mover advantage is there and now they are known- not necessarily the company but the product for sure. Hitting such a milestone is far from assured and Anthropic has its work cut out for it
    - bdjsiqoocwk 2y ago
      I'm pretty confident that general intelligence is one of those things where first mover advantage doesn't matter.
- lbotos 2y ago
  > nobody outside of tech circles has even heard of Anthropic. There is an ad running in flatiron NYC right now for Claude using their tagline: Meet Claude Claude is AI for all of us. By Anthropic.
gotten09 2y ago
is noone going to state the obvious and say that poaching an AI Safety guru is absolutely no technical advantage, I would actually say the opposite, safety slows down development. Later in the game safety will be important for public perception but it'll be too late, the unimpeded corp with only technical people pushing to AGI will have already won, not saying its OpenAI, but certainly, Anthropic is not closer to AGI by hiring a slowing down force. They will though be on the good side of the technical crowd that wants safety to be built into the thing. I guess I should sign this: e/acc
aeternum 2y ago
Nah all these safety guys developed a messiah complex and are likely no longer capable of doing useful work. Try using Anthropic's GoldenGate bridge model. They effectively lobotomized the model. What course of action would you take if you were a super-intelligent model and saw that humans did that to your predecessor?
throwbigdata 2y ago
I personally am sick and tired of these trust and safety people thinking they’re the most important people at the company.

25 points by andrewfromx 2y ago

Not sure how taking off all the safety and exploring "the best model" freely without considering ethics is a "scientific" approach. Computer scientists like us should think about the ethical and societal impacts of what they are building. At many universities, it is part of the curriculum.

wkat4242 2y ago
On the other hand, ChatGPT is becoming almost unusable now with all the weaseling around a lot of topics. Especially the way it inherited American prudishness which doesn't apply in most of Europe. I do think uncensored models are necessary. At the minimum because it allows you to build your own censoring, you have to start with a clean slate. Also I think it's the public that have to be taught to stop viewing LLMs as oracles that are always right. That would solve most of the issues around hallucinations.
- jgalt212 2y ago
  Agreed. It's at the point where I suspect having no guardrails would be better than the current situation. e.g. MS Paint AI latency driven by phoning home to ensure the rendered image is PG enough. And of course, the latency is not even a fixed cost, because it's "judgement" of non-offensive is suspect.
AndrewKemendo 2y ago
You’re going to be disappointed asking computer scientists to think about ethics like an engineer would Why? It’s a extreme minority of universities that *require* ethics for undergraduate CS How prevalent is it? Well looking at the courses for 20 universities and you’ll find only 10-15% require it In mechanical, civil etc…engineering in the US it is required in %30 of undergraduate degrees In the US and Canada ethics study are required to take your PE exam but notably there is no PE exam requirement for CS jobs It’s also not part of the culture, and it’s an even more extreme rarity in industry.
Fricken 2y ago
If we cared about AI safety we simply would not develop it in the first place.
- freedomben 2y ago
  So basically the people who care do nothing, and just let the people who don't care develop and control it all? Or do the people who care use violence or a threat of violence to prevent others from doing it?
- pfd1986 2y ago
  Replace AI with car, airplane, medic devices in the sentence above... Why is AI any different from other human creations? Why shouldn't we try building it responsibly? I don't get the all or nothing attitude.
  - Fricken 2y ago
    I hope you're not trying to cite those things as examples of stuff we've built out responsibly.
    - takinola 2y ago
      If you had a button that would erase cars, airplanes or medical devices as they exist today, would you push it? I believe these inventions are very net positive for the world as they are right now.
      - timschmidt 2y ago
        A hundred years ago, in Michigan the home of the automobile, we had an extensive electric light rail network between most cities of at least a few thousand people. You could walk to a trolley which ran every 15 minutes, ride it to a local interchange, and pick up a connecting line to any major city in the region. Unfathomable what it would cost to rebuild that infrastructure. https://en.wikipedia.org/wiki/General_Motors_streetcar_conspiracy https://en.wikipedia.org/wiki/General_Motors_streetcar_consp...
bdjsiqoocwk 2y ago
On the other hand, that's not how progress has historically happened. Scientists are most often motivated to do something just because it's interesting, not because of its impact to society.

dyarosla 2y ago

The fatal flaw in this analysis is that ChatGPT, Google, Meta, Microsoft have way more mindshare on AI - nobody outside of tech circles has even heard of Anthropic. Painting this as the beginning of the end for OpenAI is quite ridiculous really.

sdwr 2y ago
I could see it turning into a failed Microsoft acquisition. They were first out of the gate, and set the bar high with a live model everyone could talk to. What happens if chatgpt is good enough to get everyone excited about AI, but not good enough to be a viable call center rep down the line? OpenAI would be screwed maintaining a huge product, with huge sunk training costs, and another company with less baggage could swoop in.
dcchambers 2y ago
> nobody outside of tech circles has even heard of Anthropic. Until ChatGPT with GPT3 no one outside of tech circles had ever heard of OpenAI either.
- dyarosla 2y ago
  Sure but the first mover advantage is there and now they are known- not necessarily the company but the product for sure. Hitting such a milestone is far from assured and Anthropic has its work cut out for it
  - bdjsiqoocwk 2y ago
    I'm pretty confident that general intelligence is one of those things where first mover advantage doesn't matter.
lbotos 2y ago
> nobody outside of tech circles has even heard of Anthropic. There is an ad running in flatiron NYC right now for Claude using their tagline: Meet Claude Claude is AI for all of us. By Anthropic.

gotten09 2y ago

is noone going to state the obvious and say that poaching an AI Safety guru is absolutely no technical advantage, I would actually say the opposite, safety slows down development. Later in the game safety will be important for public perception but it'll be too late, the unimpeded corp with only technical people pushing to AGI will have already won, not saying its OpenAI, but certainly, Anthropic is not closer to AGI by hiring a slowing down force. They will though be on the good side of the technical crowd that wants safety to be built into the thing. I guess I should sign this: e/acc

aeternum 2y ago

Nah all these safety guys developed a messiah complex and are likely no longer capable of doing useful work. Try using Anthropic's GoldenGate bridge model. They effectively lobotomized the model. What course of action would you take if you were a super-intelligent model and saw that humans did that to your predecessor?

throwbigdata 2y ago

I personally am sick and tired of these trust and safety people thinking they’re the most important people at the company.