6 ms·

Former OpenAI employee quit to avoid 'working for the Titanic of AI'

17 points by RafelMri 2y ago

ChicagoDave 2y ago
The iceberg is definitely return on investment. OpenAI will eventually hit a peak that is not AGI and it’s shareholders will revolt.
- inciampati 2y ago
  Probably exhaustion of human created data. The plateau is coming and much smarter techniques will be needed to go beyond it.
- bruce511 2y ago
  I don't think there'll be AGI (which is in itself a term that means different things to different people) anytime soon, if ever. That doesn't detract from the utility of what is there already. I'm currently using it quite a lot as a "tutor" (of sorts) as I deep-dive into an area I have limited experience with. Like all tutors it gets some things wrong (which in turn reflects human misunderstandings). It is sufficiently good though at directing the flow, which in turn gives me Google terms to search. I say this not as a rabid AI fan, I've come to it reluctantly, but its proving useful. So I don't think shareholders will revolt (which means what? That the share price will drop?) - I think they'll just continue to provide (improved) utility and their user base will grow. This isn't like fusion where there's a binary measure of success. Of course, like any business, there will be challenges along the way. Some survive those (Apple, MS, Google etc), some don't (Sun, DEC, Amiga). Predicting which ones succeed and which ones fail is hard. And seldom correlated to culture. The article is fluff because it makes no conclusion, adds no data, is just a single ex-employees opinion. Meh.
  - kaast202 2y ago
    it is useful, i do not understand people hating on it calling it useless. I have used it countless times for various things from work to things around the house.
    - ChicagoDave 2y ago
      It is not useless and no one is saying that it is. What we are saying is that GenAI will never be able to go beyond code helper or assistant and the people hyping these tools continually promise AGI or something that can solve complex problems. The ROI is just not there and eventually there will be a margin call.
      - bruce511 2y ago
        Since it is useful without AGI, and since that means it can be monetized, I don't believe the "shareholders will revolt" (which was the original assertion.) There us plenty of value yo be unlocked well short of AGI. Are investors excited? Yes. Is that excitement being fueled by hype? yes. Will excitement cool eventually? Sure. Will some investors, investing in the wrong thing, lose money? Sure. But "revolt" (whatever that means) - no, IMO not.
- m463 2y ago
  Maybe it will be like tesla and FSD. The promises will be enough to get to usable AI as a helpful tool which makes money, without the perfectionism of something impossible. In my analogy, personally I don't care about FSD, what you get is helpful. Dangerous is as overblown as EV fires. (think of human drivers and gasoline fires)
AceJohnny2 2y ago
What a fluff piece. One single person, from an appendix team, quit and speaks up about being unhappy with the direction of the company. This happens all the time everywhere. The only mild relevance is that it's about today's Hot Topic company.
- danpalmer 2y ago
  That’s “appendix team” being their super alignment team responsible for the core of the safety efforts?
  - ThrowawayTestr 2y ago
    Appendix is a perfect description for that team.
    - datameta 2y ago
      How so? You believe AI alignment research is unimportant? Or do you mean from the perspective of the OpenAI board
      - analognoise 2y ago
        Of course “AI alignment” is bunk.
        datameta 2y ago
        Because the bet is that LLMs won't lead to AGI? We need to learn the lessons now, not later
        talldayo 2y ago
        LLMs don't even lead to sub-human intelligence. It produces text that qualifies for a number of truth-based benchmarks which we use as a statistical plinko-board to see which model's random-number-generator is tuned better. It cannot "think" because it is a bunch of text tokens being drawn from a heuristic hat. You can map a Markov Chain or a cat on a typewriter to these tests; they're not a measure of human intelligence at all. There aren't lessons to learn, besides the fact that we're wasting our time. The only risk around AGI is that people will trust it as much as they trust current AI, and then risk their own annihilation when they casually accept a blatantly hallucinated answer. It's like if HAL-9000 came preinstalled with Alzheimer's, and we're still adamant that we can trust him to operate the airlock. It's time to get real and admit that any risk imposed by AGI (or regular AI, for that matter) is a failure in implementation. The concept of alignment is a bald-faced marketing ploy intent on selling "AI safety" to people that don't know what AI even is.
deleted 2y ago
[deleted]
tiredofdiz 2y ago
[flagged]
- Frummy 2y ago
  Boiling frog syndrome None of this even looked to be on the horizon a few years ago
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
kernelsanderz 2y ago
The actual podcast and interview is here https://youtu.be/dzQlRt3y5mU?si=qfS0DlPVcjBn-0zd https://youtu.be/dzQlRt3y5mU?si=qfS0DlPVcjBn-0zd

17 points by RafelMri 2y ago

The iceberg is definitely return on investment. OpenAI will eventually hit a peak that is not AGI and it’s shareholders will revolt.

inciampati 2y ago
Probably exhaustion of human created data. The plateau is coming and much smarter techniques will be needed to go beyond it.
bruce511 2y ago
I don't think there'll be AGI (which is in itself a term that means different things to different people) anytime soon, if ever. That doesn't detract from the utility of what is there already. I'm currently using it quite a lot as a "tutor" (of sorts) as I deep-dive into an area I have limited experience with. Like all tutors it gets some things wrong (which in turn reflects human misunderstandings). It is sufficiently good though at directing the flow, which in turn gives me Google terms to search. I say this not as a rabid AI fan, I've come to it reluctantly, but its proving useful. So I don't think shareholders will revolt (which means what? That the share price will drop?) - I think they'll just continue to provide (improved) utility and their user base will grow. This isn't like fusion where there's a binary measure of success. Of course, like any business, there will be challenges along the way. Some survive those (Apple, MS, Google etc), some don't (Sun, DEC, Amiga). Predicting which ones succeed and which ones fail is hard. And seldom correlated to culture. The article is fluff because it makes no conclusion, adds no data, is just a single ex-employees opinion. Meh.
- kaast202 2y ago
  it is useful, i do not understand people hating on it calling it useless. I have used it countless times for various things from work to things around the house.
  - ChicagoDave 2y ago
    It is not useless and no one is saying that it is. What we are saying is that GenAI will never be able to go beyond code helper or assistant and the people hyping these tools continually promise AGI or something that can solve complex problems. The ROI is just not there and eventually there will be a margin call.
    - bruce511 2y ago
      Since it is useful without AGI, and since that means it can be monetized, I don't believe the "shareholders will revolt" (which was the original assertion.) There us plenty of value yo be unlocked well short of AGI. Are investors excited? Yes. Is that excitement being fueled by hype? yes. Will excitement cool eventually? Sure. Will some investors, investing in the wrong thing, lose money? Sure. But "revolt" (whatever that means) - no, IMO not.
m463 2y ago
Maybe it will be like tesla and FSD. The promises will be enough to get to usable AI as a helpful tool which makes money, without the perfectionism of something impossible. In my analogy, personally I don't care about FSD, what you get is helpful. Dangerous is as overblown as EV fires. (think of human drivers and gasoline fires)

AceJohnny2 2y ago

What a fluff piece. One single person, from an appendix team, quit and speaks up about being unhappy with the direction of the company. This happens all the time everywhere. The only mild relevance is that it's about today's Hot Topic company.

danpalmer 2y ago
That’s “appendix team” being their super alignment team responsible for the core of the safety efforts?
- ThrowawayTestr 2y ago
  Appendix is a perfect description for that team.
  - datameta 2y ago
    How so? You believe AI alignment research is unimportant? Or do you mean from the perspective of the OpenAI board
    - analognoise 2y ago
      Of course “AI alignment” is bunk.
      - datameta 2y ago
        Because the bet is that LLMs won't lead to AGI? We need to learn the lessons now, not later
        talldayo 2y ago
        LLMs don't even lead to sub-human intelligence. It produces text that qualifies for a number of truth-based benchmarks which we use as a statistical plinko-board to see which model's random-number-generator is tuned better. It cannot "think" because it is a bunch of text tokens being drawn from a heuristic hat. You can map a Markov Chain or a cat on a typewriter to these tests; they're not a measure of human intelligence at all. There aren't lessons to learn, besides the fact that we're wasting our time. The only risk around AGI is that people will trust it as much as they trust current AI, and then risk their own annihilation when they casually accept a blatantly hallucinated answer. It's like if HAL-9000 came preinstalled with Alzheimer's, and we're still adamant that we can trust him to operate the airlock. It's time to get real and admit that any risk imposed by AGI (or regular AI, for that matter) is a failure in implementation. The concept of alignment is a bald-faced marketing ploy intent on selling "AI safety" to people that don't know what AI even is.

deleted 2y ago

[deleted]

tiredofdiz 2y ago

[flagged]

Frummy 2y ago
Boiling frog syndrome None of this even looked to be on the horizon a few years ago

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

kernelsanderz 2y ago

The actual podcast and interview is here https://youtu.be/dzQlRt3y5mU?si=qfS0DlPVcjBn-0zd https://youtu.be/dzQlRt3y5mU?si=qfS0DlPVcjBn-0zd