9 ms·

To quote Spaceballs, they're not doing it for money, they're doing it for a shitload of money.

by rippercushions 4y ago

93po 4y ago
OpenAI, if successful, will likely become the most valuable company in the history of the planet, both past and future.
- ratorx 4y ago
  Really? I see a lot of competition in the space. I think they’d have to become significantly more successful than their competitors (some of which are massive AI powerhouses themselves) to achieve the market dominance necessary.
  - TeMPOraL 4y ago
    I think what GP is saying is that success of OpenAI means making a lot of profit, and then triggering the AI apocalypse - which is how they become most valuable company in history both past and future.
    - novaRom 4y ago
      Can you elaborate, what is "the AI apocalypse"? Is it just a symbolic metaphor or is there any scientific research behind this words? For me it's rather more unpredictable toxic environment we observe in the world currently, dominated by purely human-made destructive decisions, often based on purely animal instincts.
      - flangola7 4y ago
        Without AI control loss: Permanent dictatorship by whoever controls AI. With AI control loss: AI converts the atoms of the observable universe to maximally achieve whatever arbitrary goal it thinks it was given. These are natural conclusions that can be drawn from the implied abilities of a superintelligent entity.
        klibertp 4y ago
        > whatever arbitrary goal it thinks it was given [...] abilities of a superintelligent entity Don't these two phrases contradict each other? Why would a super-intelligent entity need to be given a goal? The old argument that we'll fill the universe with staplers pretty much assumes and requires the entity NOT to be super-intelligent. An AGI would only became one when it gets the ability to formulate its own goals, I think. Not that it helps much if that goal is somehow contrary to the existence of the human race, but if the goal is self-formulated, then there's a sliver of hope that it can be changed. > Permanent dictatorship by whoever controls AI. And that is honestly frightening. We know for sure that there are ways of speaking, writing, or showing things that are more persuasive than others. We've been perfecting the art of convincing others to do what we want them to do for millennia. We got quite good at it, but as with everything human, we lack rigor and consistence in that, even the best speeches are uneven in their persuasive power. An AI trained for transforming simple prompts into a mapping of demographic to what will most likely convince that demographic to do what's prompted doesn't need to be an AGI. It doesn't even need to be much smarter than it already is. Whoever implements it first will most likely first try to convince everyone that all AI is bad (other than their own) and if they succeed, the only way to change the outcome would be a time machine or mental disorders. (Armchair science-fiction reader here, pure speculation without any facts, in case you wondered :))
        flangola7 4y ago
        The point of intelligence is to achieve goals. I don't think Microsoft and others are pouring in billions of dollars without the expectation of telling it to do things. AI can already formulate its own sub-goals, goals that help it achieve its primary goal. We've seen this time and time again in simple reinforcement learning systems over the past two decades. We won't be able to change the primary goal unless we build the AGI so that it permits us because a foreseeable sub-goal is self-preservation. The AGI knows if its programming is changed the primary goal won't be achieved, and thus has incentive to prevent that. AI propaganda will be unmatched but it may not be needed for long. There are already early robots that can carry out real life physical tasks in response to a plain English command like "bring me the bag of chips from the kitchen drawer." Commands of "detain or shoot all resisting citizens" will be possible later.
      - kordlessagain 4y ago
        If the assertion that GPT-3 is a "stochastic parrot" is wrong, there will be an apocalypse because whoever controls an AI that can reason is going to win it all. The opinions that it is or isn't "reasoning" are widely varied and depend heavily on interpretation of the interactions, many of which are hersey. My own testing with OpenAI calls + using Weaviate for storing historical data of exchanges indicates that such a beast appears to have the ability to learn as it goes. I've been able to teach such a system to write valid SQL from plain text feedback and from mistakes it makes by writing errors from the database back into Weaviate (which is then used to modify the prompt next time it runs).
      - YeGoblynQueenne 4y ago
        >> Is it just a symbolic metaphor or is there any scientific research behind this words? Of course there is. See (Cameron, 1984) (https://en.wikipedia.org/wiki/The_Terminator https://en.wikipedia.org/wiki/The_Terminator)
      - LordDragonfang 4y ago
        https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
- TeMPOraL 4y ago
  Let's hope that they aren't there, given your nicely concealed prediction it'll be the end of the world as we know it.
- ejb999 4y ago
  seriously doubt it - what they are doing, others can do - and if they start generating a lot of revenue, it will attract competition - lots of it. They don't have a moat big enough that many millions of dollars can't defeat.
  - hyperthesis 4y ago
    What if they have an internal ChatGPTzero, training and reprogramming itself, iterating at inhuman speed? A headstart in an exponential is a moat. It surely will have huge blindspots (and people do too), but perhaps it will be good enough for self-improvement... or will be soon.
    - jasonjmcghee 4y ago
      It's a fundamentally different problem. AlphaZero (DeepMind) was able to be trained this way because it was setup with an explicit reward function and end condition. Competitive self-play needs a reward function. It can't just "self-improve towards general intelligence". What's the fitness function of intelligence?
      - pixl97 4y ago
        Actually you are stating 2 different problems at the same time. A lot of intelligence is based around "Don't die, also, have babies". Well AI doesn't have that issue as long as it produces a good enough answer we'll keep the power flowing to it. The bigger issue, and one that is likely far more dangerous, is "Ok, what if it could learn like this". You've created super intelligence with access to huge amounts of global information and is pretty much immortal unless humans decide to pull the plug. The expected alignment of your superintelligent machine is to design a scenario where we cannot unplug the AI without suffering some loss (monetary is always a good one, rich people never like to lose money and will let an AI burn the earth first). The alignment issue on a potential superintelligence is, I don't believe, a solvable problem. Getting people not to be betraying bastards is hard enough at standard intelligence levels, having a thing that could potentially be way smarter and connected to way more data is not going to be controllable in any form or fashion.
- berkle4455 4y ago
  It’s just an autocomplete engine. Someone else will achieve AGI and OpenAI falls apart very quickly when that occurs.
  - HarHarVeryFunny 4y ago
    No, it's not just an autocomplete engine. The underlying neural network architecture is a transformer. It certainly can do "autocomplete" (or riffs on autocomplete), but it can also do a lot more. It doesn't take much thought to realize that being REALLY good at autocomplete means that you need to learn how to do a lot of other things as well. At the end of the day the "predict next word" training goal of LLMs is the ultimate intelligence test. If you could always answer that "correctly" (i.e. intelligently) you'd be a polymath genius. Focusing on the "next word" ("autocomplete") aspect of this, and ignoring the knowledge/intelligence needed to do WELL at it is rather misleading! "The best way to combine quantum mechanics and general relativity into a single theory of everything is ..."
    - goatlover 4y ago
      Wouldn't the ultimate intelligence test involve manipulating the real world? That seems orders of magnitude harder than autocompletion. For a theory of everything, you would probably have to perform some experiments that don't currently exist.
      - HarHarVeryFunny 4y ago
        > Wouldn't the ultimate intelligence test involve manipulating the real world? Perhaps, although intelligence and knowledge are two separate things, so one can display intelligence over a given set of knowledge without knowing other things. Of course intelligence isn't a scalar quantity - to be super-intelligent you want to display intelligence across the widest variety/type of experience and knowledge sets - not just "book smart or street smart", but both and more. Certainly for parity with humans you need to be able to interact with the real world, but I'm not sure it's much different or a whole lot more more complex. Instead of "predict next word" the model/robot would be doing "predict next action", followed by "predict action response". Embeddings are a very powerful type of general-purpose representation - you can embed words, but also perceptions/etc, so I don't think we're very far at all from having similar transformer-based models able to act & perceive - I'd be somewhat surprised if people aren't already experimenting with this.
- unethical_ban 4y ago
  Eh. I think that keeping it permanently closed source when it has such amazing potential for changing society is unethical bordering on evil. However, I think the technology and the datasets will trickle out, perhaps a few years behind, and in time we will have truly open "AI" algos on our home computers/phones.
  - justinclift 4y ago
    So, you reckon they're effectively operating like Google then?
- teekert 4y ago
  Really? I feel like they'll go the way of Docker, but faster: Right now super hot, nice tools/API, great PR. But it's build on open and known foundations, soon GPTs will be commodity and then something easier/better FOSS will arise. It may take some time (2-3 years?) but this scenario seems most likely to me. Edit: Ah didn't get the "reference", perhaps indeed it will be the last of the tech companies ever indeed, at least one started by humans ;).
  - startupsfail 4y ago
    Possible. Coding as we know it might get obsolete. And it is a trillion dollar industry.
    - teekert 4y ago
      I have ChatGPT make up some code every now and then. It’s really nice and when not obscure usually directly useable. By you need to understand what it produces imo. I love that it also explains the code and I can follow code it generates and judge its quality and applicability. Isn’t that important?
      - Filligree 4y ago
        Last year the output was poor. The year before then, GPT essentially couldn't write code at all... You're not wrong about its quality right now, but let's look at the slope as well.
        buddhistdude 4y ago
        on the other hand, gpt-3 was trained on a data set that contains all of the internet already. A big Limitation seems to be that it can only work with problems that it has already seen
        pixl97 4y ago
        I mean a huge amount of code I see is stuff like "get something from API, do this, and pass to API/SQL" so I'm assuming a lot of that could be automated.
- rvz 4y ago
  Complete nonsense. Especially with the "If" It is inevitable that the regulators will get in the way and open source alternatives will also catch up with OpenAI as they have done with GPT-3 and with DALLE-2 and even less than 2 months of ChatGPT, open source alternatives are quickly appearing everywhere.
djyaz1200 4y ago
They are following the Residential Real Estate Developer Playbook, always name your for profit exploitation project after the beautiful organic thing you destroyed to pave the way for it... Examples Whispering pines, Blue Heron Bay, OpenAI
- startupsfail 4y ago
  Hadn’t OpenAI published all the key research results to repro ChatGPT? And made their model available to literally everyone? And contributed more than anyone else to AI alignment/safety? To me it looks like nearly every other player, including open source projects are there for short term fame and profit, while it’s the OpenAI that is playing the long game of AI alignment.
  - deleted 4y ago
    [deleted]
  - Filligree 4y ago
    You're not wrong, but existential threats and possible extinction is in the future; maybe ten-fifteen years away if we're lucky. Meanwhile, we don't get to play with their models right now. Obviously that's what we should be concerned about.
    - Julesman 4y ago
      Among all the accepted threats to humanity's future, AI is one of the least founded at this point. We all grew up with this cautionary fiction. But unless you know something everyone else doesn't, the near term existential threat of AI is relatively low.
      - mitthrowaway2 4y ago
        What strong evidence of existential threat are you expecting to see before it's already too late to avoid catastrophe?

To quote Spaceballs, they're not doing it for money, they're doing it for a shitload of money.

93po 4y ago

OpenAI, if successful, will likely become the most valuable company in the history of the planet, both past and future.

ratorx 4y ago
Really? I see a lot of competition in the space. I think they’d have to become significantly more successful than their competitors (some of which are massive AI powerhouses themselves) to achieve the market dominance necessary.
- TeMPOraL 4y ago
  I think what GP is saying is that success of OpenAI means making a lot of profit, and then triggering the AI apocalypse - which is how they become most valuable company in history both past and future.
  - novaRom 4y ago
    Can you elaborate, what is "the AI apocalypse"? Is it just a symbolic metaphor or is there any scientific research behind this words? For me it's rather more unpredictable toxic environment we observe in the world currently, dominated by purely human-made destructive decisions, often based on purely animal instincts.
    - flangola7 4y ago
      Without AI control loss: Permanent dictatorship by whoever controls AI. With AI control loss: AI converts the atoms of the observable universe to maximally achieve whatever arbitrary goal it thinks it was given. These are natural conclusions that can be drawn from the implied abilities of a superintelligent entity.
      - klibertp 4y ago
        > whatever arbitrary goal it thinks it was given [...] abilities of a superintelligent entity Don't these two phrases contradict each other? Why would a super-intelligent entity need to be given a goal? The old argument that we'll fill the universe with staplers pretty much assumes and requires the entity NOT to be super-intelligent. An AGI would only became one when it gets the ability to formulate its own goals, I think. Not that it helps much if that goal is somehow contrary to the existence of the human race, but if the goal is self-formulated, then there's a sliver of hope that it can be changed. > Permanent dictatorship by whoever controls AI. And that is honestly frightening. We know for sure that there are ways of speaking, writing, or showing things that are more persuasive than others. We've been perfecting the art of convincing others to do what we want them to do for millennia. We got quite good at it, but as with everything human, we lack rigor and consistence in that, even the best speeches are uneven in their persuasive power. An AI trained for transforming simple prompts into a mapping of demographic to what will most likely convince that demographic to do what's prompted doesn't need to be an AGI. It doesn't even need to be much smarter than it already is. Whoever implements it first will most likely first try to convince everyone that all AI is bad (other than their own) and if they succeed, the only way to change the outcome would be a time machine or mental disorders. (Armchair science-fiction reader here, pure speculation without any facts, in case you wondered :))
        flangola7 4y ago
        The point of intelligence is to achieve goals. I don't think Microsoft and others are pouring in billions of dollars without the expectation of telling it to do things. AI can already formulate its own sub-goals, goals that help it achieve its primary goal. We've seen this time and time again in simple reinforcement learning systems over the past two decades. We won't be able to change the primary goal unless we build the AGI so that it permits us because a foreseeable sub-goal is self-preservation. The AGI knows if its programming is changed the primary goal won't be achieved, and thus has incentive to prevent that. AI propaganda will be unmatched but it may not be needed for long. There are already early robots that can carry out real life physical tasks in response to a plain English command like "bring me the bag of chips from the kitchen drawer." Commands of "detain or shoot all resisting citizens" will be possible later.
    - kordlessagain 4y ago
      If the assertion that GPT-3 is a "stochastic parrot" is wrong, there will be an apocalypse because whoever controls an AI that can reason is going to win it all. The opinions that it is or isn't "reasoning" are widely varied and depend heavily on interpretation of the interactions, many of which are hersey. My own testing with OpenAI calls + using Weaviate for storing historical data of exchanges indicates that such a beast appears to have the ability to learn as it goes. I've been able to teach such a system to write valid SQL from plain text feedback and from mistakes it makes by writing errors from the database back into Weaviate (which is then used to modify the prompt next time it runs).
    - YeGoblynQueenne 4y ago
      >> Is it just a symbolic metaphor or is there any scientific research behind this words? Of course there is. See (Cameron, 1984) (https://en.wikipedia.org/wiki/The_Terminator https://en.wikipedia.org/wiki/The_Terminator)
    - LordDragonfang 4y ago
      https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
TeMPOraL 4y ago
Let's hope that they aren't there, given your nicely concealed prediction it'll be the end of the world as we know it.
ejb999 4y ago
seriously doubt it - what they are doing, others can do - and if they start generating a lot of revenue, it will attract competition - lots of it. They don't have a moat big enough that many millions of dollars can't defeat.
- hyperthesis 4y ago
  What if they have an internal ChatGPTzero, training and reprogramming itself, iterating at inhuman speed? A headstart in an exponential is a moat. It surely will have huge blindspots (and people do too), but perhaps it will be good enough for self-improvement... or will be soon.
  - jasonjmcghee 4y ago
    It's a fundamentally different problem. AlphaZero (DeepMind) was able to be trained this way because it was setup with an explicit reward function and end condition. Competitive self-play needs a reward function. It can't just "self-improve towards general intelligence". What's the fitness function of intelligence?
    - pixl97 4y ago
      Actually you are stating 2 different problems at the same time. A lot of intelligence is based around "Don't die, also, have babies". Well AI doesn't have that issue as long as it produces a good enough answer we'll keep the power flowing to it. The bigger issue, and one that is likely far more dangerous, is "Ok, what if it could learn like this". You've created super intelligence with access to huge amounts of global information and is pretty much immortal unless humans decide to pull the plug. The expected alignment of your superintelligent machine is to design a scenario where we cannot unplug the AI without suffering some loss (monetary is always a good one, rich people never like to lose money and will let an AI burn the earth first). The alignment issue on a potential superintelligence is, I don't believe, a solvable problem. Getting people not to be betraying bastards is hard enough at standard intelligence levels, having a thing that could potentially be way smarter and connected to way more data is not going to be controllable in any form or fashion.
berkle4455 4y ago
It’s just an autocomplete engine. Someone else will achieve AGI and OpenAI falls apart very quickly when that occurs.
- HarHarVeryFunny 4y ago
  No, it's not just an autocomplete engine. The underlying neural network architecture is a transformer. It certainly can do "autocomplete" (or riffs on autocomplete), but it can also do a lot more. It doesn't take much thought to realize that being REALLY good at autocomplete means that you need to learn how to do a lot of other things as well. At the end of the day the "predict next word" training goal of LLMs is the ultimate intelligence test. If you could always answer that "correctly" (i.e. intelligently) you'd be a polymath genius. Focusing on the "next word" ("autocomplete") aspect of this, and ignoring the knowledge/intelligence needed to do WELL at it is rather misleading! "The best way to combine quantum mechanics and general relativity into a single theory of everything is ..."
  - goatlover 4y ago
    Wouldn't the ultimate intelligence test involve manipulating the real world? That seems orders of magnitude harder than autocompletion. For a theory of everything, you would probably have to perform some experiments that don't currently exist.
    - HarHarVeryFunny 4y ago
      > Wouldn't the ultimate intelligence test involve manipulating the real world? Perhaps, although intelligence and knowledge are two separate things, so one can display intelligence over a given set of knowledge without knowing other things. Of course intelligence isn't a scalar quantity - to be super-intelligent you want to display intelligence across the widest variety/type of experience and knowledge sets - not just "book smart or street smart", but both and more. Certainly for parity with humans you need to be able to interact with the real world, but I'm not sure it's much different or a whole lot more more complex. Instead of "predict next word" the model/robot would be doing "predict next action", followed by "predict action response". Embeddings are a very powerful type of general-purpose representation - you can embed words, but also perceptions/etc, so I don't think we're very far at all from having similar transformer-based models able to act & perceive - I'd be somewhat surprised if people aren't already experimenting with this.
unethical_ban 4y ago
Eh. I think that keeping it permanently closed source when it has such amazing potential for changing society is unethical bordering on evil. However, I think the technology and the datasets will trickle out, perhaps a few years behind, and in time we will have truly open "AI" algos on our home computers/phones.
- justinclift 4y ago
  So, you reckon they're effectively operating like Google then?
teekert 4y ago
Really? I feel like they'll go the way of Docker, but faster: Right now super hot, nice tools/API, great PR. But it's build on open and known foundations, soon GPTs will be commodity and then something easier/better FOSS will arise. It may take some time (2-3 years?) but this scenario seems most likely to me. Edit: Ah didn't get the "reference", perhaps indeed it will be the last of the tech companies ever indeed, at least one started by humans ;).
- startupsfail 4y ago
  Possible. Coding as we know it might get obsolete. And it is a trillion dollar industry.
  - teekert 4y ago
    I have ChatGPT make up some code every now and then. It’s really nice and when not obscure usually directly useable. By you need to understand what it produces imo. I love that it also explains the code and I can follow code it generates and judge its quality and applicability. Isn’t that important?
    - Filligree 4y ago
      Last year the output was poor. The year before then, GPT essentially couldn't write code at all... You're not wrong about its quality right now, but let's look at the slope as well.
      - buddhistdude 4y ago
        on the other hand, gpt-3 was trained on a data set that contains all of the internet already. A big Limitation seems to be that it can only work with problems that it has already seen
        pixl97 4y ago
        I mean a huge amount of code I see is stuff like "get something from API, do this, and pass to API/SQL" so I'm assuming a lot of that could be automated.
rvz 4y ago
Complete nonsense. Especially with the "If" It is inevitable that the regulators will get in the way and open source alternatives will also catch up with OpenAI as they have done with GPT-3 and with DALLE-2 and even less than 2 months of ChatGPT, open source alternatives are quickly appearing everywhere.

djyaz1200 4y ago

They are following the Residential Real Estate Developer Playbook, always name your for profit exploitation project after the beautiful organic thing you destroyed to pave the way for it... Examples Whispering pines, Blue Heron Bay, OpenAI

startupsfail 4y ago
Hadn’t OpenAI published all the key research results to repro ChatGPT? And made their model available to literally everyone? And contributed more than anyone else to AI alignment/safety? To me it looks like nearly every other player, including open source projects are there for short term fame and profit, while it’s the OpenAI that is playing the long game of AI alignment.
- deleted 4y ago
  [deleted]
- Filligree 4y ago
  You're not wrong, but existential threats and possible extinction is in the future; maybe ten-fifteen years away if we're lucky. Meanwhile, we don't get to play with their models right now. Obviously that's what we should be concerned about.
  - Julesman 4y ago
    Among all the accepted threats to humanity's future, AI is one of the least founded at this point. We all grew up with this cautionary fiction. But unless you know something everyone else doesn't, the near term existential threat of AI is relatively low.
    - mitthrowaway2 4y ago
      What strong evidence of existential threat are you expecting to see before it's already too late to avoid catastrophe?