6 ms·

They are not leading their competition in terms of technology; they are ahead in a public theater and as the first to market. The claims of "regulate us," "it's

They are not leading their competition in terms of technology; they are ahead in a public theater and as the first to market. The claims of "regulate us," "it's almost AGI and it's dangerous," and "let's put a moratorium on larger models" are all just hype, deceiving the public. They utilize the work of scientists and other companies worldwide without acknowledging their contributions. Certainly, they have done an impressive job, and the scientists and engineers working there deserve respect and recognition. However, they are not significantly ahead of current research. They are not even weeks ahead of research. The current landscape is a game of budgets, winning public opinion, and securing investments.

FeepingCreature 3y ago
No company in the history of companies has ever advertised themselves with the concept "we're going to destroy the world", primarily because it doesn't actually work. OpenAI are true believers, they say that sort of thing because that is what they genuinely think.
- pk-protect-ai 3y ago
  No single neural network currently has the capability to destroy the world, and it will not occur until the architecture for NNs is altered to become self-driving rather than reactive. Even then, it will not be sentient. The deception occurs when the attention shifts from the people operating new powerful technology (tools) to this technology itself.
  - FeepingCreature 3y ago
    Meanwhile, every AI company ever: "We are very excited about agents. We are working hard on agents. We want to roll out agents as soon as possible. Also, persistent memory and online learning would be nice to get." Like, are you following the things that OpenAI and Deepmind are saying at all? The things that make current LLMs not a threat, they aim to tear down as soon as they can arrange. OpenAI just released a video network, and one of their core touted benefits was that you could use it as an action controller! And, um. Do you really think, when the AIs can take a simple prompt and turn it into a ten thousand step plan that requires dynamic skill acquisition, resourcing and persistence, that generating the prompt will be the one single task that stumps them? When we are at that point - and to be clear, every leading AI organisation is sprinting to reach that point earliest - then the difference between doom and safety will be one sentence: "When you are done with that, generate a new prompt." This is not how a world with a long expected lifespan looks. edit: To be clear, I'm still not accusing OpenAI of making up the doom stuff. Even though when I phrase it like that it sounds like they're directly working on things that obviously end the world, which seems contradictory, I don't think they see it like that. To be honest, I can't explain why any doomer works at OpenAI, except in the way that people sometimes move towards explosions and gunfire. I think it's just a bug in the human brain. We want to have the danger in sight.
    - pk-protect-ai 3y ago
      The agents and action controllers are the reactive components in the toolchain, aren't they?
      - FeepingCreature 3y ago
        "Self-driving" is just "reactive" with an open recursive structure. In principle, a network that processes a prompt, generates a plan, recurses a finite number of times, judges how well it did, generates a training plan to improve, outputs a corresponding follow-up prompt, and then waits for you to press a button before it repeats the whole thing with the follow-up prompt, ad infinitum, is still "reactive" - but nobody would argue that whoever presses the button is performing irreplaceable cognitive labor. So I just don't think this captures an important distinction at the limit. If a system can generate a good action plan, turning it into an agent is just plumbing.
        pk-protect-ai 3y ago
        Actually, we can't be certain that humans themselves are not reactive. It is just that their reactions are either built in (self-preservation, reproduction), or based on much broader input (sensory, biochemistry, etc.). The current level of reactivity of the LLMs is very limited by their architectures, though, and as long as these architectures stay in place, you can't expect them to be "self-driven".
        FeepingCreature 3y ago
        I just don't think this is the case. I suspect reactivity in LLMs is mostly limited by training. Human text data is just not suited to the way an AI needs to output data to plan long action chains - justification, not reasoning.
        pk-protect-ai 3y ago
        This might be true as well along with what I said.
dotnet00 3y ago
When you put it that way it kind of reminds me of FTX, they too were all about leading the way on regulation. IIRC the impetus for the entire thing coming crashing down was because the other big exchange's owner thought it was extremely hypocritical how they talked about inviting regulation while operating just like most other crypto firms, completely outside of decent financial controls. Of course not to imply that OpenAI is the same as FTX.
- dudeinjapan 3y ago
  Sam is more trustworthy than Sam.
infecto 3y ago
Yeah I dont see what you’re seeing. Ignoring research, OpenAI is bringing tools to market that nobody else to quite close to yet. Yes, they are getting closer but not as you describe it.
- hn_throwaway_99 3y ago
  Actually, I felt pk-protect-ai's post was spot on. That is, OpenAI does have, currently, the most impressive interfaces and models. But (as was famously commented on by a Google exec) they don't really have any unique moat. That is, while what OpenAI has built may be extraordinary, there is no "secret sauce" there that prevents others from copying them. And this isn't a knock on OpenAI at all. On the contrary, my understanding is that OpenAI was really the first to take the big risk in scaling their GPT models (i.e. spending the hundreds of millions to train their models before they knew what the outcome would be).
  - mupuff1234 3y ago
    > was famously commented on by a Google exec It was just a rank and file IC.
  - infecto 3y ago
    People keep saying this but after 1-2 years, nobody has gotten to their level yet.
    - pk-protect-ai 3y ago
      This isn't true. Benchmarks do not define the usefulness of the models. Mixtral is much more useful for me right now than GPT-4. Look at LLaVA, or the new one, and the very impressive LWM (the text-only version is LLaMA2-based with a 512K context, which does not require TPU to run the inference). The fine-tuned LLaMA 34B is much faster than GPT-4 Turbo and less annoying, providing pretty impressive quality of results.
      - infecto 3y ago
azemetre 3y ago
Calling it public theater is pretty laughable. The amount you get for $25 a month as a consumer is insane. I literally have a 24/7 consultant with surface level understand of any topic in human history. This consultant also happens to be an amazing artist for $25 a month that gives me the rights to commercialize any art piece they make for me. This is extremely hard to beat. This will be extremely hard to beat. As a business they are succeeding.
- caconym_ 3y ago
  You get a "consultant" whose every word has to be verified against an independent second source, and an """artist""" containing a digest of essentially everything actual human artists have ever uploaded to the web, who's somehow still only able to generate derivative slop utterly undifferentiated from what it's spitting out for everybody else. LLMs are useful tools, but if they don't lead to something you can actually somewhat rely on to generate correct/consistent/factual results (see e.g. the recent Air Canada chatbot lawsuit) then the hype is a bust. TBD.
  - mthoms 3y ago
    > You get a "consultant" whose every word has to be verified against an independent second source So, no different from YouTube Videos or [insert any website here]? That’s still a ton of value to capture and repackage. Especially when it’s packaged into an interface that humans find much more natural (ie. “conversation”) than the existing offerings.
    - caconym_ 3y ago
      Hence why I said "LLMs are useful tools". But the current hype pitches them as much, much more than ancillaries to traditional web search, hence why I said "[...] then the hype is a bust. TBD."
      - mthoms 3y ago
        I just don't believe that LLM's have to reach the impossible bar you set of near-perfect correctness in order to be world changing. Thus, I disagree that any outcome less than that should be considered a "bust".

FeepingCreature 3y ago

No company in the history of companies has ever advertised themselves with the concept "we're going to destroy the world", primarily because it doesn't actually work. OpenAI are true believers, they say that sort of thing because that is what they genuinely think.

pk-protect-ai 3y ago
No single neural network currently has the capability to destroy the world, and it will not occur until the architecture for NNs is altered to become self-driving rather than reactive. Even then, it will not be sentient. The deception occurs when the attention shifts from the people operating new powerful technology (tools) to this technology itself.
- FeepingCreature 3y ago
  Meanwhile, every AI company ever: "We are very excited about agents. We are working hard on agents. We want to roll out agents as soon as possible. Also, persistent memory and online learning would be nice to get." Like, are you following the things that OpenAI and Deepmind are saying at all? The things that make current LLMs not a threat, they aim to tear down as soon as they can arrange. OpenAI just released a video network, and one of their core touted benefits was that you could use it as an action controller! And, um. Do you really think, when the AIs can take a simple prompt and turn it into a ten thousand step plan that requires dynamic skill acquisition, resourcing and persistence, that generating the prompt will be the one single task that stumps them? When we are at that point - and to be clear, every leading AI organisation is sprinting to reach that point earliest - then the difference between doom and safety will be one sentence: "When you are done with that, generate a new prompt." This is not how a world with a long expected lifespan looks. edit: To be clear, I'm still not accusing OpenAI of making up the doom stuff. Even though when I phrase it like that it sounds like they're directly working on things that obviously end the world, which seems contradictory, I don't think they see it like that. To be honest, I can't explain why any doomer works at OpenAI, except in the way that people sometimes move towards explosions and gunfire. I think it's just a bug in the human brain. We want to have the danger in sight.
  - pk-protect-ai 3y ago
    The agents and action controllers are the reactive components in the toolchain, aren't they?
    - FeepingCreature 3y ago
      "Self-driving" is just "reactive" with an open recursive structure. In principle, a network that processes a prompt, generates a plan, recurses a finite number of times, judges how well it did, generates a training plan to improve, outputs a corresponding follow-up prompt, and then waits for you to press a button before it repeats the whole thing with the follow-up prompt, ad infinitum, is still "reactive" - but nobody would argue that whoever presses the button is performing irreplaceable cognitive labor. So I just don't think this captures an important distinction at the limit. If a system can generate a good action plan, turning it into an agent is just plumbing.
      - pk-protect-ai 3y ago
        Actually, we can't be certain that humans themselves are not reactive. It is just that their reactions are either built in (self-preservation, reproduction), or based on much broader input (sensory, biochemistry, etc.). The current level of reactivity of the LLMs is very limited by their architectures, though, and as long as these architectures stay in place, you can't expect them to be "self-driven".
        FeepingCreature 3y ago
        I just don't think this is the case. I suspect reactivity in LLMs is mostly limited by training. Human text data is just not suited to the way an AI needs to output data to plan long action chains - justification, not reasoning.
        pk-protect-ai 3y ago
        This might be true as well along with what I said.

dotnet00 3y ago

When you put it that way it kind of reminds me of FTX, they too were all about leading the way on regulation. IIRC the impetus for the entire thing coming crashing down was because the other big exchange's owner thought it was extremely hypocritical how they talked about inviting regulation while operating just like most other crypto firms, completely outside of decent financial controls. Of course not to imply that OpenAI is the same as FTX.

dudeinjapan 3y ago
Sam is more trustworthy than Sam.

infecto 3y ago

Yeah I dont see what you’re seeing. Ignoring research, OpenAI is bringing tools to market that nobody else to quite close to yet. Yes, they are getting closer but not as you describe it.

hn_throwaway_99 3y ago
Actually, I felt pk-protect-ai's post was spot on. That is, OpenAI does have, currently, the most impressive interfaces and models. But (as was famously commented on by a Google exec) they don't really have any unique moat. That is, while what OpenAI has built may be extraordinary, there is no "secret sauce" there that prevents others from copying them. And this isn't a knock on OpenAI at all. On the contrary, my understanding is that OpenAI was really the first to take the big risk in scaling their GPT models (i.e. spending the hundreds of millions to train their models before they knew what the outcome would be).
- mupuff1234 3y ago
  > was famously commented on by a Google exec It was just a rank and file IC.
- infecto 3y ago
  People keep saying this but after 1-2 years, nobody has gotten to their level yet.
  - pk-protect-ai 3y ago
    This isn't true. Benchmarks do not define the usefulness of the models. Mixtral is much more useful for me right now than GPT-4. Look at LLaVA, or the new one, and the very impressive LWM (the text-only version is LLaMA2-based with a 512K context, which does not require TPU to run the inference). The fine-tuned LLaMA 34B is much faster than GPT-4 Turbo and less annoying, providing pretty impressive quality of results.
    - infecto 3y ago

azemetre 3y ago

Calling it public theater is pretty laughable. The amount you get for $25 a month as a consumer is insane. I literally have a 24/7 consultant with surface level understand of any topic in human history. This consultant also happens to be an amazing artist for $25 a month that gives me the rights to commercialize any art piece they make for me. This is extremely hard to beat. This will be extremely hard to beat. As a business they are succeeding.

caconym_ 3y ago
You get a "consultant" whose every word has to be verified against an independent second source, and an """artist""" containing a digest of essentially everything actual human artists have ever uploaded to the web, who's somehow still only able to generate derivative slop utterly undifferentiated from what it's spitting out for everybody else. LLMs are useful tools, but if they don't lead to something you can actually somewhat rely on to generate correct/consistent/factual results (see e.g. the recent Air Canada chatbot lawsuit) then the hype is a bust. TBD.
- mthoms 3y ago
  > You get a "consultant" whose every word has to be verified against an independent second source So, no different from YouTube Videos or [insert any website here]? That’s still a ton of value to capture and repackage. Especially when it’s packaged into an interface that humans find much more natural (ie. “conversation”) than the existing offerings.
  - caconym_ 3y ago
    Hence why I said "LLMs are useful tools". But the current hype pitches them as much, much more than ancillaries to traditional web search, hence why I said "[...] then the hype is a bust. TBD."
    - mthoms 3y ago
      I just don't believe that LLM's have to reach the impossible bar you set of near-perfect correctness in order to be world changing. Thus, I disagree that any outcome less than that should be considered a "bust".