5 ms·

> literally the only company that is MILES ahead of its competition Its worth noting that, at least when working in a truly novel and untested industry, whoeve

> literally the only company that is MILES ahead of its competition Its worth noting that, at least when working in a truly novel and untested industry, whoever is leading the pack would likely be the first to know when they tech hits a dead end. By no means am I saying that's actually the case, but there is still a real possibility that LLMs and the underlying architecture don't pan out with regards to the company's goal of developing anything resembling an AGI. If there is a core challenge with the architecture that doesn't scale, OpenAI would just run into the wall when everyone else was still MILES back.

kouru225 3y ago
It doesn’t need to pan out as a way to create AGI though. It just needs to be useful and profitable.
- _heimdall 3y ago
  At least from what I've seen and how I've seen others use LLMs, the general consensus seems to be that they're useful for the basics today but are more of a promising tech than something that's already landed. If OpenAI features were to freeze at what we have today I would be surprised if the company stayed around without a major pivot. Again I'm in no on way saying this is actually the case, only a hypothetical since the tech is still very new and we don't know what we don't know.
  - nl 3y ago
    > If OpenAI features were to freeze at what we have today I would be surprised if the company stayed around without a major pivot. This isn't really true. GPT4 is incredibly useful right now and being used by lots of business processes without any improvements being needed. Every incremental improvement (eg Gemini 1.5 huge context window) opens up even more possibilities. AGI isn't required at all.
    - lmeyerov 3y ago
      Very much this. It's incredible what we can do with it - our limiter building louie.ai has been more ourselves than OpenAI. Though of course a better LLM (smarter, faster, cheaper) does enable more. The main problem with freezing is the moat disappears. Others are steadily catching up, and on specific benchmarks, even surpassing. Groq.com is insane wrt perf.
      - EVa5I7bHFq9mnYK 3y ago
        Groq's competition is nvidia, not open.ai OpenAI can run their models on groq machines as well.
        lmeyerov 3y ago
        groq has been public on launching an api, so running groq+mixtral vs openai.com (vs others) is a real topic, esp. when ~half of openai's revenue is their api
    - renonce 3y ago
      But it’s costly. It would be even more useful if it were 1000 times cheaper than it is now. And we don’t yet know if OpenAI earns more than it spent on renting cards or not
      - nl 3y ago
        Sure. Agree with that. But we can be very confident in the price of compute decreasing over time. (Although we can make inform estimates on OpenAIs cost structure for serving models well enough to guess they are probably breaking even on it, or pretty close to it. Eg listen to the gradient dissent interview with the Together AI founders and it’s clear they are doing the same, and unless you think OpenAI is remarkably worse at serving technology it implies they are in the breaking even ballpark) And we still don’t need AGI for it to be a great business.
hn_throwaway_99 3y ago
> but there is still a real possibility that LLMs and the underlying architecture don't pan out with regards to the company's goal of developing anything resembling an AGI. There is, but in one sense, who cares? The technology is already super useful, even if it ends up not being the end all and be all of AGI. That is, there are really only 2 options: 1. LLMs are an important stepping stone on the way to AGI, in which case OpenAI is in a great position as the company with the best LLM. 2. LLMs turn out to be a "local maximum" in the search for AGI. I think that even if that actually is the case, OpenAI is still in a great position with much of the infrastructure, data, expertise, etc. even if they require totally new model approaches. Also, I wouldn't worry so much about #2 because if we do ever get AGI, our economy would surely collapse, or else it would look completely unrecognizable to our current economic systems. That is, I'm always struck that when people talk about AGI fears they talk about things like Skynet and misinformation, but nearly our entire society is organized around people selling their labor for a price. I don't know what our economy will look like when/if AGI becomes real, but I do know it will change drastically if it means that the vast majority of people won't be able to price their labor more than $0.
- _heimdall 3y ago
  > LLMs are an important stepping stone on the way to AGI, in which case OpenAI is in a great position as the company with the best LLM We don't actually know this though. Assuming an AGI hasn't yet been developed, we don't know whether LLMs will actually get us there. We know they seem to have more use than previous ML systems, but until we have an AGI we can't say what will get us there. Further, are we really assuming that developing AGI is either a shared goal or a given regardless of what people would actually want to happen? It sounds like we agree on the fundamental impacts an AGI would have on our current societal structures, do we as a society not get a say in the change? Have we effectively blessed a handful of people working in the private sector to make that decision for everyone? And if so, when do we grapple with the moral questions, like whether an AGI has rights similar to humans, or if unplugging one is murder, etc?
  - hn_throwaway_99 3y ago
    I think you misunderstood my comment. I gave those two bullet points as mutually exclusive options, where one of them must, pretty much by definition, be true. That is, you responded "We don't actually know this though. Assuming an AGI hasn't yet been developed, we don't know whether LLMs will actually get us there." Exactly, we don't know if this is true, in which case if it's false my second bullet point "LLMs turn out to be a 'local maximum' in the search for AGI." is the true statement.
azemetre 3y ago
What do you mean by tech dead end? Has Oracle reached a tech dead in? Are they still a successful business? What about SalesForce, did they hit their tech brick wall yet? Are they still a successful business? I mean we all work in an industry where we literally see inferior tech win out all the time, not just in usage but in money made.
- _heimdall 3y ago
  OpenAI's stated goal is to develop an AGI. My point was just that until they get there we'll have no idea whether LLM tech will lead to an AGI. The company could always pivot as they learn the limitations of LLMs and potentially find other options, though I would argue that their current valuation is extremely high and based largely on the promise of tech they really haven't developed yet. If roadblocks are hit they could be in a tight spot having to live up to such high expectations. That said, as much as I'd like to see the business goal of developing an AGI crash and burn, I wouldn't bet against them. I just hope we somehow solve the seemingly unsolvable alignment problem first.

kouru225 3y ago

It doesn’t need to pan out as a way to create AGI though. It just needs to be useful and profitable.

_heimdall 3y ago
At least from what I've seen and how I've seen others use LLMs, the general consensus seems to be that they're useful for the basics today but are more of a promising tech than something that's already landed. If OpenAI features were to freeze at what we have today I would be surprised if the company stayed around without a major pivot. Again I'm in no on way saying this is actually the case, only a hypothetical since the tech is still very new and we don't know what we don't know.
- nl 3y ago
  > If OpenAI features were to freeze at what we have today I would be surprised if the company stayed around without a major pivot. This isn't really true. GPT4 is incredibly useful right now and being used by lots of business processes without any improvements being needed. Every incremental improvement (eg Gemini 1.5 huge context window) opens up even more possibilities. AGI isn't required at all.
  - lmeyerov 3y ago
    Very much this. It's incredible what we can do with it - our limiter building louie.ai has been more ourselves than OpenAI. Though of course a better LLM (smarter, faster, cheaper) does enable more. The main problem with freezing is the moat disappears. Others are steadily catching up, and on specific benchmarks, even surpassing. Groq.com is insane wrt perf.
    - EVa5I7bHFq9mnYK 3y ago
      Groq's competition is nvidia, not open.ai OpenAI can run their models on groq machines as well.
      - lmeyerov 3y ago
        groq has been public on launching an api, so running groq+mixtral vs openai.com (vs others) is a real topic, esp. when ~half of openai's revenue is their api
  - renonce 3y ago
    But it’s costly. It would be even more useful if it were 1000 times cheaper than it is now. And we don’t yet know if OpenAI earns more than it spent on renting cards or not
    - nl 3y ago
      Sure. Agree with that. But we can be very confident in the price of compute decreasing over time. (Although we can make inform estimates on OpenAIs cost structure for serving models well enough to guess they are probably breaking even on it, or pretty close to it. Eg listen to the gradient dissent interview with the Together AI founders and it’s clear they are doing the same, and unless you think OpenAI is remarkably worse at serving technology it implies they are in the breaking even ballpark) And we still don’t need AGI for it to be a great business.

hn_throwaway_99 3y ago

> but there is still a real possibility that LLMs and the underlying architecture don't pan out with regards to the company's goal of developing anything resembling an AGI. There is, but in one sense, who cares? The technology is already super useful, even if it ends up not being the end all and be all of AGI. That is, there are really only 2 options: 1. LLMs are an important stepping stone on the way to AGI, in which case OpenAI is in a great position as the company with the best LLM. 2. LLMs turn out to be a "local maximum" in the search for AGI. I think that even if that actually is the case, OpenAI is still in a great position with much of the infrastructure, data, expertise, etc. even if they require totally new model approaches. Also, I wouldn't worry so much about #2 because if we do ever get AGI, our economy would surely collapse, or else it would look completely unrecognizable to our current economic systems. That is, I'm always struck that when people talk about AGI fears they talk about things like Skynet and misinformation, but nearly our entire society is organized around people selling their labor for a price. I don't know what our economy will look like when/if AGI becomes real, but I do know it will change drastically if it means that the vast majority of people won't be able to price their labor more than $0.

_heimdall 3y ago
> LLMs are an important stepping stone on the way to AGI, in which case OpenAI is in a great position as the company with the best LLM We don't actually know this though. Assuming an AGI hasn't yet been developed, we don't know whether LLMs will actually get us there. We know they seem to have more use than previous ML systems, but until we have an AGI we can't say what will get us there. Further, are we really assuming that developing AGI is either a shared goal or a given regardless of what people would actually want to happen? It sounds like we agree on the fundamental impacts an AGI would have on our current societal structures, do we as a society not get a say in the change? Have we effectively blessed a handful of people working in the private sector to make that decision for everyone? And if so, when do we grapple with the moral questions, like whether an AGI has rights similar to humans, or if unplugging one is murder, etc?
- hn_throwaway_99 3y ago
  I think you misunderstood my comment. I gave those two bullet points as mutually exclusive options, where one of them must, pretty much by definition, be true. That is, you responded "We don't actually know this though. Assuming an AGI hasn't yet been developed, we don't know whether LLMs will actually get us there." Exactly, we don't know if this is true, in which case if it's false my second bullet point "LLMs turn out to be a 'local maximum' in the search for AGI." is the true statement.

azemetre 3y ago

What do you mean by tech dead end? Has Oracle reached a tech dead in? Are they still a successful business? What about SalesForce, did they hit their tech brick wall yet? Are they still a successful business? I mean we all work in an industry where we literally see inferior tech win out all the time, not just in usage but in money made.

_heimdall 3y ago
OpenAI's stated goal is to develop an AGI. My point was just that until they get there we'll have no idea whether LLM tech will lead to an AGI. The company could always pivot as they learn the limitations of LLMs and potentially find other options, though I would argue that their current valuation is extremely high and based largely on the promise of tech they really haven't developed yet. If roadblocks are hit they could be in a tight spot having to live up to such high expectations. That said, as much as I'd like to see the business goal of developing an AGI crash and burn, I wouldn't bet against them. I just hope we somehow solve the seemingly unsolvable alignment problem first.