9 ms·

I'm curious, why would there be doubts? GPT-4 is incredibly useful, and it's clear there's so much straightforward bits of additional product development on to

I'm curious, why would there be doubts? GPT-4 is incredibly useful, and it's clear there's so much straightforward bits of additional product development on top of it in various verticals that will be incredible productivity multipliers in the respective verticals.

_hypx 3y ago
There's a real danger that people will just run LLMs on their local GPUs. It's not a guarantee that an online service is a profitable idea.
- fafqg 3y ago
  You could say the same about many SaaS projects. Why pay for an expensive GPU upfront and then some guy who can install it, configure it, create some sort of interface for you to talk to it... when you can just pay openai to do it for less money?
  - awesomeMilou 3y ago
    Because GPU-Servers that can run a typical LLM are less than 5k, which includes installation. Really, running an LLM seems to be no more complicated than running a NAS from a system administrators perspective. Emphasis on running, not training.
    - MichaelZuo 3y ago
      5k can't even get you 80 GB of VRAM on a GPU. How is that possible?
- nullsense 3y ago
  Going to need some insanely beefy GPU cluster at your house to run anything close to GPT-4. The best I can run is either LLaMa 65B at 1 token per second (too slow) on my CPU or LLaMa 30B on my GPU at quite a fast ~30 tokens a second. Nowhere near the usefulness of GPT-4.
  - _hypx 3y ago
    GPUs in the future will get more powerful. Especially if they are designed with LLMs in mind.
    - nullsense 3y ago
      I wonder where a critical threshold might lie? For example, right now if you only have 24Gb of VRAM, the best you can run locally at usable speeds are the 4-bit quantized LLaMa 30B models, which are only semi-usable. If you have 48Gb however, you can run the 4-bit quantized LLaMa 65B, which is much better. I assume there will be advances that basically make the context window practically infinite. That ought to make the lower parameter count models much more powerful on its own. I also assume they will become more efficient to run through sparsity/pruning, though I'm a total novice on this topic. I wonder how many generations of hardware are we talking? One? Two? Or three? It feels like it's potentially within that range.
pharmakom 3y ago
I expect there to be GPT-4 level model (or close enough) that anyone can run in the major cloud providers soon. In this future, most AI powered products won’t be paying OpenAI anything.
- MichaelRazum 3y ago
  It sounds like it is trivial to replicate GPT-4 performance. I Would be surprised if that is the case.
  - TaylorAlexander 3y ago
    Not trivial, but not beyond the realm of possibility either.
  - jerpint 3y ago
    Cost to use openAI is also pretty low compared to hosting models
    - pharmakom 3y ago
      But they are losing money.
- two_in_one 3y ago
  Google still catching up for more than a month. Amazon still doesn't have anything comparable, but definitely needs and wants it. So it doesn't looks like it's easy to replicate. Besides, it's not a finished product. It's a service in continuous development. For how long OpenAI will be able to stay ahead of competitors is yet to be seen. They are trying to create a secondary market of resellers. This may keep them in business for long time if they are just "good enough".
Ekaros 3y ago
But do they have moat? Or could the existing big tech companies enter the same market and offer the similar enough product? It is not like Amazon, Google, Microsoft and maybe even Facebook don't have enough money to throw at it and sell something.
drumhead 3y ago
Advertising? Who's going to click on advertising that doesnt do it now? Subscription? Its useful but how many people will pay a sub for it, its not netflix or porn. Licensing maybe? Incorporating it into other products but the competion is going to be intense so prices will drop or they'll have to be the best. This isnt going to be the moneymaker people think it will be..
- davidatbu 3y ago
  > Subscription? How many people will pay a sub for it? I'm a programmer, and I pay a subscription for GPT-4 because it's ridiculously worth it. I'm curious, what do you do that you don't think GPT-4 is worth it?
  - drumhead 3y ago
    It might be worth it for you, but how many people like you are there in the market? Not many I suspect. Most wont be able to afford it or will go for free options.
    - davidatbu 3y ago
      Since we're both not providing data, I'd be interested in hearing your personal experience on this (if it's not too much of a bother, feel free not to respond!): What do you do for a living, and do you find GPT-4 helpful for it?
- two_in_one 3y ago
  I suspect as soon as top management realizes how useful it can be on all levels there will me mass subscriptions from companies. Of course OpenAI needs to convince it's safe and secure. New dark mode will help.

_hypx 3y ago

There's a real danger that people will just run LLMs on their local GPUs. It's not a guarantee that an online service is a profitable idea.

fafqg 3y ago
You could say the same about many SaaS projects. Why pay for an expensive GPU upfront and then some guy who can install it, configure it, create some sort of interface for you to talk to it... when you can just pay openai to do it for less money?
- awesomeMilou 3y ago
  Because GPU-Servers that can run a typical LLM are less than 5k, which includes installation. Really, running an LLM seems to be no more complicated than running a NAS from a system administrators perspective. Emphasis on running, not training.
  - MichaelZuo 3y ago
    5k can't even get you 80 GB of VRAM on a GPU. How is that possible?
nullsense 3y ago
Going to need some insanely beefy GPU cluster at your house to run anything close to GPT-4. The best I can run is either LLaMa 65B at 1 token per second (too slow) on my CPU or LLaMa 30B on my GPU at quite a fast ~30 tokens a second. Nowhere near the usefulness of GPT-4.
- _hypx 3y ago
  GPUs in the future will get more powerful. Especially if they are designed with LLMs in mind.
  - nullsense 3y ago
    I wonder where a critical threshold might lie? For example, right now if you only have 24Gb of VRAM, the best you can run locally at usable speeds are the 4-bit quantized LLaMa 30B models, which are only semi-usable. If you have 48Gb however, you can run the 4-bit quantized LLaMa 65B, which is much better. I assume there will be advances that basically make the context window practically infinite. That ought to make the lower parameter count models much more powerful on its own. I also assume they will become more efficient to run through sparsity/pruning, though I'm a total novice on this topic. I wonder how many generations of hardware are we talking? One? Two? Or three? It feels like it's potentially within that range.

pharmakom 3y ago

I expect there to be GPT-4 level model (or close enough) that anyone can run in the major cloud providers soon. In this future, most AI powered products won’t be paying OpenAI anything.

MichaelRazum 3y ago
It sounds like it is trivial to replicate GPT-4 performance. I Would be surprised if that is the case.
- TaylorAlexander 3y ago
  Not trivial, but not beyond the realm of possibility either.
- jerpint 3y ago
  Cost to use openAI is also pretty low compared to hosting models
  - pharmakom 3y ago
    But they are losing money.
two_in_one 3y ago
Google still catching up for more than a month. Amazon still doesn't have anything comparable, but definitely needs and wants it. So it doesn't looks like it's easy to replicate. Besides, it's not a finished product. It's a service in continuous development. For how long OpenAI will be able to stay ahead of competitors is yet to be seen. They are trying to create a secondary market of resellers. This may keep them in business for long time if they are just "good enough".

Ekaros 3y ago

But do they have moat? Or could the existing big tech companies enter the same market and offer the similar enough product? It is not like Amazon, Google, Microsoft and maybe even Facebook don't have enough money to throw at it and sell something.

drumhead 3y ago

Advertising? Who's going to click on advertising that doesnt do it now? Subscription? Its useful but how many people will pay a sub for it, its not netflix or porn. Licensing maybe? Incorporating it into other products but the competion is going to be intense so prices will drop or they'll have to be the best. This isnt going to be the moneymaker people think it will be..

davidatbu 3y ago
> Subscription? How many people will pay a sub for it? I'm a programmer, and I pay a subscription for GPT-4 because it's ridiculously worth it. I'm curious, what do you do that you don't think GPT-4 is worth it?
- drumhead 3y ago
  It might be worth it for you, but how many people like you are there in the market? Not many I suspect. Most wont be able to afford it or will go for free options.
  - davidatbu 3y ago
    Since we're both not providing data, I'd be interested in hearing your personal experience on this (if it's not too much of a bother, feel free not to respond!): What do you do for a living, and do you find GPT-4 helpful for it?
two_in_one 3y ago
I suspect as soon as top management realizes how useful it can be on all levels there will me mass subscriptions from companies. Of course OpenAI needs to convince it's safe and secure. New dark mode will help.