4 ms·

Does this mean we’ve reached the next AI winter? This is as good as it gets for quite a long time? Honest question :) perhaps this will postpone everyone’s fear

by chubs 3y ago

Does this mean we’ve reached the next AI winter? This is as good as it gets for quite a long time? Honest question :) perhaps this will postpone everyone’s fears about the singularity…

frozenport 3y ago
No. Winter means people have lost interest in the research. If anything successes in ChatGPT etc will be motivation for continued efforts.
- mkl 3y ago
  Winter means people have lost funding for the research. The ongoing productionising of large language models and multimodal models mean that that probably won't happen for quite a while.
ericabiz 3y ago
Many years ago, there was an image that floated around with Craigslist and all the websites that replaced small parts of it—personals, for sale ads, etc. It turned out the way to beat Craigslist wasn’t to build Yet Another Monolithic Craigslist, but to chunk it off in pieces and be the best at that piece. This is analogous to what’s happening with AI models. Sam Altman is saying we have reached the point where spending $100M+ trying to “beat” GPT-4 at everything isn’t the future. The next step is to chunk off a piece of it and turn it into something a particular industry would pay for. We already see small sprouts of those being launched. I think we will see some truly large companies form with this model in the next 5-10 years. To answer your question, yes, this may be as good as it gets now for monolithic language models. But it is just the beginning of what these models can achieve.
- robocat 3y ago
  https://www.today.com/money/speculation-craigslist-slowly-dying-124007 https://www.today.com/money/speculation-craigslist-slowly-dy... from 2011 - is that what you were thinking of? Strange how few of those logos have survived, and how many new logos would now be on it. It would be interesting to see a modernised version.
- keizo 3y ago
  I had the thought that many of those services broken off can now be recombined. No one likes having to use 100 saas products imho.
- The_Colonel 3y ago
  > Sam Altman is saying we have reached the point where spending $100M+ trying to “beat” GPT-4 at everything isn’t the future. I don't disagree, but it does align pretty well with the OpenAI business model, no? "No need to develop your own base model, just buy our own"
  - swexbe 3y ago
    Isn't this really bad for OpenAI? Essentially, this means Meta/Google/others will catch up to them pretty soon and there is nothing OpenAI can do in the near future to get further ahead.
    - The_Colonel 3y ago
      That's why he's saying they should just buy OpenAI's base model instead of investing the money to catch up.
015a 3y ago
The current stage is now productionizing what we have; finding product fits for it, and making it cheaper. Even GPT-4 isn't necessary to push forward what is possible with AI; if you think about something dumb like "load all of my emails into a language model in real time, give me digests, automatically write responses for ones which classify with characteristics X/Y/Z, allow me to query the model to answer questions, etc": This does not really exist yet, this would be really valuable, and this does not need GPT-4. Another good example is in the coding landscape, which feels closer to existing. Ingest all of a company's code into a model like this, then start thinking about what you can do with it. A chatbot is one thing, the most obvious thing, but there's higher order product use-cases that could be interesting (e.g. you get an error in Sentry, stack trace points Sentry to where the error happened, language model automatically PRs a fix, stuff like that). This shit excites me WAY WAY more than GPT-5. We've unlocked like 0.002% of the value that GPT-3/llama/etc could be capable of delivering. Given the context of broad concern about cost of training, accidentally inventing an AGI, intentionally inventing an AGI; If I were the BDFL of the world, I think we've got at least a decade of latent value just to capture out of GPT-3/4 (and other models). Let's hit pause. Let's actually build on these things. Let's find a level of efficiency that is still valuable without spending $5B in a dick measuring contest [1] to suss out another 50 points on the SAT. Let's work on making edge/local inference more possible. Most of all, let's work on safety, education, and privacy. [1] https://techcrunch.com/2023/04/06/anthropics-5b-4-year-plan-to-take-on-openai/?guccounter=1 https://techcrunch.com/2023/04/06/anthropics-5b-4-year-plan-...

by chubs 3y ago

Does this mean we’ve reached the next AI winter? This is as good as it gets for quite a long time? Honest question :) perhaps this will postpone everyone’s fears about the singularity…

frozenport 3y ago

No. Winter means people have lost interest in the research. If anything successes in ChatGPT etc will be motivation for continued efforts.

mkl 3y ago
Winter means people have lost funding for the research. The ongoing productionising of large language models and multimodal models mean that that probably won't happen for quite a while.

ericabiz 3y ago

Many years ago, there was an image that floated around with Craigslist and all the websites that replaced small parts of it—personals, for sale ads, etc. It turned out the way to beat Craigslist wasn’t to build Yet Another Monolithic Craigslist, but to chunk it off in pieces and be the best at that piece. This is analogous to what’s happening with AI models. Sam Altman is saying we have reached the point where spending $100M+ trying to “beat” GPT-4 at everything isn’t the future. The next step is to chunk off a piece of it and turn it into something a particular industry would pay for. We already see small sprouts of those being launched. I think we will see some truly large companies form with this model in the next 5-10 years. To answer your question, yes, this may be as good as it gets now for monolithic language models. But it is just the beginning of what these models can achieve.

robocat 3y ago
https://www.today.com/money/speculation-craigslist-slowly-dying-124007 https://www.today.com/money/speculation-craigslist-slowly-dy... from 2011 - is that what you were thinking of? Strange how few of those logos have survived, and how many new logos would now be on it. It would be interesting to see a modernised version.
keizo 3y ago
I had the thought that many of those services broken off can now be recombined. No one likes having to use 100 saas products imho.
The_Colonel 3y ago
> Sam Altman is saying we have reached the point where spending $100M+ trying to “beat” GPT-4 at everything isn’t the future. I don't disagree, but it does align pretty well with the OpenAI business model, no? "No need to develop your own base model, just buy our own"
- swexbe 3y ago
  Isn't this really bad for OpenAI? Essentially, this means Meta/Google/others will catch up to them pretty soon and there is nothing OpenAI can do in the near future to get further ahead.
  - The_Colonel 3y ago
    That's why he's saying they should just buy OpenAI's base model instead of investing the money to catch up.

015a 3y ago

The current stage is now productionizing what we have; finding product fits for it, and making it cheaper. Even GPT-4 isn't necessary to push forward what is possible with AI; if you think about something dumb like "load all of my emails into a language model in real time, give me digests, automatically write responses for ones which classify with characteristics X/Y/Z, allow me to query the model to answer questions, etc": This does not really exist yet, this would be really valuable, and this does not need GPT-4. Another good example is in the coding landscape, which feels closer to existing. Ingest all of a company's code into a model like this, then start thinking about what you can do with it. A chatbot is one thing, the most obvious thing, but there's higher order product use-cases that could be interesting (e.g. you get an error in Sentry, stack trace points Sentry to where the error happened, language model automatically PRs a fix, stuff like that). This shit excites me WAY WAY more than GPT-5. We've unlocked like 0.002% of the value that GPT-3/llama/etc could be capable of delivering. Given the context of broad concern about cost of training, accidentally inventing an AGI, intentionally inventing an AGI; If I were the BDFL of the world, I think we've got at least a decade of latent value just to capture out of GPT-3/4 (and other models). Let's hit pause. Let's actually build on these things. Let's find a level of efficiency that is still valuable without spending $5B in a dick measuring contest [1] to suss out another 50 points on the SAT. Let's work on making edge/local inference more possible. Most of all, let's work on safety, education, and privacy. [1] https://techcrunch.com/2023/04/06/anthropics-5b-4-year-plan-to-take-on-openai/?guccounter=1 https://techcrunch.com/2023/04/06/anthropics-5b-4-year-plan-...