8 ms·

AI Appears to Rapidly Be Approaching Brick Wall Where It Can't Get Smarter

60 points by joe_the_user 2y ago

deleted 2y ago
[deleted]
ben_jones 2y ago
My prediction is it’s going to increasingly call traditional software under the hood but credit the AI as innovation.
- rqtwteye 2y ago
  It's similar to us. We can reason about certain things but we often have to look up additional information in a book or ask someone. In the same way I see LLMs being augmented with traditional code. It doesn't really matter if the result is called AI or not.
  - ben_jones 2y ago
    > It doesn't really matter if the result is called AI or not I strongly disagree. Wildly overvalued companies disrupt the economy and present huge risks - see Tesla or Nvidia (currently valued at $100m per employee - $3T market cap / 30k employees)
arnaudsm 2y ago
The AGI party might be cancelled. But the current strengths of LLMs still are summarization, analogy, templating and search. Which can solve a lot of business problems already. My hot take is that UX is the biggest limiter for current LLM products other than chatGPT. Chat has too much friction for many use cases, we need to find better interfaces that are more visual and faster to interact with. Spending $100M to gain .1% on MMLU is a waste of time in comparison.
- schaefer 2y ago
  Pardon me, Asking for what you want (a.k.a: chat) is too much friction?!?
  - mensetmanusman 2y ago
    I need an LLM to chat for me.
  - arnaudsm 2y ago
    Most of the interfaces we use daily require very few clicks, while chat is multiple keystrokes. And the discoverability is the worst part of it.
  - noirbot 2y ago
    It absolutely can be. Prompt construction, as well as learning what sort of things the LLM just isn't good at doing, takes work. I've had multiple times where I've asked for something, gotten complete trash back from the LLM, complained to a friend who's more into LLMs about it, and had them go "Oh, you just have to add this extra set of words at the end/phrase it this other way and it'll work." Or they'll give me some massive multi-paragraph "seed" prompt I need to add in to make it not behave stupidly sometimes, that they evidently have to update every few days or weeks to adapt to some new behavior. Until Google decided to trash their search with their own sub-par LLM, my results were often regularly better just hitting the first link on Google Search like I had for the last decade.
  - fragmede 2y ago
    When you go to a sit-down restaurant, you aren't given a list of the ingredients that they have out back in the kitchen and get asked how you want to assemble them. Someone's done work to take those ingredients and form them into menu items for you to pick from. Given infinite possibilities to enter into a text box is more friction than picking from a menu in front of you. There's a lot of use for LLMs, and it's tossing the ingredients in a restaurant's kitchen together but sometimes people want a menu.
    - 2y ago
mewpmewp2 2y ago
What a weird article with weird statements and conclusions. If GPT-5 or whatever it would be called comes out as a failure, only then you could have a chance at concluding what the title says. Also multimodality, sensor input data, etc besides just text for the same model has a potential to generate a lot more training data. Eventually the model could get the exact same input as we humans get from sight, hearing and other sensors. To me there is still so much left to do. And even if we truly did multiple laps over all the data in the World there are still different new architectures and strategies left to try.
- fieldcny 2y ago
  Ahh the ole if you can sense every particles position and velocity you can predict the future. your comment really belies the desperation that exists now, these models are stuck where they are (hint it’s a natural limit), you are talking about exponentiation of cost to get what a 10% improvement? 5%? They are very few places for which it’s net positive to run them now, and most of those are incredibly shitty things like creating trash marketing content to drown us all in average inanity I really feel bad for this next generation, they will just be constantly inundated with generated crap, so much of the high fidelity of conversation and meaning is and will be lost.
  - mewpmewp2 2y ago
    Desperation? We have had huge advances just from 1.5 years ago with things I wouldn't have thought would be possible in near 10 years. After decades of research with far slower progress and all of sudden we now know that we have hit a wall? And I am not talking about predicting the future, but more predicting the next action to take based on current state, sensor data in a more seamless way. Like a human being reacting to different input, by moving their muscles etc. There would be huge amount of training data from there that could be incorporated into a single model.
    - chron83602 2y ago
      > And I am not talking about predicting the future, but more predicting the next action to take based on current state, sensor data in a more seamless way. Like self-driving cars? Self-driving cars is an engineering problem, let alone an AI problem, and we still cannot solve it despite trillion dollar economic incentives. Just putting together some LLMs on a fuckton of data does not work. Tesla tried that, and failed.
tkgally 2y ago
For an opposing view, see Leopold Aschenbrenner’s recently released “Situational Awareness: The Decade Ahead”: https://situational-awareness.ai https://situational-awareness.ai He did a very long interview with Dwarkesh Patel, too: https://www.youtube.com/watch?v=zdbVtZIn9IM https://www.youtube.com/watch?v=zdbVtZIn9IM
- threeseed 2y ago
  The sooner people like this get demoralised with the lack of progress towards AGI and exit the industry the better. We need calmer, more cautious heads to prevail so AI is more thoughtfully and safely incorporated into products. The whole e/acc, AGI cult is not helping anyone.
  - mpenick 2y ago
    What’s wrong with being excited and working hard towards something? Hoping for nearly anyone’s demoralization is not good. I wish you a spirit of adventure that takes you many exciting places! Also, what specific safety problems do you think need to be solved?
    - threeseed 2y ago
      Because 99% of them aren't working on anything. They are the ones on the sidelines making money selling courses, going on podcasts, getting engagement money from X etc.
      - shermantanktop 2y ago
        There’s a sudden market for a new flavor of bullshit, and brand-new bullshit providers have sprung up to fill the vacuum. But all the exec-webinar/thought-leader/hypetrain/pr stuff is a sideshow from people building actual value.
      - dwaltrip 2y ago
        Going back to the root comment of this thread… Most of the guests on Dwarkesh’s podcast are working full-time in the industry, many of them in extremely senior.
throwaway920102 2y ago
I agree. The idea behind predicting the next token is that you regress to the mean. If humans are mostly idiots, then the mean is idiots. If you filter your data to only include genius humans, then you can train and get an average answer that a genius would write online to a given prompt. But how would you train to generate an order of magnitude of intelligence or insight above a genius if the entire corpus of training data is at most genius-level? You'd have to hope for some sort of emergent intelligence or knowledge-breadth integration based intelligence I suppose. But I already get annoyed by ChatGPT 4o for having Average Redditor Tier ideas and responses.
- mewpmewp2 2y ago
  I think of it differently. The model becomes a simulator, an actor of a sort. If it has enough of good data to build good patterns or reaches a certain intelligence threshold, the content generated by less than average intelligence becomes an acting material to understand those flawed ideas and how to emulate them, which still requires to build and improve empathy. It can be challenging to exactly predict what a "stupid" person would say next, so you need to develop understanding to human psyche in order to solve for that well. And then with a prompt you can tweak, "a reddit tier response", "an expert response", etc. It kind of works already, but the more intelligent it gets the better it should get at acting different roles.
- WillAdams 2y ago
  Moreover, you only get an answer for a question which has already been posed and answered, hence is represented by a pattern which the large language model can arrive at --- it's not possible to get an answer which wasn't already present in the training data.
- 6510 2y ago
  You need a reward system or game. There are lots of professions and areas/topics of debate where one can just make up convincing nonsense. In a competitive area or one with quality feedback there is no need for the blabbering of fuzzy humans. One of my 1000 business plans I will never execute is to build an automated chemistry lab where one can order different ingredients to be subjected to different processes, treatments and measurements. The researcher/customer would have no influence on the input or output and no humans in the building. I'm clueless about the scale to have a positive cost benefit analysis but if it can be good enough to throw things at the wall and see what sticks it seems an AI could be useful to pick the best ways to blow up the lab.
hehdhdjehehegwv 2y ago
Good thing AI has moved so fast in the last decade it will take another 20 years to run out of new applications of what we have today.
deleted 2y ago
[deleted]
malka 2y ago
The next revolution won't be a bigger model imo. It will be a way to run a gpt (at least 3.5) cheap and fast enough for real time text or vocal conversation on our phones or home devices.
- WWWMMMWWW 2y ago
  Everyone will have a personal AI concierge, capable of consuming all their media and then re-displaying it for them with just the bits they want (no ads). The current digital ad industry is doomed.
panarky 2y ago
Dario Amodei, CEO and co-founder of Anthropic, recently said that AI capability has been following an exponential growth curve for the last ten years, a straight line on a log scale, and he hasn't seen any indication that the slope of that line is changing at all.
- RcouF1uZ4gsC 2y ago
  Would you expect him to say anything differently? “it is difficult to get a man to understand something, when his salary (or company valuation) depends on his not understanding it.” - Upton Sinclair (with edits)
  - panarky 2y ago
    I would expect that he's seen capabilities that haven't been released to the public yet, and that those capabilities continue to follow that straight line on a log chart.
- riku_iki 2y ago
  > AI capability has been following an exponential growth curve Math benchmark: Minevra, Jun 2022: 50.3% Opus, Mar 2024: 60.1% And there is a high chance it leaked to Opus training data, since it is old and now popular benchmark.
6510 2y ago
My joke since the 90's was that AI will have to wait for someone dumb enough to figure out how we think. I guess that means it is finally time for me to share my thoughts :P My mind expressed annoyance with how sure LLM's are about everything. This is not a sign of intelligence - quite the opposite! I know people like that. The smartest sounding training data sounds the best I'm sure but intelligence is to be as accurate as possible about how sure you are. You zoom in on the facts, estimate their certainty then prioritize the data that matches the correct level of certainty. Humans hallucinate all the time, we call it imagination, it's great stuff as long as you present it as that.
thr0w 2y ago
A lot of people seem convinced otherwise, but to me GPT-4 is only a minor improvement (in best cases) over 3/3.5. Nowhere near the significant improvement after GPT-2. I'm assuming GPT-5 won't be any sort of breakthrough either. Not that that would be hugely disappointing - all of this has been such a huge improvement in language modeling compared to the junky crap we had before.
throwaway81523 2y ago
One could say that operating on pure foundation models is running out of gas, but there is a lot more to do.
TibbityFlanders 2y ago
[dead]
Havoc 2y ago
That article, evidence and conclusion seems incredibly thin
simonw 2y ago
The idea that LLMs are "running out of training data" is pretty widespread, but does it actually hold up? Has anyone seen representatives of the cutting-edge AI labs - OpenAI, Anthropic, Meta, Mistral etc - express concern about this? The impression I've been getting is that quality turns out to be much more important than quality. This tweet from Andrej Karpathy for example: https://twitter.com/karpathy/status/1797313173449764933 https://twitter.com/karpathy/status/1797313173449764933 > Turns out that LLMs learn a lot better and faster from educational content as well. This is partly because the average Common Crawl article (internet pages) is not of very high value and distracts the training, packing in too much irrelevant information. The average webpage on the internet is so random and terrible it's not even clear how prior LLMs learn anything at all. You'd think it's random articles but it's not, it's weird data dumps, ad spam and SEO, terabytes of stock ticker updates, etc. And then there are diamonds mixed in there, the challenge is pick them out.
- pennomi 2y ago
  I suspect filtering out all the junk will make them smarter and smaller, of course. But then we will take all that saved space and pack it back full of other modalities like audio and images, and the cycle will continue for a long time yet.
- sosodev 2y ago
  Ideally a LLM would have every text in existence in its training set, right? I doubt we’re anywhere near that now.
  - simonw 2y ago
    No, I don't think that would help. Andrej certainly doesn't seem to think so.
  - Chris_Newton 2y ago
    Ideally a LLM would have every text in existence in its training set, right? Only if every text in existence provides good source material to factor into generating the response. Given that half of all texts are even worse than average, this seems unlikely, and Karpathy’s argument seems very reasonable to me. The final paragraph from the original tweet, after the quote in the GP comment, mentions another interesting aspect. Even if we do have sufficient expert-level source material in a particular field to train a useful generative AI model and that in itself produces more desirable responses than training on a larger but more average-quality data set, is there still potentially useful information that could be extracted from the larger training set as well? How can we classify which aspects of a larger training set are desirable to keep while filtering out noise that competes with higher-quality source material? It feels like the progress of generative AI over the next few years might be defined more by these kinds of questions than just trying to build ever larger models using ever larger sets of training data.
atleastoptimal 2y ago
We will never run out of training data. The idea that we will despite billions in investment is absurd. Simply scraping the web is the lowest level and there are millions of other ways to glean information about the world.
EternalFury 2y ago
I am always reminded of Severance, the TV show, when I think of LLMs. Data refinement is what is happening massively right now. Basically, take data harvested on the Internet, which may be of poor quality, and use a teacher-student approach to reformulate it in higher quality. Then use that synthetic data to train new models. It's a bit like the biblical multiplication of the bread loaves. How much farther can it get LLMs? I don't know. During the next turn, though, a lot of the same synthetic goop is going to turn up on the Internet. That's probably why OpenAI is spending the big bucks to secure access to original content producers. Now, here is to hope "original content producers" are not going to depend on LLMs to generate that original content. I think training data would be plentiful if autonomous drones roamed the face of the earth, constantly recording audio and video. There is nothing as good as reality to learn what real is. The representation of reality on the Internet has become a matter of opinion for a lot of people.
- Chris_Newton 2y ago
  There can be all kinds of strange and possibly perverse incentives at work for the “original content producers”, too. Forums like LinkedIn have had regular listings in recent times looking for programmers to help train new models for answering programming questions and generating code. However, the rates offered are nowhere near enough to compete with real programming jobs for skilled developers, at least not in the major Western economies. One wonders what quality level these new models will actually be trained at. Given that these roles are kinda-sorta asking professional programmers to train themselves out of a job, one also wonders whether 100% of those who do contribute will be doing so in good faith. And as you point out yourself, there is also a risk that the “original” content is anything but. Given the scale of the data involved here, and the sheer amount of skilled human input that would be required to refine that data set manually, it seems unlikely that a Mechanical Turk strategy alone will solve many fundamental problems here.
WalterBright 2y ago
One limit of LLMs I'm pretty sure about is they'll never be able to predict chaotic systems, i.e. systems with sensitive dependence on initial conditions. This includes things like the weather and the stock market.
- mhh__ 2y ago
  They're not LLMs to my knowledge but AI weather forecasting will increasingly be a thing. It's true that complex systems often can't be predicted at all e.g. you could design a pinball game such that the (deterministic) wiggling was arbitrarily large such that no floating point format could actually express anything, but AI as a tool for massive dimensionality reduction and pattern extraction does mean that they can make some huge gains in the cases where weather is merely difficult to predict using classical means
  - tim333 2y ago
    Yeah see >GraphCast: AI model for faster and more accurate global weather forecasting https://deepmind.google/discover/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/ https://deepmind.google/discover/blog/graphcast-ai-model-for... I think you inherently don't want a language model for modeling fluid flow. A vector model perhaps.
- deleted 2y ago
  [deleted]
woopsn 2y ago
What is the concern? These businesses didn't exist two years ago. There's nothing to reckon. Google's had a wildly profitable model forever, their "existential threat" is LLMs not petering out.
senectus1 2y ago
At some point they're going to realise that they need to make cluster of LLM AI's all trained in specialities. Then they will pass "thoughts" to each other, back and forth to self check and verify before outputting. It'll be called something like LPM (Large Philosopher Model). It'll end up chewing up more compute and power than the bloody blockchain, but It'll be really impressive and scare the crap out of Turing testers.
salamo 2y ago
What transformers have taught us is that gobbling up the whole internet can get you pretty far, but it doesn't change the fundamentals. Hallucinations still are trivial [1] to elicit. On the other hand, transformers are a software breakthrough that made training on the whole internet even feasible. Other methods (RNNs and LSTMs) were simply too slow. Transformers are a new hammer for an internet-sized nail. If I can make an analogy to chess engines, a type of AI so old that people don't call it AI anymore [2], if we had started out by applying transformers to available chess games, we would never be able to create an engine that could reliably beat a grandmaster. Predicting the next chess move based on some representation of the game and its relationship with previously-seen positions only gets you so far. Deep Mind tried [3] this. One route forward, which doesn't require a fundamental breakthrough, would be synthetic data. To stick with chess, we could create huge synthetic datasets of a huge amount of chess games, and simply continue training as we were. However, this is very expensive, and can be tricky if the synthetic distribution isn't the same as the real one. And the end result is still going to require incredible computation power to use. What an LLM really lacks is the ability to search. And search is really important [4]: without it you'll probably need orders of magnitude more data and orders of magnitude larger models. At a fundamental level, search means we don't just make a snap judgement about e.g. the likely next token, but, as Daniel Kahneman would say, "thinking slowly" about what to do/say next. Assuming we will not have search is how you get projections like needing 100 GW power plants to supply dedicated training data centers. The human brain uses 12 watts. 100 GW is the energy requirements of all human brains on the planet. And yet, if we put all human brains on the planet together, our collective capabilities would be outclassed by a chess engine I can run on my phone. [1] https://arxiv.org/pdf/2406.02061 https://arxiv.org/pdf/2406.02061 [2] https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect [3] https://arxiv.org/pdf/2402.04494 https://arxiv.org/pdf/2402.04494 [4] https://www.youtube.com/watch?v=ceCg90Q9N6Y&t=1200s https://www.youtube.com/watch?v=ceCg90Q9N6Y&t=1200s
leeeeeep101 2y ago
[dead]
tyronehed 2y ago
[dead]
KuriousCat 2y ago
We have barely scratched the surface of the algorithms that can satisfy all the attributes of significant learning. https://www.buffalo.edu/catt/teach/develop/design/learning-outcomes/finks.html https://www.buffalo.edu/catt/teach/develop/design/learning-o... Although I believe, the much more serious issues are around the impact of deploying AI systems on the Human agency. These algorithms could potentially make any kind of change harder and could seriously cap human potential.

60 points by joe_the_user 2y ago

deleted 2y ago

[deleted]

ben_jones 2y ago

My prediction is it’s going to increasingly call traditional software under the hood but credit the AI as innovation.

rqtwteye 2y ago
It's similar to us. We can reason about certain things but we often have to look up additional information in a book or ask someone. In the same way I see LLMs being augmented with traditional code. It doesn't really matter if the result is called AI or not.
- ben_jones 2y ago
  > It doesn't really matter if the result is called AI or not I strongly disagree. Wildly overvalued companies disrupt the economy and present huge risks - see Tesla or Nvidia (currently valued at $100m per employee - $3T market cap / 30k employees)

arnaudsm 2y ago

The AGI party might be cancelled. But the current strengths of LLMs still are summarization, analogy, templating and search. Which can solve a lot of business problems already. My hot take is that UX is the biggest limiter for current LLM products other than chatGPT. Chat has too much friction for many use cases, we need to find better interfaces that are more visual and faster to interact with. Spending $100M to gain .1% on MMLU is a waste of time in comparison.

schaefer 2y ago
Pardon me, Asking for what you want (a.k.a: chat) is too much friction?!?
- mensetmanusman 2y ago
  I need an LLM to chat for me.
- arnaudsm 2y ago
  Most of the interfaces we use daily require very few clicks, while chat is multiple keystrokes. And the discoverability is the worst part of it.
- noirbot 2y ago
  It absolutely can be. Prompt construction, as well as learning what sort of things the LLM just isn't good at doing, takes work. I've had multiple times where I've asked for something, gotten complete trash back from the LLM, complained to a friend who's more into LLMs about it, and had them go "Oh, you just have to add this extra set of words at the end/phrase it this other way and it'll work." Or they'll give me some massive multi-paragraph "seed" prompt I need to add in to make it not behave stupidly sometimes, that they evidently have to update every few days or weeks to adapt to some new behavior. Until Google decided to trash their search with their own sub-par LLM, my results were often regularly better just hitting the first link on Google Search like I had for the last decade.
- fragmede 2y ago
  When you go to a sit-down restaurant, you aren't given a list of the ingredients that they have out back in the kitchen and get asked how you want to assemble them. Someone's done work to take those ingredients and form them into menu items for you to pick from. Given infinite possibilities to enter into a text box is more friction than picking from a menu in front of you. There's a lot of use for LLMs, and it's tossing the ingredients in a restaurant's kitchen together but sometimes people want a menu.
  - 2y ago

mewpmewp2 2y ago

What a weird article with weird statements and conclusions. If GPT-5 or whatever it would be called comes out as a failure, only then you could have a chance at concluding what the title says. Also multimodality, sensor input data, etc besides just text for the same model has a potential to generate a lot more training data. Eventually the model could get the exact same input as we humans get from sight, hearing and other sensors. To me there is still so much left to do. And even if we truly did multiple laps over all the data in the World there are still different new architectures and strategies left to try.

fieldcny 2y ago
Ahh the ole if you can sense every particles position and velocity you can predict the future. your comment really belies the desperation that exists now, these models are stuck where they are (hint it’s a natural limit), you are talking about exponentiation of cost to get what a 10% improvement? 5%? They are very few places for which it’s net positive to run them now, and most of those are incredibly shitty things like creating trash marketing content to drown us all in average inanity I really feel bad for this next generation, they will just be constantly inundated with generated crap, so much of the high fidelity of conversation and meaning is and will be lost.
- mewpmewp2 2y ago
  Desperation? We have had huge advances just from 1.5 years ago with things I wouldn't have thought would be possible in near 10 years. After decades of research with far slower progress and all of sudden we now know that we have hit a wall? And I am not talking about predicting the future, but more predicting the next action to take based on current state, sensor data in a more seamless way. Like a human being reacting to different input, by moving their muscles etc. There would be huge amount of training data from there that could be incorporated into a single model.
  - chron83602 2y ago
    > And I am not talking about predicting the future, but more predicting the next action to take based on current state, sensor data in a more seamless way. Like self-driving cars? Self-driving cars is an engineering problem, let alone an AI problem, and we still cannot solve it despite trillion dollar economic incentives. Just putting together some LLMs on a fuckton of data does not work. Tesla tried that, and failed.

tkgally 2y ago

For an opposing view, see Leopold Aschenbrenner’s recently released “Situational Awareness: The Decade Ahead”: https://situational-awareness.ai https://situational-awareness.ai He did a very long interview with Dwarkesh Patel, too: https://www.youtube.com/watch?v=zdbVtZIn9IM https://www.youtube.com/watch?v=zdbVtZIn9IM

threeseed 2y ago
The sooner people like this get demoralised with the lack of progress towards AGI and exit the industry the better. We need calmer, more cautious heads to prevail so AI is more thoughtfully and safely incorporated into products. The whole e/acc, AGI cult is not helping anyone.
- mpenick 2y ago
  What’s wrong with being excited and working hard towards something? Hoping for nearly anyone’s demoralization is not good. I wish you a spirit of adventure that takes you many exciting places! Also, what specific safety problems do you think need to be solved?
  - threeseed 2y ago
    Because 99% of them aren't working on anything. They are the ones on the sidelines making money selling courses, going on podcasts, getting engagement money from X etc.
    - shermantanktop 2y ago
      There’s a sudden market for a new flavor of bullshit, and brand-new bullshit providers have sprung up to fill the vacuum. But all the exec-webinar/thought-leader/hypetrain/pr stuff is a sideshow from people building actual value.
    - dwaltrip 2y ago
      Going back to the root comment of this thread… Most of the guests on Dwarkesh’s podcast are working full-time in the industry, many of them in extremely senior.

throwaway920102 2y ago

I agree. The idea behind predicting the next token is that you regress to the mean. If humans are mostly idiots, then the mean is idiots. If you filter your data to only include genius humans, then you can train and get an average answer that a genius would write online to a given prompt. But how would you train to generate an order of magnitude of intelligence or insight above a genius if the entire corpus of training data is at most genius-level? You'd have to hope for some sort of emergent intelligence or knowledge-breadth integration based intelligence I suppose. But I already get annoyed by ChatGPT 4o for having Average Redditor Tier ideas and responses.

mewpmewp2 2y ago
I think of it differently. The model becomes a simulator, an actor of a sort. If it has enough of good data to build good patterns or reaches a certain intelligence threshold, the content generated by less than average intelligence becomes an acting material to understand those flawed ideas and how to emulate them, which still requires to build and improve empathy. It can be challenging to exactly predict what a "stupid" person would say next, so you need to develop understanding to human psyche in order to solve for that well. And then with a prompt you can tweak, "a reddit tier response", "an expert response", etc. It kind of works already, but the more intelligent it gets the better it should get at acting different roles.
WillAdams 2y ago
Moreover, you only get an answer for a question which has already been posed and answered, hence is represented by a pattern which the large language model can arrive at --- it's not possible to get an answer which wasn't already present in the training data.
6510 2y ago
You need a reward system or game. There are lots of professions and areas/topics of debate where one can just make up convincing nonsense. In a competitive area or one with quality feedback there is no need for the blabbering of fuzzy humans. One of my 1000 business plans I will never execute is to build an automated chemistry lab where one can order different ingredients to be subjected to different processes, treatments and measurements. The researcher/customer would have no influence on the input or output and no humans in the building. I'm clueless about the scale to have a positive cost benefit analysis but if it can be good enough to throw things at the wall and see what sticks it seems an AI could be useful to pick the best ways to blow up the lab.

hehdhdjehehegwv 2y ago

Good thing AI has moved so fast in the last decade it will take another 20 years to run out of new applications of what we have today.

deleted 2y ago

[deleted]

malka 2y ago

The next revolution won't be a bigger model imo. It will be a way to run a gpt (at least 3.5) cheap and fast enough for real time text or vocal conversation on our phones or home devices.

WWWMMMWWW 2y ago
Everyone will have a personal AI concierge, capable of consuming all their media and then re-displaying it for them with just the bits they want (no ads). The current digital ad industry is doomed.

panarky 2y ago

Dario Amodei, CEO and co-founder of Anthropic, recently said that AI capability has been following an exponential growth curve for the last ten years, a straight line on a log scale, and he hasn't seen any indication that the slope of that line is changing at all.

RcouF1uZ4gsC 2y ago
Would you expect him to say anything differently? “it is difficult to get a man to understand something, when his salary (or company valuation) depends on his not understanding it.” - Upton Sinclair (with edits)
- panarky 2y ago
  I would expect that he's seen capabilities that haven't been released to the public yet, and that those capabilities continue to follow that straight line on a log chart.
riku_iki 2y ago
> AI capability has been following an exponential growth curve Math benchmark: Minevra, Jun 2022: 50.3% Opus, Mar 2024: 60.1% And there is a high chance it leaked to Opus training data, since it is old and now popular benchmark.

6510 2y ago

My joke since the 90's was that AI will have to wait for someone dumb enough to figure out how we think. I guess that means it is finally time for me to share my thoughts :P My mind expressed annoyance with how sure LLM's are about everything. This is not a sign of intelligence - quite the opposite! I know people like that. The smartest sounding training data sounds the best I'm sure but intelligence is to be as accurate as possible about how sure you are. You zoom in on the facts, estimate their certainty then prioritize the data that matches the correct level of certainty. Humans hallucinate all the time, we call it imagination, it's great stuff as long as you present it as that.

thr0w 2y ago

A lot of people seem convinced otherwise, but to me GPT-4 is only a minor improvement (in best cases) over 3/3.5. Nowhere near the significant improvement after GPT-2. I'm assuming GPT-5 won't be any sort of breakthrough either. Not that that would be hugely disappointing - all of this has been such a huge improvement in language modeling compared to the junky crap we had before.

throwaway81523 2y ago

One could say that operating on pure foundation models is running out of gas, but there is a lot more to do.

TibbityFlanders 2y ago

[dead]

Havoc 2y ago

That article, evidence and conclusion seems incredibly thin

simonw 2y ago

The idea that LLMs are "running out of training data" is pretty widespread, but does it actually hold up? Has anyone seen representatives of the cutting-edge AI labs - OpenAI, Anthropic, Meta, Mistral etc - express concern about this? The impression I've been getting is that quality turns out to be much more important than quality. This tweet from Andrej Karpathy for example: https://twitter.com/karpathy/status/1797313173449764933 https://twitter.com/karpathy/status/1797313173449764933 > Turns out that LLMs learn a lot better and faster from educational content as well. This is partly because the average Common Crawl article (internet pages) is not of very high value and distracts the training, packing in too much irrelevant information. The average webpage on the internet is so random and terrible it's not even clear how prior LLMs learn anything at all. You'd think it's random articles but it's not, it's weird data dumps, ad spam and SEO, terabytes of stock ticker updates, etc. And then there are diamonds mixed in there, the challenge is pick them out.

pennomi 2y ago
I suspect filtering out all the junk will make them smarter and smaller, of course. But then we will take all that saved space and pack it back full of other modalities like audio and images, and the cycle will continue for a long time yet.
sosodev 2y ago
Ideally a LLM would have every text in existence in its training set, right? I doubt we’re anywhere near that now.
- simonw 2y ago
  No, I don't think that would help. Andrej certainly doesn't seem to think so.
- Chris_Newton 2y ago
  Ideally a LLM would have every text in existence in its training set, right? Only if every text in existence provides good source material to factor into generating the response. Given that half of all texts are even worse than average, this seems unlikely, and Karpathy’s argument seems very reasonable to me. The final paragraph from the original tweet, after the quote in the GP comment, mentions another interesting aspect. Even if we do have sufficient expert-level source material in a particular field to train a useful generative AI model and that in itself produces more desirable responses than training on a larger but more average-quality data set, is there still potentially useful information that could be extracted from the larger training set as well? How can we classify which aspects of a larger training set are desirable to keep while filtering out noise that competes with higher-quality source material? It feels like the progress of generative AI over the next few years might be defined more by these kinds of questions than just trying to build ever larger models using ever larger sets of training data.

atleastoptimal 2y ago

We will never run out of training data. The idea that we will despite billions in investment is absurd. Simply scraping the web is the lowest level and there are millions of other ways to glean information about the world.

EternalFury 2y ago

I am always reminded of Severance, the TV show, when I think of LLMs. Data refinement is what is happening massively right now. Basically, take data harvested on the Internet, which may be of poor quality, and use a teacher-student approach to reformulate it in higher quality. Then use that synthetic data to train new models. It's a bit like the biblical multiplication of the bread loaves. How much farther can it get LLMs? I don't know. During the next turn, though, a lot of the same synthetic goop is going to turn up on the Internet. That's probably why OpenAI is spending the big bucks to secure access to original content producers. Now, here is to hope "original content producers" are not going to depend on LLMs to generate that original content. I think training data would be plentiful if autonomous drones roamed the face of the earth, constantly recording audio and video. There is nothing as good as reality to learn what real is. The representation of reality on the Internet has become a matter of opinion for a lot of people.

Chris_Newton 2y ago
There can be all kinds of strange and possibly perverse incentives at work for the “original content producers”, too. Forums like LinkedIn have had regular listings in recent times looking for programmers to help train new models for answering programming questions and generating code. However, the rates offered are nowhere near enough to compete with real programming jobs for skilled developers, at least not in the major Western economies. One wonders what quality level these new models will actually be trained at. Given that these roles are kinda-sorta asking professional programmers to train themselves out of a job, one also wonders whether 100% of those who do contribute will be doing so in good faith. And as you point out yourself, there is also a risk that the “original” content is anything but. Given the scale of the data involved here, and the sheer amount of skilled human input that would be required to refine that data set manually, it seems unlikely that a Mechanical Turk strategy alone will solve many fundamental problems here.

WalterBright 2y ago

One limit of LLMs I'm pretty sure about is they'll never be able to predict chaotic systems, i.e. systems with sensitive dependence on initial conditions. This includes things like the weather and the stock market.

mhh__ 2y ago
They're not LLMs to my knowledge but AI weather forecasting will increasingly be a thing. It's true that complex systems often can't be predicted at all e.g. you could design a pinball game such that the (deterministic) wiggling was arbitrarily large such that no floating point format could actually express anything, but AI as a tool for massive dimensionality reduction and pattern extraction does mean that they can make some huge gains in the cases where weather is merely difficult to predict using classical means
- tim333 2y ago
  Yeah see >GraphCast: AI model for faster and more accurate global weather forecasting https://deepmind.google/discover/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/ https://deepmind.google/discover/blog/graphcast-ai-model-for... I think you inherently don't want a language model for modeling fluid flow. A vector model perhaps.
deleted 2y ago
[deleted]

woopsn 2y ago

What is the concern? These businesses didn't exist two years ago. There's nothing to reckon. Google's had a wildly profitable model forever, their "existential threat" is LLMs not petering out.

senectus1 2y ago

At some point they're going to realise that they need to make cluster of LLM AI's all trained in specialities. Then they will pass "thoughts" to each other, back and forth to self check and verify before outputting. It'll be called something like LPM (Large Philosopher Model). It'll end up chewing up more compute and power than the bloody blockchain, but It'll be really impressive and scare the crap out of Turing testers.

salamo 2y ago

What transformers have taught us is that gobbling up the whole internet can get you pretty far, but it doesn't change the fundamentals. Hallucinations still are trivial [1] to elicit. On the other hand, transformers are a software breakthrough that made training on the whole internet even feasible. Other methods (RNNs and LSTMs) were simply too slow. Transformers are a new hammer for an internet-sized nail. If I can make an analogy to chess engines, a type of AI so old that people don't call it AI anymore [2], if we had started out by applying transformers to available chess games, we would never be able to create an engine that could reliably beat a grandmaster. Predicting the next chess move based on some representation of the game and its relationship with previously-seen positions only gets you so far. Deep Mind tried [3] this. One route forward, which doesn't require a fundamental breakthrough, would be synthetic data. To stick with chess, we could create huge synthetic datasets of a huge amount of chess games, and simply continue training as we were. However, this is very expensive, and can be tricky if the synthetic distribution isn't the same as the real one. And the end result is still going to require incredible computation power to use. What an LLM really lacks is the ability to search. And search is really important [4]: without it you'll probably need orders of magnitude more data and orders of magnitude larger models. At a fundamental level, search means we don't just make a snap judgement about e.g. the likely next token, but, as Daniel Kahneman would say, "thinking slowly" about what to do/say next. Assuming we will not have search is how you get projections like needing 100 GW power plants to supply dedicated training data centers. The human brain uses 12 watts. 100 GW is the energy requirements of all human brains on the planet. And yet, if we put all human brains on the planet together, our collective capabilities would be outclassed by a chess engine I can run on my phone. [1] https://arxiv.org/pdf/2406.02061 https://arxiv.org/pdf/2406.02061 [2] https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect [3] https://arxiv.org/pdf/2402.04494 https://arxiv.org/pdf/2402.04494 [4] https://www.youtube.com/watch?v=ceCg90Q9N6Y&t=1200s https://www.youtube.com/watch?v=ceCg90Q9N6Y&t=1200s

leeeeeep101 2y ago

[dead]

tyronehed 2y ago

[dead]

KuriousCat 2y ago

We have barely scratched the surface of the algorithms that can satisfy all the attributes of significant learning. https://www.buffalo.edu/catt/teach/develop/design/learning-outcomes/finks.html https://www.buffalo.edu/catt/teach/develop/design/learning-o... Although I believe, the much more serious issues are around the impact of deploying AI systems on the Human agency. These algorithms could potentially make any kind of change harder and could seriously cap human potential.