4 ms·

I don’t have an account and can’t read the article - but this is obvious to anyone with decent ML experience. Models are good with data they have seen. New un

I don’t have an account and can’t read the article - but this is obvious to anyone with decent ML experience. Models are good with data they have seen. New unseen data is hit or miss. Most laypeople using AI these days could benefit greatly from this one piece of knowledge. If it’s never seen the kind of data you’re throwing at it, the likelihood of error goes up.

jsheard 2y ago
It may be obvious, but it does resurface the question of how these models are supposed to absorb post-2021 information effectively when the open internet from that point forwards is increasingly being filled with low-quality AI slurry. Internet scrapes being a representative sample of human-created media was a one-time-only deal.
- tomrod 2y ago
  This will be an area where smaller, but more focused, models shine.
- firewolf34 2y ago
  If the internet is being filled with AI generative content post-2021, then doesn't that just imply that the next generation of AI training on this "slurry" would be analogous to a "multi-round fixing" operation (as quoted above)? While currently this is a relatively weak strength of genAI - assuming technological improvement of this technique over time, isn't it just as possible that the data quality will converge positively rather than negatively over time, in the future? That is to say, the web would be consistently "refined" as time goes on, by predominant VLLM? Assuming that the internet is even "filled" as you say in the first place (personally I don't think organically-generated content is ever going to be pushed out of the internet, but that's my opinion, and I'll entertain the opposite case for the sake of the discussion). It also assumes that people are using models trained on the current state of internet "slurry" in the first place - that we are continually ingesting more internet YoY into these models. If we come up with a better model that needs less data to produce high-quality content, neither my nor your assertion is even relevant. Same case if the internet just decides to use small, low-quality models trained on only a portion of the internet. But if the internet is continually recycling the entirety of itself through a model that has tens of millions of dollars of funding and research focused on directly improving the quality of it's answer metrics, it's not necessarily 100% locked into a downward quality convergence slide. Especially if we assert that humans /will/ continue to be consistently putting more organic data into the internet over time. It's a pessimistic take.
- ravenstine 2y ago
  Perhaps there will be a day where human beings are paid specifically to train these models rather than by scraping existing corpuses. It would be pretty interesting if many of the same individuals put out of work by LLMs end up being employed to train those same models.
- thejazzman 2y ago
  I agree with you, but then I think about it and.. that's what humans do. Humans are gonna grow based on regurgitated internet stuffs. Maybe forever? With or without AI. We never used to have such an elaborate or accessible thing before. Kids and senior citizens likely engage constantly without even realizing it - and adopt one another's, idk the words, but I'm going for something akin to dialects/accents/language/opinions/etc We're all a bunch of parrots
sdenton4 2y ago
The distinction here is between 'the /kind/ of data you're throwing at it' and 'data you have already thrown at it.' The result shows that the LLM has a hard time generalizing to new (basic!) problems that were not in its training set; this suggests it is memorizing solutions rather than understanding the mechanisms... It's the undergrad who has crammed Leetcode solutions for two weeks before their interview, rather than a student with a deeper understanding who can successfully address new problems. Having a friend who is objectively pretty dumb but has memorized the entire existing literature of your field of study is still pretty damned useful, however. You just need to understand what kinds of questions you can trust them with. Along these lines, my own best use of LLMs is looking up jargon and old results. I see some complicated problem, propose a method for attacking it, and ask my friend who has read every stats paper ever digitized whether they have seen this kind of analysis before, what it's called, why it's a bad idea, and what people do instead. The answers point me to relevant literature by giving me the right jargon and method names to search for using Good-Old-Fashioned-Google.
- qsort 2y ago
  In addition to that -- maybe it's just my tinfoil hat speaking -- I think it calls into question whether benchmark numbers are really that meaningful. "The machine has a 90% success rate when we tell the machine the answer" is a bit weaker than how those numbers were presented.
  - OtherShrezzing 2y ago
    I think it's becoming widely accepted that the current benchmarks aren't especially useful metrics for intelligence - rather, they're useful metrics for measuring how well a system can answer the benchmark questions.
    - sdenton4 2y ago
      Just like the SATs!
- conception 2y ago
exe34 2y ago
given how much of "writing code" is yet another rewrite of something really interesting, I imagine it's still useful to a lot of programmers out there. the ones doing interesting work will probably know what it can and can't do, and those regurgitating decades old solutions in the latest and shiniest framework will probably still benefit from it.

jsheard 2y ago

It may be obvious, but it does resurface the question of how these models are supposed to absorb post-2021 information effectively when the open internet from that point forwards is increasingly being filled with low-quality AI slurry. Internet scrapes being a representative sample of human-created media was a one-time-only deal.

tomrod 2y ago
This will be an area where smaller, but more focused, models shine.
firewolf34 2y ago
If the internet is being filled with AI generative content post-2021, then doesn't that just imply that the next generation of AI training on this "slurry" would be analogous to a "multi-round fixing" operation (as quoted above)? While currently this is a relatively weak strength of genAI - assuming technological improvement of this technique over time, isn't it just as possible that the data quality will converge positively rather than negatively over time, in the future? That is to say, the web would be consistently "refined" as time goes on, by predominant VLLM? Assuming that the internet is even "filled" as you say in the first place (personally I don't think organically-generated content is ever going to be pushed out of the internet, but that's my opinion, and I'll entertain the opposite case for the sake of the discussion). It also assumes that people are using models trained on the current state of internet "slurry" in the first place - that we are continually ingesting more internet YoY into these models. If we come up with a better model that needs less data to produce high-quality content, neither my nor your assertion is even relevant. Same case if the internet just decides to use small, low-quality models trained on only a portion of the internet. But if the internet is continually recycling the entirety of itself through a model that has tens of millions of dollars of funding and research focused on directly improving the quality of it's answer metrics, it's not necessarily 100% locked into a downward quality convergence slide. Especially if we assert that humans /will/ continue to be consistently putting more organic data into the internet over time. It's a pessimistic take.
ravenstine 2y ago
Perhaps there will be a day where human beings are paid specifically to train these models rather than by scraping existing corpuses. It would be pretty interesting if many of the same individuals put out of work by LLMs end up being employed to train those same models.
thejazzman 2y ago
I agree with you, but then I think about it and.. that's what humans do. Humans are gonna grow based on regurgitated internet stuffs. Maybe forever? With or without AI. We never used to have such an elaborate or accessible thing before. Kids and senior citizens likely engage constantly without even realizing it - and adopt one another's, idk the words, but I'm going for something akin to dialects/accents/language/opinions/etc We're all a bunch of parrots

sdenton4 2y ago

The distinction here is between 'the /kind/ of data you're throwing at it' and 'data you have already thrown at it.' The result shows that the LLM has a hard time generalizing to new (basic!) problems that were not in its training set; this suggests it is memorizing solutions rather than understanding the mechanisms... It's the undergrad who has crammed Leetcode solutions for two weeks before their interview, rather than a student with a deeper understanding who can successfully address new problems. Having a friend who is objectively pretty dumb but has memorized the entire existing literature of your field of study is still pretty damned useful, however. You just need to understand what kinds of questions you can trust them with. Along these lines, my own best use of LLMs is looking up jargon and old results. I see some complicated problem, propose a method for attacking it, and ask my friend who has read every stats paper ever digitized whether they have seen this kind of analysis before, what it's called, why it's a bad idea, and what people do instead. The answers point me to relevant literature by giving me the right jargon and method names to search for using Good-Old-Fashioned-Google.

qsort 2y ago
In addition to that -- maybe it's just my tinfoil hat speaking -- I think it calls into question whether benchmark numbers are really that meaningful. "The machine has a 90% success rate when we tell the machine the answer" is a bit weaker than how those numbers were presented.
- OtherShrezzing 2y ago
  I think it's becoming widely accepted that the current benchmarks aren't especially useful metrics for intelligence - rather, they're useful metrics for measuring how well a system can answer the benchmark questions.
  - sdenton4 2y ago
    Just like the SATs!
conception 2y ago

exe34 2y ago

given how much of "writing code" is yet another rewrite of something really interesting, I imagine it's still useful to a lot of programmers out there. the ones doing interesting work will probably know what it can and can't do, and those regurgitating decades old solutions in the latest and shiniest framework will probably still benefit from it.