5 ms·

This was an intuitively-appealing belief, even with some qualified experimental support, as of a few years ago. However, since then, a bunch of capability brea

• DennisP 1y ago
  AI generates useful stuff, but unless it took a lot of complicated prompting, it's still true that you could "just ask the question yourself." This will change as contexts get longer and people start feeding large stacks of books and papers into their prompts.
  • Swizec 1y ago
    > you could "just ask the question yourself." Just like googling, AIing is a skill. You have to know how to evaluate and judge AI responses. Even how to ask the right questions. Especially asking the right questions is harder than people realize. You see this difference in human managers where some are able to get good results and others aren’t, even when given the same underlying team.
    • multjoy 1y ago
      If you don’t know the answers, how can you judge the machine output?
      • aydyn 1y ago
        A lot of inquiries are like hash functions. Hard to find, easy to verify.
        • multjoy 1y ago
          “Siri, show me an example of overconfidence”
    • DennisP 1y ago
      You have to know how to evaluate and judge articles on the internet, too. Prompt engineering might not be something people do for much longer: https://spectrum.ieee.org/prompt-engineering-is-dead https://spectrum.ieee.org/prompt-engineering-is-dead
  • deleted 1y ago
    [deleted]
  • gojomo 1y ago
    No, new more-capable and/or efficient models have been forged using bulk outputs of other models as training data. These inproved models do some valuable things better & cheaper than the models, or ensembles of models, that generated their training data. So you could not "just ask" the upstream models. The benefits emerge from further bulk training on well-selected synthetic data from the upstream models. Yes, it's counterintuitive! That's why it's worth paying attention to, & describing accurately, rather than remaining stuck repeating obsolete folk misunderstandings.
    • DennisP 1y ago
      That's a process that's internal to companies doing training. It has nothing to do with publishing outputs on the internet.
• wahern 1y ago
  > a bunch of capability breakthroughs from (well-curated) AI generations has definitively disproven it. How much work is "well-curated" doing in that statement?
  • gojomo 1y ago
    Less than you might think! Some of the frontier-advancing training-on-model-outputs ('synthetic data') work just uses other models & automated-checkers to select suitable prompts and desirable subsets of generations. I find it (very) vaguely like how a person can improve at a sport or an instrument without an expert guiding them through every step up, just by drilling certain behaviors in an adequately-proper way. Training on synthetic data somehow seems to extract a similar iterative improvement in certain directions, without requiring any more natural data. It's somehow succeeding in using more compute to refine yet more value from the original non-synthetic-training-data's entropy.
    • Marazan 1y ago
      "adequately-proper way" is doing an incredible amount of heavy lifting in that sentence.
      • gojomo 1y ago
        Yes, but: for humans, even without an expert-over-the-shoulder providing fresh feedback, drilling/practice works – with the right caveats. And, counter to much intuition & forum folklore, it works for AI models, too – with analogous caveats.
• nicbou 1y ago
  How will AI write about a world it never experiences? By training on the work of human beings.
  • gojomo 1y ago
    The training sets can already include direct data series about the world, where the "work of human beings" is just setting up the the collection devices. So models can absolutely "experience the world". But I'm not suggesting they'll advance much, in the near term, without any human-authored training data. I'm just pointing out the cold hard fact that lots of recent breakthroughs came via training on synthetic data - text prompted by, generated by, & selected by other AI models. That practice has now generated a bunch of notable wins in model capabilities – contra the upthread post's sweeping & confident wrongness alleging "Ai generated content is inherently a regression to the mean and harms both training and human utility".
    • andsoitis 1y ago
      > data series about the world, where the "work of human beings" is just setting up the the collection devices. So models can absolutely "experience the world" But not experience it the way humans do. We don’t experience a data series; we experience sensory input in a complicated, nuanced way, modified by prior experiences and emotions, etc. remember that qualia is subjective, with a biological underpinning.
      • gojomo 1y ago
        Perhaps. But these models can already clearly write about the world, in useful ways, without such 'qualia' or 'biological underpinnings'.
        • andsoitis 1y ago
          Sure, and there are many such writings that can be useful. No denying. But the LLM cannot experience like humans do and so will forever be outside our circle. Whether it also remains outside our circle of empathy, or us outside of its, remains to be discovered.
• K0balt 1y ago
  I didn’t mean to imply that -no- ai generated content is useful, only that the vast, vast majority is pollution. The problem is that it is so cheap to produce garbage content with AI that writing actual content is disincentivized, and doing web searches has become an exercise is sifting through AI generated slop. That at least will add extra work to filter usable training data, and costs users minutes a day wading through the refuse.
• K0balt 1y ago
  One example of useful output does not negate the flood of pollution. I’m not denying or downplaying the usefulness of AI. I am doubting the wisdom of blindly publishing -anything- without making at least a trivial attempt to ensure that it is useful and worth publishing. It is a form of pollution. The problem is that it lowers the effort required to produce SEO spam and to “publish” to nearly zero, which creates a perverse incentive to shit on the sidewalk. The amount of AI created, blatantly false blog posts about drug interactions, for example. Not advertising, just banal filler to create site visits, with dangerously false information. It’s not like shitting on the sidewalk was never a problem before, it’s just that shitting on the sidewalk as a service (SOTSAAS) maybe is something we should try to avoid.