6 ms·

So many reports like this, it's not a question of working out the kinks. Are we getting close to our very own Stop the Slop campaign?

by rf15 1y ago

randcraw 1y ago
Yeah, after daily working with AI for a decade in a domain where it _does_ work predictably and reliably (image analysis), I continue to be amazed how many of us continue to trust LLM-based text output as being useful. If any human source got their facts wrong this often, we'd surely dismiss them as a counterproductive imbecile. Or elect them President.
- BobbyTables2 1y ago
  HAL 9000 in 2028!
- locallost 1y ago
  I am beginning to wonder why I use it, but the idea of it is so tempting. Try to google it and get stuck because it's difficult to find, or ask and get an instant response. It's not hard to guess which one is more inviting, but it ends up being a huge time sink anyway.
trod1234 1y ago
Regulation with active enforcement is the only civil way. The whole point of regulation is for when the profit motive forces companies towards destructive ends for the majority of society. The companies are legally obligated to seek profit above all else, absent regulation.
- Aurornis 1y ago
  > Regulation with active enforcement is the only civil way. What regulation? What enforcement? These terms are useless without details. Are we going to fine LLM providers every time their output is wrong? That’s the kind of proposition that sounds good as a passing angry comment but obviously has zero chance of becoming a real regulation. Any country who instituted a regulation like that would see all of the LLM advancements and research instantly leave and move to other countries. People who use LLMs would sign up for VPNs and carry on with their lives.
  - trod1234 1y ago
    Regulations exist to override profit motive when corporations are unable to police themselves. Enforcement ensures accountability. Fines don't do much in a fiat money-printing environment. Enforcement is accountability, the kind that stakeholders pay attention to. Something appropriate would be where if AI was used in a safety-critical or life-sustaining environment and harm or loss was caused; those who chose to use it are guilty until they prove they are innocent I think would be sufficient, not just civil but also criminal; where that person and decision must be documented ahead of time. > Any country who instituted a regulation like that would see all of the LLM advances and research instantly leave and move to other countries. This is fallacy. Its a spectrum, research would still occur, it would be tempered by the law and accountability, instead of the wild-west where its much more profitable to destroy everything through chaos. Chaos is quite profitable until it spread systemically and ends everything. AI integration at a point where it can impact the operation of nuclear power plants through interference (perceptual or otherwise) is just asking for a short path to extinction. Its quite reasonable that the needs for national security trump private business making profit in a destructive way.
    - Ukv 1y ago
      > Something appropriate would be where if AI was used in a safety-critical or life-sustaining environment and harm or loss was caused; those who chose to use it are guilty until they prove they are innocent I think would be sufficient, not just civil but also criminal Would this guilty-until-proven-innocent rule apply also to non-ML code and manual decisions? If not, I feel it's kind of arbitrarily deterring certain approaches potentially at the cost of safety ("sure this CNN blows traditional methods out of the water in terms of accuracy, but the legal risk isn't worth it"). In most cases I think it'd make more sense to have fines and incentives for above-average and below-average incident rates (and liability for negligence in the worse cases), then let methods win/fail on their own merit.
      - trod1234 1y ago
        > Would this guilty-until-proven-innocent rule apply also to non-ML code and manual decisions? I would say yes because the person deciding must be the one making the entire decision but there are many examples where someone might be paid to just rubberstamp decisions already made. Letting the person who decided to implement the solution off scot-free. The mere presence of AI (anything based on underlying work of perceptrons) being used accompanied by a loss should prompt a thorough review which corporations currently are incapable of performing for themselves due to lack of consequences/accountability. Lack of disclosure, and the limits of current standing, is another issue that really requires this approach. The problem of fines is that they don't provide the needed incentives to large entities as a result of money-printing through debt-issuance, or indirectly through government contracts. Its also far easier to employ corruption to work around the fine later for these entities as market leaders. We've seen this a number of times in various markets/sectors like JPM and the 10+ year silver price fixing scandal. Merit of subjective rates isn't something that can be enforced, because it is so easily manipulated. Gross negligence already exists and occurs frighteningly common but never makes it to court because proof often requires showing standing to get discovery which isn't generally granted absent a smoking gun or the whim of a judge. Bad things happen certainly where no one is at fault, but most business structure today is given far too much lee-way and have promoted the 3Ds. Its all about: deny, defend, depose.
weatherlite 1y ago
> Are we getting close to our very own Stop the Slop campaign? I don't think so. We read about the handful of failures while there are billions of successful queries every day, in fact I think AI Overviews is sticky and here to stay.
- mepiethree 1y ago
  Are we sure these billions of queries are “successful” for the actual user journey? Maybe this is particular to my circle, but as the only “tech guy” most of my friends and family know, I am regularly asked if I know how to turn off Google AI overviews because many people find them to be garbage
  - gtsop 1y ago
    Why on earth are you accepting his premise that there are billions of successful requests? I just asked chatgpt about query success rate and it replied (part): "...Semantic Errors / Hallucinations On factual queries—especially legal ones—models hallucininate roughly 58–88% of the time A journalism‑focused study found LLM-based search tools (e.g., ChatGPT Search, Perplexity, Grok) were incorrect in 60%+ of news‑related queries Specialized legal AI tools (e.g., Lexis+, Westlaw) still showed error rates between 17% and 34%, despite being domain‑tuned "

So many reports like this, it's not a question of working out the kinks. Are we getting close to our very own Stop the Slop campaign?

by rf15 1y ago

So many reports like this, it's not a question of working out the kinks. Are we getting close to our very own Stop the Slop campaign?

randcraw 1y ago

Yeah, after daily working with AI for a decade in a domain where it _does_ work predictably and reliably (image analysis), I continue to be amazed how many of us continue to trust LLM-based text output as being useful. If any human source got their facts wrong this often, we'd surely dismiss them as a counterproductive imbecile. Or elect them President.

BobbyTables2 1y ago
HAL 9000 in 2028!
locallost 1y ago
I am beginning to wonder why I use it, but the idea of it is so tempting. Try to google it and get stuck because it's difficult to find, or ask and get an instant response. It's not hard to guess which one is more inviting, but it ends up being a huge time sink anyway.

trod1234 1y ago

Regulation with active enforcement is the only civil way. The whole point of regulation is for when the profit motive forces companies towards destructive ends for the majority of society. The companies are legally obligated to seek profit above all else, absent regulation.

Aurornis 1y ago
> Regulation with active enforcement is the only civil way. What regulation? What enforcement? These terms are useless without details. Are we going to fine LLM providers every time their output is wrong? That’s the kind of proposition that sounds good as a passing angry comment but obviously has zero chance of becoming a real regulation. Any country who instituted a regulation like that would see all of the LLM advancements and research instantly leave and move to other countries. People who use LLMs would sign up for VPNs and carry on with their lives.
- trod1234 1y ago
  Regulations exist to override profit motive when corporations are unable to police themselves. Enforcement ensures accountability. Fines don't do much in a fiat money-printing environment. Enforcement is accountability, the kind that stakeholders pay attention to. Something appropriate would be where if AI was used in a safety-critical or life-sustaining environment and harm or loss was caused; those who chose to use it are guilty until they prove they are innocent I think would be sufficient, not just civil but also criminal; where that person and decision must be documented ahead of time. > Any country who instituted a regulation like that would see all of the LLM advances and research instantly leave and move to other countries. This is fallacy. Its a spectrum, research would still occur, it would be tempered by the law and accountability, instead of the wild-west where its much more profitable to destroy everything through chaos. Chaos is quite profitable until it spread systemically and ends everything. AI integration at a point where it can impact the operation of nuclear power plants through interference (perceptual or otherwise) is just asking for a short path to extinction. Its quite reasonable that the needs for national security trump private business making profit in a destructive way.
  - Ukv 1y ago
    > Something appropriate would be where if AI was used in a safety-critical or life-sustaining environment and harm or loss was caused; those who chose to use it are guilty until they prove they are innocent I think would be sufficient, not just civil but also criminal Would this guilty-until-proven-innocent rule apply also to non-ML code and manual decisions? If not, I feel it's kind of arbitrarily deterring certain approaches potentially at the cost of safety ("sure this CNN blows traditional methods out of the water in terms of accuracy, but the legal risk isn't worth it"). In most cases I think it'd make more sense to have fines and incentives for above-average and below-average incident rates (and liability for negligence in the worse cases), then let methods win/fail on their own merit.
    - trod1234 1y ago
      > Would this guilty-until-proven-innocent rule apply also to non-ML code and manual decisions? I would say yes because the person deciding must be the one making the entire decision but there are many examples where someone might be paid to just rubberstamp decisions already made. Letting the person who decided to implement the solution off scot-free. The mere presence of AI (anything based on underlying work of perceptrons) being used accompanied by a loss should prompt a thorough review which corporations currently are incapable of performing for themselves due to lack of consequences/accountability. Lack of disclosure, and the limits of current standing, is another issue that really requires this approach. The problem of fines is that they don't provide the needed incentives to large entities as a result of money-printing through debt-issuance, or indirectly through government contracts. Its also far easier to employ corruption to work around the fine later for these entities as market leaders. We've seen this a number of times in various markets/sectors like JPM and the 10+ year silver price fixing scandal. Merit of subjective rates isn't something that can be enforced, because it is so easily manipulated. Gross negligence already exists and occurs frighteningly common but never makes it to court because proof often requires showing standing to get discovery which isn't generally granted absent a smoking gun or the whim of a judge. Bad things happen certainly where no one is at fault, but most business structure today is given far too much lee-way and have promoted the 3Ds. Its all about: deny, defend, depose.

weatherlite 1y ago

> Are we getting close to our very own Stop the Slop campaign? I don't think so. We read about the handful of failures while there are billions of successful queries every day, in fact I think AI Overviews is sticky and here to stay.

mepiethree 1y ago
Are we sure these billions of queries are “successful” for the actual user journey? Maybe this is particular to my circle, but as the only “tech guy” most of my friends and family know, I am regularly asked if I know how to turn off Google AI overviews because many people find them to be garbage
- gtsop 1y ago
  Why on earth are you accepting his premise that there are billions of successful requests? I just asked chatgpt about query success rate and it replied (part): "...Semantic Errors / Hallucinations On factual queries—especially legal ones—models hallucininate roughly 58–88% of the time A journalism‑focused study found LLM-based search tools (e.g., ChatGPT Search, Perplexity, Grok) were incorrect in 60%+ of news‑related queries Specialized legal AI tools (e.g., Lexis+, Westlaw) still showed error rates between 17% and 34%, despite being domain‑tuned "