4 ms·

If you think that's crazy, think again. Just yesterday was trying to learn more about Chinese medicine and landed on this page I thoroughly read before noticing

by dudus 3y ago

If you think that's crazy, think again. Just yesterday was trying to learn more about Chinese medicine and landed on this page I thoroughly read before noticing the disclaimer at the top. "The articles on this database are automatically generated by our AI system" https://www.digicomply.com/dietary-supplements-database/panax-notoginseng#:~:text=The%20articles%20on%20this%20database%20are%20automatically%20generated%20by%20our%20AI%20system https://www.digicomply.com/dietary-supplements-database/pana... Is the information on that page correct? I'm not sure but as soon as I noticed it was AI generated I lost all trust. And that's because they bothered to include the warning.

observationist 3y ago
You shouldn't have had any trust to begin with; I don't know why we are so quick to hold up humans as bastions of truth and integrity. This is stereotypical Gell-Mann amnesia - you have to validate information, for yourself, within your own model of the world. You need the tools to be able to verify information that's important to you, whether it's research or knowing which experts or sources are likely to be trustworthy. With AI video and audio on the horizon, you're left with having to determine for yourself whether to trust any given piece of media, and the only thing you'll know for sure is your own experience of events in the real world. That doesn't mean you need to discard all information online as untrustworthy. It just means we're going to need better tools and webs of trust based on repeated good-faith interactions. It's likely I can trust that information posted by individuals on HN will be of a higher quality than the comments section in YouTube or some random newspaper site. I don't need more than a superficial confirmation that information provided here is true - but if it's important, then I will want corroboration from many sources, with validation by an expert extant human. There's no downside in trusting the information you're provided by AI just as much as any piece of information provided by a human, if you're reasonable about it. Right now is as bad as they'll ever be, and all sorts of development is going in to making them more reliable, factual, and verifiable, with appropriately sourced validation. Based on my own knowledge of ginseng and a superficial verification of what that site says, it's more or less as correct as any copy produced by a human copy writer would be. It tracks with wikipedia and numerous other sources. All that said, however, I think the killer app for AI will be e-butlers that interface with content for us, extracting meaningful information, identifying biases, ulterior motives, political and commercial influences, providing background research, and local indexing so that we can offload much of the uncertainty and work required to sift the content we want from the SEO boilerplate garbage pit that is the internet.
- tmpz22 3y ago
  > This is stereotypical Gell-Mann amnesia - you have to validate information, for yourself, within your own model of the world. You need the tools to be able to verify information that's important to you, whether it's research or knowing which experts or sources are likely to be trustworthy. Except anthropologically speaking we still live in trust-based society. We trust water to be available. We trust the grocery stores to be stocked. We trust that our Government institutions are always going to be there. All this to say we have a moral obligation not to let AI spam off the hook as "trust but verify". It is fucked up that people make money abusing innate trust-based mechanism that society depends on to be society.
  - satellite2 3y ago
    And most importantly we trust money to not only be paper or bits
  - observationist 3y ago
    Oh, for sure - I'm not saying don't do anything about it. I'm just saying you should have been treating all information online like this anyway. The lesson from Gell-Mann is that you should bring the same level of skepticism to bear on any source of information that you would on an article where you have expertise and can identify bad information, sloppy thinking, or other significant problems you're particularly qualified to spot. The mistake was ever not using "Trust but verify" as the default mode. AI is just scaling the problem up, but then again, millions of bots online and troll farms aren't exactly new, either. So yes, don't let AI off the hook, but also, if AI is used to good purposes, with repeatable positive results, then don't dismiss something merely because AI is being used. AI being involved in the pipeline isn't a good proxy for quality or authenticity, and AI is only going to get better than it is now.

by dudus 3y ago

observationist 3y ago

You shouldn't have had any trust to begin with; I don't know why we are so quick to hold up humans as bastions of truth and integrity. This is stereotypical Gell-Mann amnesia - you have to validate information, for yourself, within your own model of the world. You need the tools to be able to verify information that's important to you, whether it's research or knowing which experts or sources are likely to be trustworthy. With AI video and audio on the horizon, you're left with having to determine for yourself whether to trust any given piece of media, and the only thing you'll know for sure is your own experience of events in the real world. That doesn't mean you need to discard all information online as untrustworthy. It just means we're going to need better tools and webs of trust based on repeated good-faith interactions. It's likely I can trust that information posted by individuals on HN will be of a higher quality than the comments section in YouTube or some random newspaper site. I don't need more than a superficial confirmation that information provided here is true - but if it's important, then I will want corroboration from many sources, with validation by an expert extant human. There's no downside in trusting the information you're provided by AI just as much as any piece of information provided by a human, if you're reasonable about it. Right now is as bad as they'll ever be, and all sorts of development is going in to making them more reliable, factual, and verifiable, with appropriately sourced validation. Based on my own knowledge of ginseng and a superficial verification of what that site says, it's more or less as correct as any copy produced by a human copy writer would be. It tracks with wikipedia and numerous other sources. All that said, however, I think the killer app for AI will be e-butlers that interface with content for us, extracting meaningful information, identifying biases, ulterior motives, political and commercial influences, providing background research, and local indexing so that we can offload much of the uncertainty and work required to sift the content we want from the SEO boilerplate garbage pit that is the internet.

tmpz22 3y ago
> This is stereotypical Gell-Mann amnesia - you have to validate information, for yourself, within your own model of the world. You need the tools to be able to verify information that's important to you, whether it's research or knowing which experts or sources are likely to be trustworthy. Except anthropologically speaking we still live in trust-based society. We trust water to be available. We trust the grocery stores to be stocked. We trust that our Government institutions are always going to be there. All this to say we have a moral obligation not to let AI spam off the hook as "trust but verify". It is fucked up that people make money abusing innate trust-based mechanism that society depends on to be society.
- satellite2 3y ago
  And most importantly we trust money to not only be paper or bits
- observationist 3y ago
  Oh, for sure - I'm not saying don't do anything about it. I'm just saying you should have been treating all information online like this anyway. The lesson from Gell-Mann is that you should bring the same level of skepticism to bear on any source of information that you would on an article where you have expertise and can identify bad information, sloppy thinking, or other significant problems you're particularly qualified to spot. The mistake was ever not using "Trust but verify" as the default mode. AI is just scaling the problem up, but then again, millions of bots online and troll farms aren't exactly new, either. So yes, don't let AI off the hook, but also, if AI is used to good purposes, with repeatable positive results, then don't dismiss something merely because AI is being used. AI being involved in the pipeline isn't a good proxy for quality or authenticity, and AI is only going to get better than it is now.