5 ms·

Whether or not we "need" it, it turns out that in practice the Semantic Web was actually harder than AI and we're getting the latter before the former. This ha

by jerf 3y ago

Whether or not we "need" it, it turns out that in practice the Semantic Web was actually harder than AI and we're getting the latter before the former. This has really always been the disconnect between the semantic web advocates and the skeptics like me. It's not that I don't buy your analysis of the benefits... it's that I do not and never have bought your analysis of the costs. The only question I have is how many orders of magnitude they undersell it by. It's certainly a non-trivial number. What AI will do is basically swamp the semantic web. The problems the world will be dealing with with AI, like the deluge of AI-generated meaningless garbage, training AIs accidentally on other AI-generated garbage, the complete overwhelming of all human voices by AI voices by sheer weight [1], will be so large and pressing that the problems of the semantic web will just get washed away in the flood. And, sadly, the tools of the semantic web will be effectively useless to help; all the signals they are based on are even easier to forge than a human voice in arbitrary text. No help at all. [1] What will it look like to live in a world where for every kilobyte of bespoke humanity I produce, an AI can produce megabytes? What will it look like when someone can afford to use AI to build an entire Reddit, just for me and only me, and isolate me there?

brookst 3y ago
Humans represent a tiny fraction of stock trades, telecom bits sent, hours of video created. If you're happier with a unique personalized reddit, is that worse than preferring video games to movies?
- pixl97 3y ago
  >a tiny fraction of stock trade In which the rich have been getting richer, and asset inflation has been growing ever higher. >If you're happier with a unique personalized reddit The question here is this depends. Simulacra of which Reddit? Of the one that shows you a good world where we try to become better people? Of the Reddit that tells you "They" are out to get you so you should be afraid? Personalized of the 'Real' world, or just made up and trending towards the imaginary? I mean, when I play a video game, I know I'm not playing reality. The line starts breaking down as the level of fidelity of reality is neared.
  - brookst 3y ago
    You're rebutting a moral argument I never made. My point is that humans have never been the sole or even dominant actors in the world, so it's odd to decry AI for somehow creating that reality.
    - pixl97 3y ago
      Humans are not the dominant actors in human society, that is a very strange argument to make.
- jerf 3y ago
  Obviously in context I'm talking about "human content", such as we are creating here. Of course the majority of the traffic on the internet by raw bytes is video, but that was clearly not what I'm talking about. "If you're happier with a unique personalized reddit" There is an element of deception there. If I think I'm interacting with real humans, I want to be interacting with real humans, not bots. And nobody but nobody will just generously spend the money to do that, and then leave me a perfect simulation of a human space. They're going to have some ulterior motive and it will not be in my best interests. I will agree that a shocking-to-me number of people are not bothered by this prospect. Perhaps you are one of them who would not mind. You do you, as the saying goes. I am bothered about being deceived. Truth matters to me and I have no interest in being comprehensively lied to like that.
  - brookst 3y ago
    I think you underestimate people's intelligence. Your hypothetical AI reddit won't deceive anyone because the people using it will be perfectly happy knowing it's all generated, the same way people playing video games are not "deceived" into thinking they are watching a film with actors, and people driving cars are not "deceived" into thinking there's a horse in there somewhere. AI is complicated, but seeing it as mimicry of pre-AI reality is far too narrow to capture the real change (and the real concerns).
    - jart 3y ago
      Generally I agree, although it'll totally happen to people who get shadowbanned, since having an LLM reply to a banned user's invisible comments will make it less likely they'll refresh their VPN and create another account.
    - jerf 3y ago
      I am assuming that someone is setting out to deceive, not advertising it as a fake reddit. We already had that. (It was pretty funny.) I'm also not just talking about someone hooking up the publicly-available and known language models of May 2023 to this. You going to guarantee me that "nobody" will be deceived in even just another year's worth of public progress? Or five? Ten? And "nobody" is a really tall bar... should we at least set the bar at 100IQ or 110, just in the interests of fairness? 2022 discussions of "income inequality" will have nothing on 2032's if a 150 IQ and heavy education becomes necessary to navigate the internet successfully.
shrimp_emoji 3y ago
> was actually harder than AI Than the throw-more-data-at-the-wall-leveraging-ever-growing-parallel-GPU-compute-and-hope-the-black-box-that-comes-out-is-good-enough shortcut to AI* :p
- jerf 3y ago
  I am looking not just at the now, but given the long history of failure of the Semantic Web, the next 10-20 years as well. It is fair in this context. But otherwise, yes, I would agree with you, the LLM-based AIs that we have right this very second don't "do" the Semantic Web. But we're clearly much closer on that front than by asking Facebook to pretty please grease up everything in pristine RDF files, let alone the entire web.
heresie-dabord 3y ago
The disconnect between advocates of a "Semantic Web" and the commoditised reality of the WWW is that the advocates implicitly see an orderly structure of information as being a first principle of knowability/usability and education. The commoditised reality of the WWW is that content is lucrative, but the content need not be of any social or educational value as long as the ice cream cone licks itself. Cyberlarceny is more lucrative than order, usability, and education. And now, algorithmic text generation will provide the full experience of half-baked verbiage refreshed daily. Society does need reliable, true information. But we will not have a semantic Web for economic reasons far beyond the value of education.
- sandworm101 3y ago
  >> Society does need reliable, true information. I believe otherwise. They are perfectly happy with the errors. I see this as wikipedia 2.0. Wikipedia was once derided for being full of errors and holes. Now it is a de facto encyclopedia, the go-to place for the "true" description of most anything. AI will do the same. Eventually its error and subtle biases will become the accepted version of reality. Society needs only actionable information. Accuracy is great, but volume of content wins the audience.
- api 3y ago
  It’s not just the cost, it’s what humans are drawn to. If you sort content by what leads to maximum engagement the stuff that comes out on top is absolutely not informative, intelligent, or positive. You get repetitive addictive filler, inflammatory fear/hate/outrage porn, and lurid tabloid style trash. When you scroll the limbic system appears to be in charge, not the neocortex. The lambic system seeks only dopamine. The easiest path to dopamine wins. This means that on an ad supported web content like the above will swamp everything else. There is zero incentive to produce anything else. AI can now produce maximally engaging content on demand, so the future is Twitter or TikTok but largely without human creators. The future of the open web and free apps is a Skinner box full of AI generated outrage porn and silly filler that scrolls forever, only pausing for ads. The dynamics can be very different when people pay directly for content. The neocortex appears to get more involved in buying decisions than passive consumption. But very very few people pay for content so there is either not much of this or it will have to be expensive. We may be going back to the days when actual knowledge and quality discourse was costly, not because of the cost of printing it but as a filter mechanism to find it in the vast sea of trash. The net was already headed here before AI. Generative models will just accelerate this.
throwaway290 3y ago
The way to handle the deluge [for data that matters] is with human review.
- pjc50 3y ago
  The economics of that are going to get much worse, though, because AI can saturate your humans doing the reviewing.
  - throwaway290 3y ago
    It is always possible to scale reviewers. And if someone is abusing access credentials to propose unvetted machine-generated changes those can be quickly revoked.
    - asherah 3y ago
      "scaling reviewers" implies the money to pay this new amount of reviewers, which I'm not sure the economics of this setup can bear
      - throwaway290 3y ago
        tax AI companies
        pjc50 3y ago
        "Siri, what is something even less feasible than AGI"
        throwaway290 3y ago
        "certainly not anarcho-capitalism!"
quantum_state 3y ago
It’s fun to read your comments.
dgb23 3y ago
Interesting thoughts and concerns. I wonder if AI trained on other AI content inevitably leads to a garbage in garbage out scenario that gets worse over time or whether it actually improves, maybe even at a non linear rate? I believe the former is true, because of financial and political incentives, but the latter isn’t unthinkable.
- jart 3y ago
  When Deepmind built AlphaGo, they trained their AI on other AI-generated content once they ran out of training data, and that was what gave it superhuman abilities. https://en.wikipedia.org/wiki/AlphaGo#Algorithm https://en.wikipedia.org/wiki/AlphaGo#Algorithm
sdwr 3y ago
I've had a nerd boner for semantic web constructs for years. My take is it's more useful than ever in the age of AI. Both for machine synthesis, but also for escaping the flood. Metadata, shared bookmarks, make the internet a place to explore and stake out again. Multiplayer Roam + NFTs
- hirundo 3y ago
  The prohibitively costly part of the semantic web is tagging ... which LLMs can make dramatically cheaper.

by jerf 3y ago

brookst 3y ago

Humans represent a tiny fraction of stock trades, telecom bits sent, hours of video created. If you're happier with a unique personalized reddit, is that worse than preferring video games to movies?

pixl97 3y ago
>a tiny fraction of stock trade In which the rich have been getting richer, and asset inflation has been growing ever higher. >If you're happier with a unique personalized reddit The question here is this depends. Simulacra of which Reddit? Of the one that shows you a good world where we try to become better people? Of the Reddit that tells you "They" are out to get you so you should be afraid? Personalized of the 'Real' world, or just made up and trending towards the imaginary? I mean, when I play a video game, I know I'm not playing reality. The line starts breaking down as the level of fidelity of reality is neared.
- brookst 3y ago
  You're rebutting a moral argument I never made. My point is that humans have never been the sole or even dominant actors in the world, so it's odd to decry AI for somehow creating that reality.
  - pixl97 3y ago
    Humans are not the dominant actors in human society, that is a very strange argument to make.
jerf 3y ago
Obviously in context I'm talking about "human content", such as we are creating here. Of course the majority of the traffic on the internet by raw bytes is video, but that was clearly not what I'm talking about. "If you're happier with a unique personalized reddit" There is an element of deception there. If I think I'm interacting with real humans, I want to be interacting with real humans, not bots. And nobody but nobody will just generously spend the money to do that, and then leave me a perfect simulation of a human space. They're going to have some ulterior motive and it will not be in my best interests. I will agree that a shocking-to-me number of people are not bothered by this prospect. Perhaps you are one of them who would not mind. You do you, as the saying goes. I am bothered about being deceived. Truth matters to me and I have no interest in being comprehensively lied to like that.
- brookst 3y ago
  I think you underestimate people's intelligence. Your hypothetical AI reddit won't deceive anyone because the people using it will be perfectly happy knowing it's all generated, the same way people playing video games are not "deceived" into thinking they are watching a film with actors, and people driving cars are not "deceived" into thinking there's a horse in there somewhere. AI is complicated, but seeing it as mimicry of pre-AI reality is far too narrow to capture the real change (and the real concerns).
  - jart 3y ago
    Generally I agree, although it'll totally happen to people who get shadowbanned, since having an LLM reply to a banned user's invisible comments will make it less likely they'll refresh their VPN and create another account.
  - jerf 3y ago
    I am assuming that someone is setting out to deceive, not advertising it as a fake reddit. We already had that. (It was pretty funny.) I'm also not just talking about someone hooking up the publicly-available and known language models of May 2023 to this. You going to guarantee me that "nobody" will be deceived in even just another year's worth of public progress? Or five? Ten? And "nobody" is a really tall bar... should we at least set the bar at 100IQ or 110, just in the interests of fairness? 2022 discussions of "income inequality" will have nothing on 2032's if a 150 IQ and heavy education becomes necessary to navigate the internet successfully.

shrimp_emoji 3y ago

> was actually harder than AI Than the throw-more-data-at-the-wall-leveraging-ever-growing-parallel-GPU-compute-and-hope-the-black-box-that-comes-out-is-good-enough shortcut to AI* :p

jerf 3y ago
I am looking not just at the now, but given the long history of failure of the Semantic Web, the next 10-20 years as well. It is fair in this context. But otherwise, yes, I would agree with you, the LLM-based AIs that we have right this very second don't "do" the Semantic Web. But we're clearly much closer on that front than by asking Facebook to pretty please grease up everything in pristine RDF files, let alone the entire web.

heresie-dabord 3y ago

The disconnect between advocates of a "Semantic Web" and the commoditised reality of the WWW is that the advocates implicitly see an orderly structure of information as being a first principle of knowability/usability and education. The commoditised reality of the WWW is that content is lucrative, but the content need not be of any social or educational value as long as the ice cream cone licks itself. Cyberlarceny is more lucrative than order, usability, and education. And now, algorithmic text generation will provide the full experience of half-baked verbiage refreshed daily. Society does need reliable, true information. But we will not have a semantic Web for economic reasons far beyond the value of education.

sandworm101 3y ago
>> Society does need reliable, true information. I believe otherwise. They are perfectly happy with the errors. I see this as wikipedia 2.0. Wikipedia was once derided for being full of errors and holes. Now it is a de facto encyclopedia, the go-to place for the "true" description of most anything. AI will do the same. Eventually its error and subtle biases will become the accepted version of reality. Society needs only actionable information. Accuracy is great, but volume of content wins the audience.
api 3y ago
It’s not just the cost, it’s what humans are drawn to. If you sort content by what leads to maximum engagement the stuff that comes out on top is absolutely not informative, intelligent, or positive. You get repetitive addictive filler, inflammatory fear/hate/outrage porn, and lurid tabloid style trash. When you scroll the limbic system appears to be in charge, not the neocortex. The lambic system seeks only dopamine. The easiest path to dopamine wins. This means that on an ad supported web content like the above will swamp everything else. There is zero incentive to produce anything else. AI can now produce maximally engaging content on demand, so the future is Twitter or TikTok but largely without human creators. The future of the open web and free apps is a Skinner box full of AI generated outrage porn and silly filler that scrolls forever, only pausing for ads. The dynamics can be very different when people pay directly for content. The neocortex appears to get more involved in buying decisions than passive consumption. But very very few people pay for content so there is either not much of this or it will have to be expensive. We may be going back to the days when actual knowledge and quality discourse was costly, not because of the cost of printing it but as a filter mechanism to find it in the vast sea of trash. The net was already headed here before AI. Generative models will just accelerate this.

throwaway290 3y ago

The way to handle the deluge [for data that matters] is with human review.

pjc50 3y ago
The economics of that are going to get much worse, though, because AI can saturate your humans doing the reviewing.
- throwaway290 3y ago
  It is always possible to scale reviewers. And if someone is abusing access credentials to propose unvetted machine-generated changes those can be quickly revoked.
  - asherah 3y ago
    "scaling reviewers" implies the money to pay this new amount of reviewers, which I'm not sure the economics of this setup can bear
    - throwaway290 3y ago
      tax AI companies
      - pjc50 3y ago
        "Siri, what is something even less feasible than AGI"
        throwaway290 3y ago
        "certainly not anarcho-capitalism!"

quantum_state 3y ago

It’s fun to read your comments.

dgb23 3y ago

Interesting thoughts and concerns. I wonder if AI trained on other AI content inevitably leads to a garbage in garbage out scenario that gets worse over time or whether it actually improves, maybe even at a non linear rate? I believe the former is true, because of financial and political incentives, but the latter isn’t unthinkable.

jart 3y ago
When Deepmind built AlphaGo, they trained their AI on other AI-generated content once they ran out of training data, and that was what gave it superhuman abilities. https://en.wikipedia.org/wiki/AlphaGo#Algorithm https://en.wikipedia.org/wiki/AlphaGo#Algorithm

sdwr 3y ago

I've had a nerd boner for semantic web constructs for years. My take is it's more useful than ever in the age of AI. Both for machine synthesis, but also for escaping the flood. Metadata, shared bookmarks, make the internet a place to explore and stake out again. Multiplayer Roam + NFTs

hirundo 3y ago
The prohibitively costly part of the semantic web is tagging ... which LLMs can make dramatically cheaper.