3 ms·

Maybe? The semantic web is useful because it makes us catalog our thoughts and language in a machine readable way. The key thing is the cost of running these se

Maybe? The semantic web is useful because it makes us catalog our thoughts and language in a machine readable way. The key thing is the cost of running these semantic web machines is smaller than running a GPU powered language model. Also, didn't these models learn from the semantic web? No idea if that matters, but still.

chrismorgan 3y ago
Another hazard of the AI approach is that it’s probabilistic. Presuming the semantic information and implementation are all correct (which I will admit is not a given), they’ll always be correct, whereas the AI approach will sometimes give wrong answers.
- brookst 3y ago
  As you note, semantic data created by humans is also probabilistic and can be wrong. Ditto with semantic data created by automated systems that were created by humans. There will always be incorrect data in the world. AI doesn't change that, but so what?
  - chrismorgan 3y ago
    There’s a difference, though: if the erroneous data comes from human input error, it can easily be corrected, and if the error is due to a software bug, it can be diagnosed and corrected; but errors that come from LLMs (the main type of AI in question here, and indicative enough of AIs in general, as it stands, though probably worse) are basically unfixable (that is: we don’t know how it would even be possible to fix them in general, so all you can do is patch things here and there, which will only probably fix the problems, and will increase the frequency of other problems).
    - brookst 3y ago
      How is an error from an LLM less correctable than one from a human? If "Paris is the capital of Germany" makes it into a news article, how does the origin of the error impact its correctability? Reinforcement learning seems pretty analogous to how we correct people who are wrong about something. I'm really not seeing a category difference here?
      - chrismorgan 3y ago
        With a human, if you have an employee, you can instruct them to correct an error, and if they refuse to or just don’t, you can fire them—you have recourse. LLMs are complex like humans, but you lack the ability to make it do what you want, or to have recourse. If the model has gained the impression that Paris is the capital of Germany (a more concrete and fixable error than most, frankly), this is woven into its parameters throughout, and it’s really hard to fix, and for LLMs we basically have no idea how to actually truly do so. With such a simple error as “Paris is the capital of Germany”, we do know how to fine-tune the model to fairly effectively mask that error, but the important errors are often closer to what you might consider procedural than factual—rather than its “memory” containing a wrong fact, it’s “thinking” in the wrong way. But all up, I’d call “lack of recourse” the most important difference.
- pixl97 3y ago
  Based on the failure of the semantic web as of so far, I think the statement 'not a given' is a given in itself. Humans are horrible at giving systems correct information.
- grumpymouse 3y ago
  LLMs always remind me of pigeons you see in the city that have spent their whole life eating out of bins and pecking dropped chunks of kebab. We trained it on a bunch of stuff, but we're not sure quite what. Looks like it works OK, so let's get it on a plate!
pjc50 3y ago
> The semantic web is useful because it makes us catalog our thoughts and language in a machine readable way That's not a use, that's a cost.
tgv 3y ago
Idk what definition of "semantic web" you have in mind, but AFAIK, it usually relies labeling words and names, and adding some vague relations. That's not enough. You can't build enough, inspectable, correct knowledge that way. Sure, you can list genealogical trees and and the components of a car, but it won't help you one bit when someone asks "give the five most important things about XML in gangsta rap style" (0), because none of the needed knowledge can be represented in any form of semantic web I've seen, and there isn't an engine capable of running the required inferences. [0] https://libreddit.de/r/ProgrammerHumor/comments/13lgv3u/chatgpt_in_gangster_rap_style_give_the_top_five/ https://libreddit.de/r/ProgrammerHumor/comments/13lgv3u/chat... You may think that's a joke, but from the performance perspective, it's utterly impressive. Summarizing a paper is even more impressive, and is equally far outside the scope of the semantic web.

chrismorgan 3y ago

Another hazard of the AI approach is that it’s probabilistic. Presuming the semantic information and implementation are all correct (which I will admit is not a given), they’ll always be correct, whereas the AI approach will sometimes give wrong answers.

brookst 3y ago
As you note, semantic data created by humans is also probabilistic and can be wrong. Ditto with semantic data created by automated systems that were created by humans. There will always be incorrect data in the world. AI doesn't change that, but so what?
- chrismorgan 3y ago
  There’s a difference, though: if the erroneous data comes from human input error, it can easily be corrected, and if the error is due to a software bug, it can be diagnosed and corrected; but errors that come from LLMs (the main type of AI in question here, and indicative enough of AIs in general, as it stands, though probably worse) are basically unfixable (that is: we don’t know how it would even be possible to fix them in general, so all you can do is patch things here and there, which will only probably fix the problems, and will increase the frequency of other problems).
  - brookst 3y ago
    How is an error from an LLM less correctable than one from a human? If "Paris is the capital of Germany" makes it into a news article, how does the origin of the error impact its correctability? Reinforcement learning seems pretty analogous to how we correct people who are wrong about something. I'm really not seeing a category difference here?
    - chrismorgan 3y ago
      With a human, if you have an employee, you can instruct them to correct an error, and if they refuse to or just don’t, you can fire them—you have recourse. LLMs are complex like humans, but you lack the ability to make it do what you want, or to have recourse. If the model has gained the impression that Paris is the capital of Germany (a more concrete and fixable error than most, frankly), this is woven into its parameters throughout, and it’s really hard to fix, and for LLMs we basically have no idea how to actually truly do so. With such a simple error as “Paris is the capital of Germany”, we do know how to fine-tune the model to fairly effectively mask that error, but the important errors are often closer to what you might consider procedural than factual—rather than its “memory” containing a wrong fact, it’s “thinking” in the wrong way. But all up, I’d call “lack of recourse” the most important difference.
pixl97 3y ago
Based on the failure of the semantic web as of so far, I think the statement 'not a given' is a given in itself. Humans are horrible at giving systems correct information.
grumpymouse 3y ago
LLMs always remind me of pigeons you see in the city that have spent their whole life eating out of bins and pecking dropped chunks of kebab. We trained it on a bunch of stuff, but we're not sure quite what. Looks like it works OK, so let's get it on a plate!

pjc50 3y ago

> The semantic web is useful because it makes us catalog our thoughts and language in a machine readable way That's not a use, that's a cost.

tgv 3y ago

Idk what definition of "semantic web" you have in mind, but AFAIK, it usually relies labeling words and names, and adding some vague relations. That's not enough. You can't build enough, inspectable, correct knowledge that way. Sure, you can list genealogical trees and and the components of a car, but it won't help you one bit when someone asks "give the five most important things about XML in gangsta rap style" (0), because none of the needed knowledge can be represented in any form of semantic web I've seen, and there isn't an engine capable of running the required inferences. [0] https://libreddit.de/r/ProgrammerHumor/comments/13lgv3u/chatgpt_in_gangster_rap_style_give_the_top_five/ https://libreddit.de/r/ProgrammerHumor/comments/13lgv3u/chat... You may think that's a joke, but from the performance perspective, it's utterly impressive. Summarizing a paper is even more impressive, and is equally far outside the scope of the semantic web.