3 ms·

Semantic Web lost itself in fine details of machine-readable formats, but never solved the problem of getting correctly marked up data from humans. In the curr

by pornel 4y ago

Semantic Web lost itself in fine details of machine-readable formats, but never solved the problem of getting correctly marked up data from humans. In the current web and apps people mostly produce information for other people, and this can work even with plain text. Documents may lack semantic markup, or may even have invalid markup, and have totally incorrect invisible metadata, and still be perfectly usable for humans reading them. This is a systemic problem, and won't get better by inventing a nicer RDF syntax. In language translation, attempts of building rigid formal grammar-based models have failed, and throwing lots of text at a machine learning has succeeded. Semantic Web is most likely doomed in the same way. GPT-3 already seems to have more awareness of the world than anything you can scrape from any semantic database.

pphysch 4y ago
Sure, but there are still a lot of decisions being made behind the curtain, when it comes to producing a model like GPT-3. How was the training data ontologized? Where did it come from? To some extent, these are the same problems facing manual curation.
- pornel 4y ago
  GPT may have had some manual curation to avoid making it too horny and racist, but on a technical level for such models you can just throw anything at it. The more the better, shove it all in.
mwelt 4y ago
Comparing "rigid formal grammar-based models" (whatever that might actually mean for now) to machine learning is like comparing apples to bananas. The former one is a rigorous syntactical formalization, aimed at being readable by machine and humans alike. The latter one is a learned interpolation of a probability distribution function. I do not see a single way to compare these two "things". Nevertheless, I may guess, what you actually are trying to say: Annotating data by hand (the syntax is completely irrelevant) is inferior to annotating data by machine learning. And this claim is at least debatable and domain-dependent. There are domains where even a 3% false-positive rate translates to "death of a human being in 3 out of 100 identified cases", and there are domains where it's to much work to formalize every bits and pieces of the domain and extracting (i.e. learning) knowledge is a feasible endeavor. I have experience in both fields, and I dare to say, that extracting concepts and relations out of text in a way that it can be further processed and used for some kind of decision process is way more complicated than you might imagine, and GPT-3 et al. do not achieve that.

by pornel 4y ago

pphysch 4y ago

Sure, but there are still a lot of decisions being made behind the curtain, when it comes to producing a model like GPT-3. How was the training data ontologized? Where did it come from? To some extent, these are the same problems facing manual curation.

pornel 4y ago
GPT may have had some manual curation to avoid making it too horny and racist, but on a technical level for such models you can just throw anything at it. The more the better, shove it all in.

mwelt 4y ago

Comparing "rigid formal grammar-based models" (whatever that might actually mean for now) to machine learning is like comparing apples to bananas. The former one is a rigorous syntactical formalization, aimed at being readable by machine and humans alike. The latter one is a learned interpolation of a probability distribution function. I do not see a single way to compare these two "things". Nevertheless, I may guess, what you actually are trying to say: Annotating data by hand (the syntax is completely irrelevant) is inferior to annotating data by machine learning. And this claim is at least debatable and domain-dependent. There are domains where even a 3% false-positive rate translates to "death of a human being in 3 out of 100 identified cases", and there are domains where it's to much work to formalize every bits and pieces of the domain and extracting (i.e. learning) knowledge is a feasible endeavor. I have experience in both fields, and I dare to say, that extracting concepts and relations out of text in a way that it can be further processed and used for some kind of decision process is way more complicated than you might imagine, and GPT-3 et al. do not achieve that.