4 ms·

> Isn't the hint in the name? Actually the hint is not in the name, that only tells a portion of the truth. If LLM were trained on textbooks teaching foreign l

> Isn't the hint in the name? Actually the hint is not in the name, that only tells a portion of the truth. If LLM were trained on textbooks teaching foreign languages from every other language, the name would suffice. But it does not, LLMs are trained on wide variety of texts, many of them social media, blog, and general web posts. The larger web is an ocean of bullshit, and that is where the bullshit within LLMs comes from. 'Textbooks are all you need' ought to be how LLMs are trained so people can use them for their "knowledge work" without bullshit ramifications.

helsinkiandrew 3y ago
> 'Textbooks are all you need' ought to be how LLMs are trained so people can use them for their "knowledge work" without bullshit ramifications. But in many spheres of knowledge there are still a lot of contradictory opinions in textbooks. Why did Rome fall?, What's the best X? In science there have been many schisms: quantum theory, tectonic plates, GMO/Organic etc. Writing code is probably one of the few areas where just about all content is 'good' - there may be a lot of bad or poor performing code in books and online but the vast majority of it 'works'.
- bsenftner 3y ago
  Very good points. That gives me the idea of specifically training LLMs on each side of the active scientific controversies and then host/publish an event where they debate. Might be very interesting to geeks, or they will hate it. Probably both.
OfSanguineFire 3y ago
Textbooks are a rather bad source of info. Consider that history textbooks in many countries look at everything through the lens of a particular national mythology, since one of the goals of a country’s public education is to forge a cohesive society, and that is done by downplaying past controversies. For many other fields, introductory textbooks even at the university level may abound in oversimplifications, and it is only in later years of study that one begins, by reading specialized papers and monographs, to grasp how complicated the facts really are.
pdonis 3y ago
> The larger web is an ocean of bullshit, and that is where the bullshit within LLMs comes from. Not only that: there's also the fact that LLMs do not "fact check" their output against any data source. They just generate text based on relative frequencies of different kinds of text in their training data. They don't even have the concept of text being related to other things in the world; in fact they don't even have the concept of "other things in the world". Any text generator built that way will output bullshit, no matter how accurate its training data is. "Bullshit" does not mean the output is necessarily wrong; it means, just as Frankfurt said, that the thing generating the output doesn't care about whether it's true or not. LLMs meet that criterion: they don't care about anything other than relative frequencies in their training data.

helsinkiandrew 3y ago

> 'Textbooks are all you need' ought to be how LLMs are trained so people can use them for their "knowledge work" without bullshit ramifications. But in many spheres of knowledge there are still a lot of contradictory opinions in textbooks. Why did Rome fall?, What's the best X? In science there have been many schisms: quantum theory, tectonic plates, GMO/Organic etc. Writing code is probably one of the few areas where just about all content is 'good' - there may be a lot of bad or poor performing code in books and online but the vast majority of it 'works'.

bsenftner 3y ago
Very good points. That gives me the idea of specifically training LLMs on each side of the active scientific controversies and then host/publish an event where they debate. Might be very interesting to geeks, or they will hate it. Probably both.

OfSanguineFire 3y ago

Textbooks are a rather bad source of info. Consider that history textbooks in many countries look at everything through the lens of a particular national mythology, since one of the goals of a country’s public education is to forge a cohesive society, and that is done by downplaying past controversies. For many other fields, introductory textbooks even at the university level may abound in oversimplifications, and it is only in later years of study that one begins, by reading specialized papers and monographs, to grasp how complicated the facts really are.

pdonis 3y ago

> The larger web is an ocean of bullshit, and that is where the bullshit within LLMs comes from. Not only that: there's also the fact that LLMs do not "fact check" their output against any data source. They just generate text based on relative frequencies of different kinds of text in their training data. They don't even have the concept of text being related to other things in the world; in fact they don't even have the concept of "other things in the world". Any text generator built that way will output bullshit, no matter how accurate its training data is. "Bullshit" does not mean the output is necessarily wrong; it means, just as Frankfurt said, that the thing generating the output doesn't care about whether it's true or not. LLMs meet that criterion: they don't care about anything other than relative frequencies in their training data.