6 ms·

I feel like this is a bit overblown. Growing up we heard, ad nauseam, that "wikipedia is not a reliable source". People just need to state the same thing about

I feel like this is a bit overblown. Growing up we heard, ad nauseam, that "wikipedia is not a reliable source". People just need to state the same thing about LLMs- they aren't reliable, but can potentially point you to primary sources that -are- reliable. Once the shininess of the new toy wears off, people will adjust.

nebulous1 2y ago
I think it's pretty clear that as a tool they can be used very effectively if you have at least some understanding of their limitations. I will say that wikipedia, as things stand, is way more accurate than chatgpt. So much so that comparing them doesn't even make sense to me.
- cheschire 2y ago
  Until you can quantifiably prove Wikipedia is more accurate than ChatGPT, I’m not sure how you can say GP’s comparison doesn’t make sense.
  - nebulous1 2y ago
    Okay, I can't defend that because in reality I'm talking purely from my subjective experience. I will say that in my experience it isn't close at all. However, I will also say that a lot of the things that chatgpt gets wrong for me, wikipedia just won't contain at all, but then chatgpt seeming confident when it's wrong is basically the whole problem.
    - cheschire 2y ago
      Wikipedia has well documented and explored issues related to vandalism, bias, and misinformation. But don’t get me wrong, I also view ChatGPT conversations as being on par with pub chats as far as “confident facts” go. I think our difference may be in how we view Wikipedia.
      - ninkendo 2y ago
        > Wikipedia has well documented and explored issues related to vandalism, bias, and misinformation. Last I read about this, the error rate in Wikipedia was actually lower than in the Encyclopedia Britannica, by a measurable amount. This was a while back, and admittedly it only counted articles where there was an equivalent article in both (which probably gives a better picture of Wikipedia, as those kinds of “boring” articles have less vandalism…) but it’s not immediately a given that Wikipedia is just objectively bad at being accurate.
    - sieste 2y ago
      One way of looking at it is that Wikipedia has a transparent and auditable way of correcting and updating false information, which makes it inherently more reliable and trustworthy than tensor weights finetuned by unreproducible human feedback.
roenxi 2y ago
And the LLMs are still improving at a brisk rate. If they were outperforming teachers by the end of the decade that'd be well within expectations. Anyone pretending that human authority figures routinely get things right is defining correctness using circular logic.
- sieste 2y ago
  Current LLMs are lacking introspection, plausibility checking, consulting external sources, and belief updating in the presence of new evidence. All of these you'd need to replace human teachers and it's not clear that the next token prediction paradigm can ever emulate these features reliably. So "outperforming teachers", while not impossible, is a very optimistic expectation as it requires more than mere improvement on existing methodology.
  - jdietrich 2y ago
    Maybe I was particularly unfortunate, but none of my human teachers had any of those features. There was one inviolable source of truth in my classrooms - the examination syllabus and the accompanying textbook.
    - wholinator2 2y ago
      That's sad. Probably you were particularly unfortunate. Though i definitely do know people who echo your sentiment. I'm wondering at what level of education this occurred? A 2nd grade teacher requires a different set of introspection capabilities than a high school calculus teacher than a college world history class. Possibly I'm particularly fortunate but at all levels i have at least had experienced with teachers that had the proper introspection. Emotional introspection and reflection in grade school and high school. And academic introspection and curiosity in high school and college. Certainly there are some bad eggs, but my small state school was rife with professors there for the love of learning. Our learning and theirs.
cheschire 2y ago
Well even before Wikipedia… remember calculators being banned? I think you’re right, we will adjust. Curriculum development will start to include new checkpoints and controls.
- rsanek 2y ago
  Aren't calculators still banned in primary education? From what I know, you cannot use them during most tests.
watwut 2y ago
Wikipedia is massively more reliable then ChatGPT.
shepherdjerred 2y ago
> wikipedia is not a reliable source Wikipedia is as good as anything else. As an example, I frequently use Wikipedia to read about history, computer science topics (e.g. how to implement an algorithm), or scientific topics. The exception is current events, but even then, I suspect that Wikipedia is not any more biased than the news. I'm open to having my mind changed, though.
- Perceval 2y ago
  > Wikipedia is as good as anything else. Encyclopedias – including Wikipedia – are not acceptable sources for college-level work certainly. They are tertiary literature, which can provide an overview to someone trying to get a toehold into a subject, and which can hopefully point them toward primary and secondary sources. But tertiary sources are not typically allowable citations for college research.
  - shepherdjerred 2y ago
    Yes, you are right that Wikipedia is inappropriate in academic settings. But, this thread has discussed the reliability of Wikipedia.

nebulous1 2y ago

I think it's pretty clear that as a tool they can be used very effectively if you have at least some understanding of their limitations. I will say that wikipedia, as things stand, is way more accurate than chatgpt. So much so that comparing them doesn't even make sense to me.

cheschire 2y ago
Until you can quantifiably prove Wikipedia is more accurate than ChatGPT, I’m not sure how you can say GP’s comparison doesn’t make sense.
- nebulous1 2y ago
  Okay, I can't defend that because in reality I'm talking purely from my subjective experience. I will say that in my experience it isn't close at all. However, I will also say that a lot of the things that chatgpt gets wrong for me, wikipedia just won't contain at all, but then chatgpt seeming confident when it's wrong is basically the whole problem.
  - cheschire 2y ago
    Wikipedia has well documented and explored issues related to vandalism, bias, and misinformation. But don’t get me wrong, I also view ChatGPT conversations as being on par with pub chats as far as “confident facts” go. I think our difference may be in how we view Wikipedia.
    - ninkendo 2y ago
      > Wikipedia has well documented and explored issues related to vandalism, bias, and misinformation. Last I read about this, the error rate in Wikipedia was actually lower than in the Encyclopedia Britannica, by a measurable amount. This was a while back, and admittedly it only counted articles where there was an equivalent article in both (which probably gives a better picture of Wikipedia, as those kinds of “boring” articles have less vandalism…) but it’s not immediately a given that Wikipedia is just objectively bad at being accurate.
  - sieste 2y ago
    One way of looking at it is that Wikipedia has a transparent and auditable way of correcting and updating false information, which makes it inherently more reliable and trustworthy than tensor weights finetuned by unreproducible human feedback.

roenxi 2y ago

And the LLMs are still improving at a brisk rate. If they were outperforming teachers by the end of the decade that'd be well within expectations. Anyone pretending that human authority figures routinely get things right is defining correctness using circular logic.

sieste 2y ago
Current LLMs are lacking introspection, plausibility checking, consulting external sources, and belief updating in the presence of new evidence. All of these you'd need to replace human teachers and it's not clear that the next token prediction paradigm can ever emulate these features reliably. So "outperforming teachers", while not impossible, is a very optimistic expectation as it requires more than mere improvement on existing methodology.
- jdietrich 2y ago
  Maybe I was particularly unfortunate, but none of my human teachers had any of those features. There was one inviolable source of truth in my classrooms - the examination syllabus and the accompanying textbook.
  - wholinator2 2y ago
    That's sad. Probably you were particularly unfortunate. Though i definitely do know people who echo your sentiment. I'm wondering at what level of education this occurred? A 2nd grade teacher requires a different set of introspection capabilities than a high school calculus teacher than a college world history class. Possibly I'm particularly fortunate but at all levels i have at least had experienced with teachers that had the proper introspection. Emotional introspection and reflection in grade school and high school. And academic introspection and curiosity in high school and college. Certainly there are some bad eggs, but my small state school was rife with professors there for the love of learning. Our learning and theirs.

cheschire 2y ago

Well even before Wikipedia… remember calculators being banned? I think you’re right, we will adjust. Curriculum development will start to include new checkpoints and controls.

rsanek 2y ago
Aren't calculators still banned in primary education? From what I know, you cannot use them during most tests.

watwut 2y ago

Wikipedia is massively more reliable then ChatGPT.

shepherdjerred 2y ago

> wikipedia is not a reliable source Wikipedia is as good as anything else. As an example, I frequently use Wikipedia to read about history, computer science topics (e.g. how to implement an algorithm), or scientific topics. The exception is current events, but even then, I suspect that Wikipedia is not any more biased than the news. I'm open to having my mind changed, though.

Perceval 2y ago
> Wikipedia is as good as anything else. Encyclopedias – including Wikipedia – are not acceptable sources for college-level work certainly. They are tertiary literature, which can provide an overview to someone trying to get a toehold into a subject, and which can hopefully point them toward primary and secondary sources. But tertiary sources are not typically allowable citations for college research.
- shepherdjerred 2y ago
  Yes, you are right that Wikipedia is inappropriate in academic settings. But, this thread has discussed the reliability of Wikipedia.