7 ms·
"AI", students, and epistemic crisis
- lionkor 2y agowell point the student at the little disclaimer that says that it may sometimes be incorrect! Of course they changed it multiple times, it used to say that you shouldnt take what it says at face value, now it says it may sometimes be inaccurate, soon it'll say "learn more" and link to a page about how their model is super accurate but potentially, one in a million, makes mistakes. Im worried about these students because they will be in power in a few decades. Its already a shitshow with people who didnt grow up as AI iPad toddlers. I feel like parents are failing here, more than anything else. You can't stop these companies from doing this if it drives up the stock price, equally you cant vote against it effectively because a majority of the voting population either doesn't care, doesn't know, or asks ChatGPT what to vote for. Of course Plato was right and the only way we can fix it is to have philosophers in charge, not demagogues. Good luck with that, maybe in another 2000 years we'll be smart enough to make that happen!
- adammarples 2y ago"I am one person trying to work against a campaign of misinformation so vast that it fucking terrifies me. This kid is being set up for a life lived entirely inside the hall of mirrors." This is a little hyperbolic and instead maybe the kid can learn that chatgpt.com is not a reliable source. It even says at the bottom of the page "chat gpt can make mistakes". Lots of things are not reliable sources. Wikipedia is not a reliable source. Teachers are supposed to teach this, and teach cross referencing at the very least, not freak out.
- auggierose 2y agoYou could probably make the same argument for search bar vs. peer-reviewed publications. Of course, the search bar (which is also AI, by the way) can help you to get to the peer-reviewed publications. But the same is true for ChatGPT. The problem is that ChatGPT sounds like presenting objective facts. But maybe the lesson here is that just because something sounds right, it isn't necessarily right, and that is something that should be taught in school. Of course, that undermines the function of school to produce obedient citizens.
- llm_trw 2y agoI had that same argument, the teacher who told me to not trust non peer reviewed articles ended up flooding her Facebook was with pro-brexit lies a decade later. Turns out critical thinking is not outsourcing your thinking to a third party, be it a peer reviewes journal, google search results, or chatgpt.
- lionkor 2y agoWhat is a 'pro brexit lie'?
- auggierose 2y ago£350m a week for the NHS
- Angostura 2y agoImmigration will be reduced We will be able to negotiate excellent trade deals outside the EU effortlessly The EU need our trade so badly we will be able to negotiate a seamless trade deal with them … and many others.
- Semaphor 2y agoIf they only trust ChatGPT, why not just show them? From asking leading questions, to simply asking ChatGPT what the chances are of it making mistakes and hallucinating, there are tons of options. Considering this is a transcription of a deleted twitter post, my internet radar of "yeah, that happened" gives a lower chance this is true than your average ChatGPT answer.
- lionkor 2y agoI'm sure it happened, I've had conversations with younger people that went like that. They pick this up before they start learning critical thinking
- Semaphor 2y agoAnd you didn’t get the idea to just show them, as I said, either? I’m not saying it’s impossible for little kids not to know about it, I’m saying it’s very straightforward to show them why that’s wrong by using the one tool they seem to believe.
- lionkor 2y agoI fully agree with you, yeah. What helped for me is demonstrating that it will say whatever to seem smart, once thats demonstrated its easier to see that its not facts, its just convincing language.
- relaxing 2y agoYou don’t work with kids as public school teacher.
- Semaphor 2y agoPlease explain what’s wrong about my reasoning then?
- visarga 2y ago> they only trust ChatGPT And somehow this is an improvement over the previous status quo. ChatGPT is pretty well read and on important issues rarely says stupid things. Better than believing some influencer.
- pietmichal 2y agoThis sentiment reminds me of my teachers who complained that Wikipedia is not a reliable source to learn from.
- puttycat 2y agoWikipedia and LLMs are completely different entities that work in completely different ways.
- auggierose 2y agoAnd yet, much of the same criticism applies to both.
- hyperhopper 2y agoWikipedia is generally right and you can check it's sources. Also everything there was written by a human. LLMs are often wrong and you cannot check their sources. Also since it is generated you can trick it into spitting out falsehoods intentionally. They are not even close.
- auggierose 2y agoYou can also check the sources of LLMs, just ask them for it, and then check that. An LLM is simply more flexible and more powerful than Wikipedia, and thus you have to be more cautious with regards to its results. "Generally right" is not the same as "reliably right", and therefore if you really need to rely on a fact for something important, I would trust neither Wikipedia nor LLMs.
- jacooper 2y ago> LLMs are often wrong and you cannot check their sources. Depends on which LLM you are using, perplexity and copilot both cite their sources.
- ClumsyPilot 2y agoWikipedia is a marvel of the commons and did wonders for education. hostility to it from academia was surprising
- deadbabe 2y agoI really don’t see any other solution to this kind of problem except for one: LLMs must become perfect, and never be wrong. Relying on kids to do cross referencing and deeper fact checks into everything they ask an LLM is just not going to happen at scale.
- tomhoward 2y agoMaybe it’s the opposite; kids need to learn that LLM’s can bullshit just like every other person and institution can bullshit, and the most important skill they can have is verification of information, no matter where it’s coming from.
- rchaud 2y agoWhat's so hard about following the Wikipedia model of citing sources? Even 20 years ago, it was clear to college students that they cannot cite "Wikipedia", but they can cite the academic literature it referenced. It would take seconds to figure out of the ChatGPT citation is real or made up.
- sieste 2y agoLike probably many people here I still remember having to find facts in books in libraries, before the internet made this skill mostly redundant. Then, as a student I remember having to put together facts from various (internet) sources into a coherent narrative. Now chatbots can just generate text and that skill seems less valuable. I use both the internet and GenAI extensively now. But I feel that having gone through the "knowledge work" activities without the crutches puts me in a better position to assess the correctness and plausibility of internet sources and AI in a way that kids who grow up using them constantly don't have. I feel quite privileged to be in that position, that I wouldn't be in had I been born 10 or 20 years later. I also feel sorry for kids these days for not having the opportunity to learn things "the hard way" like I had to. And I feel extremely snobbish and old for thinking that way.
- mrcartmeneses 2y agoIt’s something reflected in the conversations I have with my academic friends. I’m told that every essay is written in the same “voice” and that although they are usually a simulacra of an academic paper, they say nothing. The sad part comes when we reflect that these students are not learning the deep thinking skills that comes with academic writing
- deleted 2y ago[deleted]
- ryandv 2y agoThere are two views of writing: one, as the production of a literary artifact that has value in its own right and stands alone as the embodiment of a complex idea itself; on the other hand, as a process and tool for thought, where the literary artifact is merely meant to represent the cognitive work that went into its production. From the latter perspective, the bulk of the value is not derived from the output of the process of writing, but rather from the understanding and insight that was gained during its production - "it's the journey, not the destination". With generative AI we are now shortcutting directly to the destination while eliding all the knowledge and understanding we are supposed to be gaining on the way. It's deemed sufficient to merely produce and exchange the symbols of understanding without needing to possess any real underlying wisdom (the first view above), because it's through the exchange of these abstract symbols, irrespective of whether or not there is anything behind them, that we can play and win certain social games. This is merely the natural continuation of trends that began during the dawn of the Internet era, when we started to consider pixels on a screen an accurate map of reality.
- padolsey 2y agoI don't buy into the rhetoric about misinformation but the author touches on a real concern. I blame ChatGPT and other LLM clients for not surfacing their fallibility more clearly. They should highlight claims in their UX and allow options for the user to "research" or "verify" in their own way, without relying on that very flakey single one-shot inference. The big honchos (Anthropic, OpenAI) need to make it clearer that their output is, at best, an informed guess. A tiny disclaimer doesn't cut it.
- LUmBULtERA 2y agoJust to be clear, the "tiny disclaimer" is directly below the prompt box and says "ChatGPT can make mistakes. Check important info." So it is interesting that anyone would take a hard stance that their prompt results are true facts without checking any important info (particularly after someone highlights this if they somehow missed it).
- visarga 2y agoAll you need to do is ask it to perform a web search. Or you can sample a few answers at high temperature. I think young people growing with LLMs will know all the ins and outs of their limitations. But it's a rapidly evolving field, maybe in 5 years we won't doubt LLMs very often.
- huimang 2y agoI feel like this is a bit overblown. Growing up we heard, ad nauseam, that "wikipedia is not a reliable source". People just need to state the same thing about LLMs- they aren't reliable, but can potentially point you to primary sources that -are- reliable. Once the shininess of the new toy wears off, people will adjust.
- nebulous1 2y agoI think it's pretty clear that as a tool they can be used very effectively if you have at least some understanding of their limitations. I will say that wikipedia, as things stand, is way more accurate than chatgpt. So much so that comparing them doesn't even make sense to me.
- cheschire 2y agoUntil you can quantifiably prove Wikipedia is more accurate than ChatGPT, I’m not sure how you can say GP’s comparison doesn’t make sense.
- nebulous1 2y agoOkay, I can't defend that because in reality I'm talking purely from my subjective experience. I will say that in my experience it isn't close at all. However, I will also say that a lot of the things that chatgpt gets wrong for me, wikipedia just won't contain at all, but then chatgpt seeming confident when it's wrong is basically the whole problem.
- cheschire 2y agoWikipedia has well documented and explored issues related to vandalism, bias, and misinformation. But don’t get me wrong, I also view ChatGPT conversations as being on par with pub chats as far as “confident facts” go. I think our difference may be in how we view Wikipedia.
- 2y ago
- i_am_proteus 2y agoI've seen this happen, too, including student incredulity that ChatGPT can be wrong, and recalcitrance when guided to find proper sources. Up to the point of arguing for a higher grade based on the correctness of LLM output.
- JohnKemeny 2y agoOne thing is correctness, another thing is that GPTs can output long passages of copyrighted work verbatim, and users risk unknowingly submitting plagiarized work.
- werdnk 2y agoIn the current system, where students can anonymously report teachers (most of whom do not have tenure and are afraid) it will be hard to change anything. Otherwise, you could do a mixture of very strict exams without multiple choice and large individual projects (no group projects). If you only do exams, people who don't do well thinking in a room crammed full of people at 8am will be disadvantaged.
- A_D_E_P_T 2y agoThe great problem with ChatGPT is that it's a sycophant and aims to please. If you ask it about something it doesn't know, right then and there, it will concoct a fiction for you. It won't say "I don't know," or "I can't help with that." If you coach it to respond to something in a certain way, it'll respond that way for you as its top priority. If you ask it to review a text, it'll usually find a way to give you at least a 7 or 8 out of 10. (Though, interestingly, rarely a 10/10 score. You can upload excerpts from some of the great works of literature and philosophy and see ChatGPT give them an 8/10, just as it gives an 8/10 to your essay or blog post.) Practically the only way to get a halfway critical response is to add the words "be critical" to your prompt. A more circumspect and less obsequious ChatGPT would solve a lot of problems.
- zertar 2y agoInteresting. Copilot on the other hand is more like a conceited, passive aggressive brat who uses some polite phrases but aborts conversations if you contradict it too much. The only smooth conversations with Copilot are the ones where you allow it to regurgitate "facts" and act in a submissive manner.
- jodrellblank 2y ago> “If you ask it about something it doesn't know, right then and there, it will concoct a fiction for you. It won't say "I don't know,"” It doesn’t know that it doesn’t know. Texts it was trained on rarely go “On the History of the Greek Language, by Andros Dimos; ahem what is the history of the Greek language? I don’t know. The end.”
- A_D_E_P_T 2y agoHmmm... well... Let me give you an example. The other day I asked for a summary of the short story "The Palace at Midnight" by Robert Silverberg. GPT-4o responded: "The Palace at Midnight" by Robert Silverberg is a science fiction short story that was first published in 1980. It is part of Silverberg’s "Majipoor" series, but stands alone in terms of its setting and characters, distinct from the expansive world-building typical of the series. The story is set in a dystopian future in India and revolves around themes of memory, loss, and the haunting allure of the past. The narrative follows the protagonist, Edward, and his group of friends who are survivors of a nuclear war that has devastated much of the world. . . All of which was complete nonsense. "The Palace at Midnight" is a short story of a few pages that has nothing to do with Majipoor, was published in 1981, and was set in San Francisco. It doesn't know -- it knows that it doesn't know -- so it just makes something up that seems superficially credible. Because it aims to please and be helpful; altogether too helpful.
- Kiro 2y agoAm I the only one who almost never experience any hallucinations when talking to ChatGPT? I have to really push it into a corner with trick questions and conflicting instructions about obscure topics in order to trigger hallucinations. That it would just come up with random false facts about something as common and "basic" as the history of the Greek language sounds like a made-up issue.
- nebulous1 2y agoWell, I can't say that you're the only one, but I can say that I don't even interact with it that frequently and I've had numerous instances of it giving me false information. Enough times that I'm surprised to see you say otherwise.
- deleted 2y ago[deleted]
- cheschire 2y agoI know it feels like 4o has been out forever but go back to the 3.5 model without internet access for a while, which is all that a lot of students can afford. It’s kind of terrible. But yeah ever since I could start adding “and include an explanation with citations” to my prompts I’ve not had an issue with hallucinations more than once or twice.
- hayley-patton 2y agoDoes it give real citations? I've heard it gives bogus citations, though can't remember how well it went last time.
- cheschire 2y agoI haven’t had any issues, but I’m not using it to do “real” research so the veracity of sources has been less of an issue for me. I typically ask, then spot check sources. If I’m using it to make purchase decisions or scheduling my day then I use it purely to find me source pages. It’s really terrible at calculating the cost of services.
- gitfan86 2y agoPerplexity links to sources. All he has to do was show the student the same search on perplexity. I do feel sorry for professors like this that cannot adapt to technology changes at the rate they are happening
- nebulous1 2y agoWhy would the teacher (this is almost certainly not third level education we're talking about here) use a different AI when they already have primary sources? Your suggestion makes no sense.
- gitfan86 2y agoThe goal of teaching is to help students learn. By showing the student Perplexity the student would learn the value of sources and how he can get to those sources using a "search". It is unrealistic to think that there is any value in teaching students the Dewey Decimal system for finding sources.
- atoav 2y agoThe problem with not writing yourself isn't that people didn't do the writing themselves, the problem is that they didn't do the thinking themselves that is a prerequisite for writing. Now of course like any tool this can be used without falling into that trap, but people are lazy and the truth is that if you don't absolutely have to do it yourself most people won't.
- csantini 2y ago<< They keep coming up with weird “facts” (“Greek is actually a combination of four other languages) >> Not as wrong as the author thinks. From Britannica.com: "Greek language, Indo-European language spoken mostly in Greece. Its history can be divided into four phases: Ancient Greek, Koine, Byzantine Greek, and Modern Greek."
- moritzruth 2y agoThe language evolution is commonly divided into four stages. isn’t the same as Four distinct languages had an influence on the development of this fifth language.
- earl_gray 2y agoIf I had to suggest where the “combination of four languages” idea came from, it would be from Homeric Greek (the language the Iliad and Odyssey were written down in). This was genuinely a complete mess, formed of a hodgepodge of different dialects. From wikipedia: “[Homeric Greek] is a literary dialect of Ancient Greek consisting mainly of an archaic form of Ionic, with some Aeolic forms, a few from Arcadocypriot, and a written form influenced by Attic.” I’m not sure if this is a plausible explanation as I don’t have much experience using LLMs.
- allsummer 2y agoWhat happened was either the teacher is severely biased against ChatGPT and fabricated the fact to fit their narrative. Or ChatGPT gave the correct answer, but the student interpreted it wrong. I do believe the students keep coming up with weird (correct) facts, and that this can be scary for a teacher who is stuck at a search bar.
- joaquincabezas 2y agoso it’s the classical “it’s on the internet so it’s true” but on steroids. I remember a US student in the early 2000s citing a geocities website as source for the FACT that aliens created the pyramids of Egypt
- xg15 2y agoI would have thought this problem was easy to solve: "Yes, look it up, but remember your source, there is a lot of bullshit on the internet. Especially don't trust AI tools, we know those often return nonsense information." (Actually, didn't ChatGPT have a disclaimer right next to the prompt box that warns against incorrect answers?) So I'm more surprised (and scared) that students don't just use LLMs for sourcing but are also convinced they are authoritative. Maybe being in the tech bubble gave a false impression here, but weren't hallucinations one of the core points of the whole AI discurse for the last 1.5 years? How do you learn about and regularly use ChatGPT, but miss all of that?
- BlaDeKke 2y agoAs we grow older, we learn a lot of facts that we can use to test the correctness of ChatGPT and identify its hallucinations because we have the knowledge to do so. However, a young person who is just starting to understand the world and gather knowledge might not have enough information to notice these hallucinations.
- xg15 2y agoYup, no question here why he believes ChatGPT's initial statements. I'm more baffled that when the teacher corrects him, he goes on and defends ChatGPT.
- relaxing 2y agoAre these tools being promoted and sold as fallible implements? Or are they being hyped as super-human intelligence? Which takeaway is an impressionable child going to latch onto? One who wasn’t in on the last 1.5 years of discourse?
- jdietrich 2y agoChatGPT says "ChatGPT can make mistakes. Check important info." directly under the prompt box. If people will blindly trust a source that explicitly states that it isn't a reliable source, then they've got much bigger problems than AI.
- pitt1980 2y agoOne thing I’ve noticed, is that at least in the free version, if you ask chatGPT for sources outside itself, ‘can you give me a link to somewhere on it internet where it says that?’, it won’t do that. It is very much a black box in terms of letting you track its logic. Seems like future versions should be much more transparent in terms of letting you track the logic of why it’s telling you what it’s telling you.
- shzhdbi09gv8ioi 2y agoThey trained on copyrighted material and scraped stuff they shouldn't have. That's why they "cant" show sources.
- elforce002 2y agoBrave AI search engine gives you various sources if you want to dig deeper.
- allsummer 2y agoYou can just ask it "What scholars or source material books could I check out to verify these linguistics facts?". Many facts and answers are gathered/aggregated from many different (sometimes conflicting) sources. It won't link the internet page where it found the information, because it didn't find the information on a single internet page.
- pitt1980 2y agoAt on point I asked it for academic papers that said a particular thing, as best I can tell, only about 60% of the papers it cited actually existed
- allsummer 2y agoThis is a problem for this particular teacher (who sees their students surpassing them in understanding and using AI), but of course it is projected to be a problem for the student. No student is ever hurt by the introduction of a more advanced knowledge system. We heard similar laments decades ago, with: Students just believe the first 10 search results of Google. Those students are now the teachers of today, starting at the search bar. I'd go so far as saying (if version other than 3.5 was used) that ChatGPT was correct and has far more linguistics knowledge than this teacher ever will. "Greek is actually a combination of four other languages" is not an answer that ChatGPT will ever give, but something a teacher makes up to claim Ch*tGPT is a Nonsense Machine. ChatGPT: Greek has evolved in stages from Mycenaean Greek (Linear B script) through Classical Greek, Hellenistic (Koine) Greek, Byzantine Greek, and Modern Greek. It has been influenced by ancient Near Eastern languages, Latin, Turkish, Italian, and French. If there really is an epistemic crisis, then it already existed and ChatGPT merely reflects it, not caused or contributed to it.
- rchaud 2y ago> who sees their students surpassing them in understanding and using AI This would be the equivalent of my saying that a basketball coach is obsolete because NBA 2K is available. ChatGPT output some garbage, and the student doesn't understand why or how it can be wrong. Presumably this is where a professor would attempt to help the student develop some critical thinking skills.
- deleted 2y ago[deleted]
- tatrajim 2y agoAnd, on a related note for education, AI is quickly obviating the need to master and plumb the depths of foreign languages. Dating myself, doubtless, but as an undergraduate, it was an unalloyed joy to study ancient Greek and read Plato and Euripides in the original, however haltingly. And later Korean, Japanese, and Chinese beckoned providing a lifetime of rich understanding of life outside the confines of English. For Americans, at least, perhaps ours is the last generation that will seek rewire our understanding of reality through linguistic hacking.
- nathansherburn 2y agoAI will always be far better at languages than I'll ever be and I expect it to get much better very quickly. But I'm still learning my partner's language and don't think I'll stop any time soon. I think it's interesting and fun in and of itself. It's also a great scaffold for learning about another culture and learning to respect and understand folks from different walks of life.
- ryandv 2y agoPresuming that those languages exist within OpenAI's training dataset. Try to have a conversation in Rinconada with ChatGPT; my last attempt led to its admission that it had no proficiency in this language. You will have to find a native speaker, and their numbers are dwindling as Tagalog and English are being favored in education - the latter thanks to American linguistic imperialism. The AI does not circumscribe all of reality; not all of reality is captured on the internet.
- unraveller 2y agoEvery teaching moment is also a learning moment. Asking kids to refrain from wanting tiny bits of assistance at hand in tough times is tantamount to asking them to lie or accept a poorer grade as others do cheat the whole hog undetected. Human's crave ease. Education promises an easier adulthood and it is no way clear you can provide it by legacy regurgitation means.