14 ms·
Teaching with AI
- Der_Einzige 3y agoThe reality is that effective LLMs, combined with some kind of knowledge retrieval, are coming close to becoming the idealized individual tutor. This is also a daily reminder that studies show that individual tutoring is objectively the best way to educate people: https://en.wikipedia.org/wiki/Bloom%27s_2_sigma_problem https://en.wikipedia.org/wiki/Bloom%27s_2_sigma_problem
- swyx 3y ago[deleted because dont want to be drawn into flamewar]
- empath-nirvana 3y agoChatgpt does tutoring just fine, i've had it draw up a lesson plan for me and execute with hardly any special prompt engineering at all, just sort of like: "Please tutor me on french adverbs, please start by asking me a few questions to find out what I already know," and it dialed in fairly well to my level.
- BoorishBears 3y ago[flagged]
- minimaxir 3y agoYou are replying to swyx.
- BoorishBears 3y agoThank you, updated indirect references to direct: I know it's un-HN but Jesus christ am I tired of hearing this person's garbage quoted ad nauseam like a gospel
- maxbond 3y agoJeez bro. This is a pretty intense reaction to a lukewarm and reasonable take. Personally I appreciate an "AI influencer" being down to earth and being willing to say that the technology isn't magic, amidst a huge amount of hype. If you think people are parroting swyx uncritically - that's hardly a criticism of swyx, is it? I think you should keep reflecting on your realization about how people got swept up in the cryptoasset hype. You can believe this technology is promising and will improve dramatically without being a fanatic. You can disagree without going for the jugular.
- BoorishBears 3y agoTo hear: "the man talks about things he has no clue about with disturbing loudness" And turn it into: "that's actually not a criticism of the guy at all!" Maybe when you start translating everything into what you want to hear, it's hard to turn off.
- deleted 3y ago[deleted]
- maxbond 3y ago> To hear: "the man talks about things he has no clue about with disturbing loudness" > And turn it into: "that's actually not a criticism of the guy at all!" Ironically, that's not at all what I wrote. You made statements such as these: > But they read tweets like this, and they have the typical developer blindspot of not questioning motive enough and they believe it! > I'm tired of hearing this person's garbage quoted ad nauseam like a gospel And so I observed: > If you think people are parroting swyx uncritically - that's hardly a criticism of swyx, is it? To be clear, if you feel other people are repeating swyx uncritically, that is a criticism of those people. When it comes to people being loud while not adding knowledge to the discussion - with all due respect, you should consider the composition of your house before casting that stone. I apologize if my comments are frustrating for you to read; I am doing my best to give you useful feedback. You behaved in a pretty extreme way, which is not considered acceptable in this community (or most communities). And I take it you knew better: > I know it's un-HN but...
- rvz 3y agoWho? It has been known that LLMs cannot reason transparently nor can these black-boxes explain themselves without regurgitating and rewording their sentences to sound intelligent, but instead are confident sophists no matter what random anyone tells you otherwise. EDIT: This is the context before it was deleted by the grandparent comment: >> i have yet to see any ai system properly implement individual level-adjusting tutoring. i suspect because the LLM needs a proper theory of mind (https://twitter.com/swyx/status/1697121327143150004 https://twitter.com/swyx/status/1697121327143150004) before you can put this to practice. My point still stands.
- gojomo 3y agoEmphatic assertions "it has been known" are anti-convincing.
- BoorishBears 3y agoYou're showing why I'm so annoyed by this perfectly! It's malicious to rope theory of mind into justifying that point because it's just wrong enough. If the reader doesn't think deeply about why on earth you would ever to rope theory of mind into this, your brain will happily go down the stochastic parrot route: "How can it have theory of mind, theory of mind is understanding emotions outside of your own, the LLM has no emotions" But that's a complete nerdsnipe. — If instead you distrust this person's underlying motivations to not be genuine intellectual curiosity, but rather to present a statement that is easily agreed to even at the cost of being wrong... you examine that comment at a higher level: What is theory of mind adding here besides triggering your typical engineer's well established "LLMs are over-anthropomorphized" response? Even in psychology it's a hairy non-universally accepted or agreed upon concept! Theory of mind gives two things at the highest level: inward regulation: which is nonsensical for the LLM, you can tell it what emotion it's outputting as, it does not need theory of mind to act angry outward recognition: we've let computers do this with linear algebra for over 2 decades. It's what 5 of the largest companies in technology are built on... — Commentary like that accounts is built on being just wrong enough: You calmly state wild opinions. There are people who want to agree with any calm voice because they're seeking guidance in the storm of <insert hype cycle>. They invent a foothold in your wild statement, some sliver of truth they can squint and maybe almost make out. Then you gain a following, which then starts to add a social aspect: If I don't get it but this is a figure head, I must be looking it wrong. Now people are squinting harder. This repeats itself until everyone has their eyes closed following someone who has never actually said anything with any intention other than advancing their own influence. They don't care how many useful ideas die along the way, there's no intellectual curiosity to entice them to even stumble upon something more meaningful, it's just draining the energy out of what should be a truly rewarding time for self-thinking.
- gojomo 3y agoAs parent deleted, which tweet was being referenced?
- BoorishBears 3y agohttps://twitter.com/swyx/status/1697121327143150004 https://twitter.com/swyx/status/1697121327143150004 There was no need to delete except being so trivially shown to be wrong, I didn't chase them to twitter or something. But that's the MO for the tech grifter: - you herd the few people who are unsure and will listen to any confident voice - the people who know the most about <insert tech> tend to not like that, but when the herd is small just defer to their confrontations with humility and grace, and use that show of virtue to continue herding - the more people you herd, the easier it is to get incrementally smarter people to follow: We're all subject to certain blindspots in a large enough crowd - the more people who follow someone who's clearly wrong, the more annoyed people who are knowledgeable about <insert tech> will get about the grifter - This makes each future confrontation more heated, so now the heated nature of the confrontation is justification to disengage without deferring. Just be confidence and continue herding - rinse and repeat until people who don't follow the grifter gospel are a minority. — The actual VC dollars start chasing whatever story their ilk has weaved by then. And eventually it all collapses because there was no intellectual underlying: just self-enrichment. That realization from the crowd exhausts any good will that was left for <insert tech> and the grifters move on to the next bubble.
- gojomo 3y agoThanks for ref! I share your frustration at those who confidently & prematurely write-off rapidly-changing AI tech based on dated examples, cherry-picked anecdotes from the unskilled, & zero extrapolation based on momentum. They do a a double-disservice to those who trust them: 1st, by discouraging beneficial work on ripe, solvable challenges, and 2nd, encouraging a complacency about rapid new capabilities that may leave vulnerable people at the mercy of others who were better prepared. But, not being familiar with the account in question, I don't see those attitudes in that tweet. It seems more an assessment "no one has quite nailed this yet" than defeatism over whether it's possible.
- maxbond 3y ago> [deleted because don't want to be drawn into flamewar] Good on you. That is an inspiring demonstration of restraint.
- Madmallard 3y agopersonal tutoring and coaching is basically mandatory for mastery. name a professional concert pianist or athlete who doesn’t have one. I act as personal tutor for comp sci students and I’m envious of them. I didn’t have one and I think it really limited my growth.
- blibble 3y agofrom their own FAQ linked from this page: Is ChatGPT safe for all ages? ChatGPT is not meant for children under 13, and we require that children ages 13 to 18 obtain parental consent before using ChatGPT. so in other words: no it's grossly irresponsible to be pushing "Teaching with AI" in this scenario
- teacpde 3y agoTeaching doesn't prescribe students to be younger than 18.
- catchnear4321 3y agoyou act like parental consent wasn’t listed as a requirement. though it may not be broadly recognized as such, that requirement is an admission that it is foolish to hand a child access without guidance. you know, like in the form of a parent. parental guidance. which starts with parental consent. so in other words: it depends. it’s grossly irresponsible to treat a hammer as inherently dangerous.
- whywhywhywhy 3y agoCan’t imagine I’d have bothered engaging with any subject I wasn’t interested in if ChatGPT existed back then. Always remember the glorious few months when I had Encarta at home before too many students had it and before teachers clocked on where homework became just printing the page on the subject off after removing identifying bits.
- harry8 3y agoYou make a strong case about lack of education quality and make-work time-wasting foisted upon children. Education is not a problem the human race has solved despite progress made.
- waffletower 3y agoI disagree. Ethical teachers audit and examine all content they intend to be consumed by students -- it is their responsibility regardless of what medium or agents are used to create them. It is common for people to disregard that generative AI is currently a tool without agency whose use requires a selection process. Just as a camera needs to be aimed, AI does as well.
- minimaxir 3y agoI'm surprised OpenAI is encouraging large system-style prompts for the main ChatGPT webapp where they are less effective there. Now that the ChatGPT Playground is the default interface for the ChatGPT API with full system prompt customization, they should be encouraging more use there, with potential usage credits for educational institutions.
- jjcm 3y agoI've personally found AI to be a great help whenever I'm diving into a topic that I'm less familiar with. Recently I used it to help me prep for an interview as well. My partner uses it to help explain STEM concepts that she didn't cover in her schooling. I do wonder how far away we are from an actual Young Lady's Illustrated Primer. Three years ago I'd say we were 50 years away. Now it feels more like 10.
- shepardrtc 3y agoThe same for me, I love it for this sort of thing. I can bounce ideas off of it and it'll give me a solid response without getting tired of my questioning. And it'll explain in detail why I'm wrong. I really can't express how useful this is for my style of learning - I like to take things apart and figure out how they go back together.
- danielbln 3y agoThis is exactly it, this sort of infinitely patient tutor dialog interaction is absolutely perfect for my style of learning and I can't help but be a little bit sad that I didn't have something like that available during school. That being said, I'm beyond thrilled to have it available now, makes it easy to never stop learning.
- yoyohello13 3y agoI just don't know about this. I also find it answers great when I'm not familiar with a topic. However, when I am familiar with a topic I find all sorts of inconsistencies or wrong facts. I'm concerned the same inconsistencies are there in the topics I'm not familiar with, I just don't know enough about the subject to spot them.
- sunsetMurk 3y agoconceptually, I get it.
- jprete 3y agoI would expect the distribution of inconsistencies and errors to be the same for both sets of topics, in the absence of any other information.
- dtnewman 3y agoI built a chrome plugin called Revision History [1] that I released about 2.5 weeks ago, so I've been talking to a good number of educators about this recently. I'd say the majority of teachers are terrified of AI, because it means that they have to completely change how they teach with only a few months' notice. It's not easy to change your lesson plans or assignment structures that quickly and it'll take time to see where this all lands. Some teachers are looking for ways not to adapt, which is why there's a surge of interest in AI detection (which doesn't work well), but the sharpest educators I talk to are cognizant of the fact that there is no going back. So the plan is to incorporate AI into their curriculum and try to make assignments more "AI proof". This means more in-class work (e.g., the "flipped classroom" model [2]). Others are looking for ways to encourage students to use AI on assignments, but to revise and annotate what AI generates for them (this is what I am marketing my plugin for). Either way, it's going to be very rough over the next few years as educators scramble to keep us with a monstrous change that came about practically overnight. [1] https://www.revisionhistory.com https://www.revisionhistory.com. The plugin helps teachers see the students' process for drafting papers, unlike many than other plugins that are trying to be "AI detectors". [2] https://bokcenter.harvard.edu/flipped-classrooms#:~:text=A%20flipped%20classroom%20is%20structured,that%20involve%20higher%20order%20thinking https://bokcenter.harvard.edu/flipped-classrooms#:~:text=A%2....
- geek_at 3y agoGood job! Btw your site is exposing the .git directory https://www.revisionhistory.com/.git/config https://www.revisionhistory.com/.git/config Might want to set a filter rule for that
- imachine1980_ 3y agothere is any extension to do that or you manually do it, i have adhd so this type of tools save y life, this happens to me a few months back i kill the vm host and make new os, maybe too overboard but i works.
- geek_at 3y ago
- grozmovoi 3y agoThat's how I've been using it for a week now to clarify certain concepts from computer science that I always had little confidence in and it has been excellent.
- yura 3y agoCare to share an example? (the concept and how it was clarified)
- seydor 3y ago"Please teach our models how to replace you"
- elashri 3y agoChatGPT (and other LLMs) still cannot do (and probably never will) well in any consistent manner in physics. I don't think physics departments are worrying much about the AI. The only thing that can help students in a more reliable way is some coding projects. Which is okay because in most of these classes (computational physics) students are encouraged to work together, seek help and even ask on the internet (before ChatGPT…etc.) It was always about how to explain and describe the thinking. AI (at least in its current form) is very weak at the problem-solving aspects and in understanding concepts. On the other hand, as non-native English speaker, it save me much time into paraphrasing my poorly thoughts and writing that I would need an hour to express in a good formal manner. It can guide you in some aspects of coding tasks, introduce you to some APIs …etc. This is actually a good tool that I agree that a good student (researcher) would use wisely to gain some knowledge and save sometime. It will not help much with solving a cart on an inclined plane with some friction and a pendulum hanging from the cart. No, it will not be able to give you the normal modes. This is just a personal experience and opinion, though. It might be completely different in other areas.
- chaxor 3y agoYou should really think about making statements like "AI will probably never do X well". Many formal linguists made very strong statements about the impossibility of (__insert feature here__, such as pragmatic implicature) to be learned by AI, which they are now being shown to be wrong. For instance, Miles Cranmers work on using GNNs for symbolic regression is a start towards useful new discoveries in physics. Transformers are just GNNs with a specific message passing function and position embeddings. It's not hard to see that either by a different architecture, augmentation, or potentially even just more of the same, we can get to new discoveries in physics with AI. The GNN symbolic regression work is evidence that it's already happened. As for grounding knowledge in the LLMs we have exactly just this moment (a rather short-sighted view) there is plenty of interest and work in the area, for which I expect will be addressed in a multitude of ways. It's ability with grounded physics knowledge is not perfect, but it's very good w.r.t. the common knowledge of a human off the street. External sources alone make it much better, and that's just the exceedingly short-sighted analysis of what we have today.
- 3y ago
- dustincoates 3y agoI'm ambivalent on LLMs, but I have found one really good use for it: helping me with language learning. I'm now at a level (C1) with my second language that it's really difficult to find resources or even tutors to help refine it. So what I've been doing is chatting with Claude and asking it to correct whatever faults I make or asking it to give me exercises on things where I need to focus. For example, "Give me some exercises where I need to conjugate the past tense and choose the correct form." It's like a personal language learning treadmill.
- Miraste 3y agoI'm surprised languages aren't more of a focus in the LLM hype. They're like if Rosetta Stone ads were true. They translate at state of the art levels, but you can also give and ask for context, and they're trained on native resources and culture. There hasn't been a jump in machine translation this big and fast, ever.
- tkgally 3y agoI’m not sure if it reaches the level of hype, but in at least one country where English is not the dominant language—Japan, where I live—using LLMs for language learning is frequently mentioned in the press and elsewhere. Some educators are starting to use them in classes, too: https://mainichi.jp/english/articles/20230824/p2a/00m/0na/029000c https://mainichi.jp/english/articles/20230824/p2a/00m/0na/02...
- ChatGTP 3y agoPretty much all the major LLMs are trained on 99% English text, so it's good for people who want to learn English, not so good the other way around.
- hereforcomments 3y agoI agree, they are perfect for language learning. Reusing another project I put together a site as a weekend project mainly for myself to learn Dutch but decided to release it publicly for free (invite only) to see how others interact with it as it's almost free to run it and looks good in my portfolio. It's an audio chat app, it's not my idea, I've seen many of these but wanted to create one for my own needs. It uses Chrome's speech to text API (webspeechapi) for input, then it sends it to ChatGPT 3.5 and I use Google's text to speech API to generate audio response along with the text. So practically I can talk to ChatGPT. I also want to add a dictionary module where I can generate and add example sentences and images to a words and phrases. I'm sure there are complete teams working on apps like this as it's such a straightforward use case. You can try it, it's all Hungarian but you can translate the page. https://convo.hu/ https://convo.hu/ Invite code: CONVO
- binarymax 3y agoMy sister (who is a middle school teacher) and I developed a real training program for teachers, and this "guide" from OpenAI is quite underwhelming. It doesn't address 90% of the problems teachers actually face with AI...this is mostly a brochure on how to use ChatGPT to get info. If you are a teacher or know a teacher who is struggling to adapt this school year, I'd be honored to speak with them and see if we can help.
- miketery 3y agoCan you share some of the outline or problems your guide solves?
- binarymax 3y agoSure thing! https://max.io/teacher-training.html https://max.io/teacher-training.html
- fsloth 3y agoThe prompt 4 ”AI teacher” is pretty good for learning group theory at least. (Just trying it right now on ChatGPT 4.0)
- rmbyrro 3y agoI found lots of good value in their publication as well. Especially for teachers, who I believe (most at least) have no clue about prompt engineering and how to talk to an LLM.
- fsloth 3y agoIMO ’Prompt engineering’ is an implication that the LLM:s are really immature technology. There is no intrinsic value in prompt engineering - it’s ok to wait a bit until LLM:s get a proper product shell you don’t need to walk on eggshells over. I would not promote LLM:s as production ready offerings until this aspect becomes better. Using an LLM is like having a therapy session - where you the user are the therapist. Humans should not need to learn en masse become AI therapists, that’s a the inverse of what should happen :D
- wavesounds 3y agoThe elephant in the room here is that these LLM's still have problems with hallucinations. Even its only 1% or even 0.1% of the time thats still a huge problem. You could have someone go their whole lives believing something they were confidently taught by an AI which is completely wrong. Teachers should be very careful using a vanilla LLM for education without some kinds of extra guardrails or extra verification.
- chaxor 3y agoThis is also the case if taught by any educator who happens to trust the source they looked up as well. The internet, text books, and even scientific articles can all be factually incorrect. GNNs (for which LLMs are a subclass of) have a potential to be optimized in such a way that all the knowledge contained within them remains as parsimonious as possible. This is not the case for a human reading some internet article for which they have not gained extensive context within the field. There are plenty of people that strongly believe in strange ideas that were taught to them by some 4th grade teacher that was never corrected over their life. While you're statements are correct in this miniscule snapshot of time, it's exceedingly short-sighted to assert that language modeling is to be avoided due to some issues that exists this month, and disregard the clear future of improvements that will come very soon.
- cocoto 3y ago[flagged]
- pixl97 3y agoDamned, I'd have loved if my teachers only hallucinated 1% of the time. Instead we had the southern Baptist football coaches attempting to teach us science... poorly.
- jheriko 3y agoin my experience, its sometimes 100% of the time, even after repeated attempts to correct it with more specific prompts. Even on simple problems involving divisions or multiples of numbers from 1 to 10 with one additional operation.
- nirmel 3y agoI made https://anylearn.ai https://anylearn.ai, an education app built on OpenAI. if you click the settings icon, then the teach tab, it will generate a teaching guide on any topic. Try it!
- ineptitude 3y agoIf there was a tab with a code example when the lesson is related to programming, it would be perfect, as the chat doesn't detect markdown's code block.
- burkaman 3y agoI put in "8th grade french" and it gave me a guide on how to develop a teaching guide, not the teaching guide itself. Like "Step 4: Prepare instructional materials", "Step 5: sequence the lesson", etc., with generic instructions for each. The Test Questions tab has questions about my knowledge of lesson planning, not questions for French students. "College-level calculus" was similar, just vague generic high-level advice with no lesson plan or specific guide.
- nirmel 3y agoGood catch. Will modify the prompts to make it produce the desired content.
- azertykeys 3y agoAnecdotally, my friend who's just starting out teaching high school physics has used ChatGPT to generate worksheet questions with mixed results, having to throw out the majority of what it generates, but still saving time overall
- pixl97 3y agoJust asking it to make up 10 question isn't a great way of doing it most of the time. It turns out making a single question really is a bunch of different questions in itself. You have to ask on each question "How can this be misinterpreted", "Can the question be written better", "Is this a challenging question that actually causes a person to learn". A lot of human generated question are just confusing hot garbage in and of themselves. Quite often we encode cultural biases in the questions. Or, if a person actually knows about the topic they can get the question wrong, based if they are only supposed to formulate the answer based on the paragraph shown to the user. The AI Explained channel on youtube just had an episode about this in relation to the tests we're giving AI. Turns out a lot of the questions just suck.
- vouaobrasil 3y agoSoon to be: teachers ARE AI. You're all having fun now. But you'll regret using AI for anything because soon humans will become mostly fit for manual labour while AI concentrates the wealth of the world into the hands of the tech elite. Then, without a human connection in teaching, children will grow up into psychologically damaged adults.
- r3trohack3r 3y ago> Soon to be: teachers ARE AI. > soon humans will become mostly fit for manual labour > without a human connection in teaching, children will grow up into psychologically damaged adults. If humans are only going to be doing manual labor, what will the AI teacher be teaching? Do you need 16+ years of education for manual labor? Just taking your argument at face value, I don't understand how "AI replaces nearly all human knowledge workers" leads to "children become psychologically damaged adults." It seems like it would free them from being strapped into a chair for 16 years and denied the opportunity to be children in an attempt to prepare them for a life of knowledge work? Unless we just keep up the ruse of an entire childhood of classroom based education for ... reasons? To push past your argument, society and knowledge isn't zero sum. I'm not writing software because it's the single most important thing in the universe for me to focus on right now. It's actually pretty low on the list of important things on the grand scale of important things. I'm writing software because it's the work that needs to be done right now and there isn't a replacement for me doing it. I feel like you are asserting that plugging numbers into spreadsheets as an accountant or doing string transformations "at scale" to convert DB queries into HTML and JSON is both: 1) A fulfilling life 2) The only thing humans could possibly be doing of value right now; if you take this away there is nothing left There are a tonne of fundamental questions/problems about life, the universe, interstellar travel, preservation of our species, etc. that I _just don't have time for_ right now because I'm over here trying to figure out how to take these bytes coming over the wire from an SQL query and pack them into a JSON object so a browser can hydrate this bit of HTML. And I'm sorry, but, this isn't how I'd choose to live my life if there was someone else I could put in this seat. Please AI take my job so I can be free to focus on all of the stuff that comes with the next layer of abstraction/automation.
- 3y ago
- ajhai 3y ago> Building quizzes, tests, and lesson plans from curriculum materials Example prompts that OpenAI shared here are a great start. However I think these use-cases are better served as micro apps built on top of these prompts. For example, a teacher will keep coming back to use this prompt with same/similar set of responses most of the year. On top of that, enriching the context with additional information pulled from local sources will quickly become a need. ChatGPT's custom instructions will help with not having to repeat prompts but the interface falls short when it comes to repeat narrow use cases. This is where imo LLM apps shine. A simple app built with langchain or some low-code platforms and providing local data from a vector store can be super powerful. We recently open-sourced LLMStack (https://github.com/trypromptly/LLMStack https://github.com/trypromptly/LLMStack), a platform that allows users to build these micro apps to automate their workflows. Our goal is to make these workflows sharable so someone can download a yaml file for this prompt and chain and start using it in their job.
- j_gravestein 3y agoFinally OpenAI admits AI detectors are useless.
- tomlue 3y agosomebody write a textbook chunker that generates context from textbooks for LLMs to build anki cards please. Extra credit if you build a new anki that dynamically generates cards with different text and the same meaning to prevent answer memorization.
- lemmox 3y agoInteresting prompts! IME the quality of the answers the users give to the ChatGPT questions in these prompts will make or break the experience. I played around with this use case in the spring when my teenage daughter was looking for extra test prep materials. At first the experience was interesting but there was an "AI uncanny valley" shaped problem: the material just didn't seem to fit. It felt wrong. This uncanny valley was significantly reduced, even eliminated in some instances, by including the entirety of our school district's online material about the course; information about the core competencies (across communication, thinking, personal & societal), the big ideas, the curricular competency & content about the learning standards. Our district has a pretty good website with all of this information laid out for each course and grade level. Including all of this information in the prompt context resulted in relevant and harmonious content when asking to generate course outlines, student study-prep handouts, and even sample study session pre-tests (although ChatGPT wasn't strong at reliably creating answer sheets for the pre-tests). Context is key!
- lemmox 3y agoAn interesting trick I found here was to ask ChatGPT to produce tables of concept definitions and include a metaphor for each concept to help understanding. It was quite good at coming up with metaphors and that actually felt kind of magical.
- jheriko 3y ago[flagged]
- theprivacydad 3y agoThe main issue that is not addressed is that students need points to pass their subjects and get a high school diploma. LLMs are a magical shortcut to these points for many students, and therefore very tempting to use, for a number of normal reasons (time-shortage, laziness, fatigue, not comprehending, insecurity, parental pressure, status, etc.). This is the current, urgent problem with ChatGPT in schools that is not being addressed well. Anyone who has spent some time with ChatGPT knows that the 'show your work' (plan, outline, draft, etc.) argument is moot, because AI can retroactively produce all of these earlier drafts and plans.
- phalangion 3y agoI suspect it’s not being addressed well because it’s one of the fundamental challenges of school in the first place. For many, assessment and grades are the end goal, and any learning that happens is secondary.
- blueboo 3y agoThe status quo is a miserable mess, but consider “assessment and grades” is the best apparent evidence of ultimate-goal-“learning”. Is that not reasonable for people who pay for education to ask for? If it is reasonable, then the problem is likely the form of evidence and not its requirement se de
- Alupis 3y agoI can't be the only one thinking, given how much ChatGPT gets confidently wrong, that it's way too early to be talking about funneling this into classrooms? The internet is bursting with anecdotes of it getting basics wrong. From dates and fictional citations, to basic math questions... how on earth can this be a learning tool for those who are not wise enough to understand the limitations? OpenAI's examples include making lesson plans, tutoring, etc. Just like with self driving cars - too much too quick, and many are not capable of understanding the limits or blindly trust the system. ChatGPT isn't even a year old yet...
- spion 3y agoThe internet is sampling the interesting samples, not necessarily a realistic picture. I'd love to see a good research study on this that shows the actual error rate as well as a comparison with other non-human alternatives (e.g. googling, using textbook only, etc) as well as possibly human (personal tutor, group instructor, ...)
- Alupis 3y ago> The internet is sampling the interesting samples, not necessarily a realistic picture. A tutor is expected to know the subject and guide the student. If, say, 10% of the time it guides the student into a false understanding, the damages are significant. It's very hard to unlearn something, particularly when you have confidence you know it well. My personal adventures with ChatGPT are probably close to a 50% success rate. It gets some stuff entirely wrong, a lot of stuff non-obviously wrong, and even more stuff subtly wrong - and it's up to you to be knowledgeable enough to wade through the BS. Students, learning a subject in school are by definition not knowledgeable enough to discern confident BS from correctness. Will ChatGPT be useful in the future? Yes, almost certainly. But let's not rush this and get it very wrong. The consequences can be staggering in the education space - children or adults.
- spion 3y agoI'm getting north of 90% success with GPT4, and while a dedicated tutor or a group instructor would definitely be better, none of the other non-human alternatives come close. Searching the internet and youtube tutorials can also lead to wrong information and false understanding - all self-directed methods have this pitfall. ChatGPT, however, is the only one where you can probe deeper once you find problems. If I had to place it somewhere, it would be between a study buddy and a tutor, closer to a study buddy. Still - a well designed study will give us a much better picture of where we actually are. I think that would be extremely valuable.
- MostlyStable 3y agoSurprised not to see any discussion here of Khanmigo[0], which I believe has been using GPT-4 as a tutor for quite a while now (in a beta form). It's been long enough that I've actually been (idly) trying to find any efficacy data. I'm sure that by now Khan academy has it, but I haven't seen them release it anywhere. The famous tutoring 2-sigma result (referenced elsewhere in the comments), only took place over 6 weeks of learning, and Khanmigo should have over 6 months (I believe) of data by this point [0]https://www.khanacademy.org/khan-labs https://www.khanacademy.org/khan-labs
- rcarr 3y agoI don’t understand why they’ve made it US only during the beta period, seems like a weird decision.
- wrsh07 3y agoI imagine they're doing a lot of development and it can be hard to make sure new products are eg gdpr compatible (not to mention the fact that they might want to have it do different things in order to help students in other countries)
- uolmir 3y agoIt's not efficacy data exactly but there was a news article a while back with some interesting perspectives on implementation. https://www.nytimes.com/2023/06/26/technology/newark-schools-khan-tutoring-ai.html https://www.nytimes.com/2023/06/26/technology/newark-schools...
- globalnode 3y agoPlease ignore this comment, I'm only typing it here to get frustration with this topic off my chest. The talk and marketing from these places is all about improving humanity and using AI to benefit everyone whereas the reality is far from this. A few are benefiting and profiting right now, while open AI became closed AI.
- sillysaurusx 3y agoWell, it’s ok to vent about things like that as long as it’s substantive. One way to facilitate this kind of discussion is to remember that topics like this need to become more substantive, not less. As it stands there’s very little in your comment, let alone a substantive point about why OpenAI is bad. It’s hard to argue that OpenAI hasn’t helped pretty much everyone who’s used ChatGPT at all. That they do it for a profit doesn’t really change that equation much. Obviously I’d love to see them open source GPT-4 too, since it was trained on the world’s data. But they’re not compelled to.
- kashyapc 3y agoOf course they're "not compelled to". Why would they? The legal/regulatory "situation" allows them to get away at this point. I hope we fix this kind of pilfering soon. We're still navigating the sleaziness (or whatever you call it) involved here to train unfettered on open source and public data, and not meaningfully propagate the benefits back to the contributors. This is uncharted terrain as of now.
- lannisterstark 3y ago>while open AI became closed AI OpenAI was never Open though. Remember pre GPT-3 they had this whole "Omg we made something but its so dangerous we're never going to make it public" bs? https://techcrunch.com/2019/02/17/openai-text-generator-dangerous/?guccounter=1 https://techcrunch.com/2019/02/17/openai-text-generator-dang... >OpenAI built a text generator so good, it’s considered too dangerous to release
- Sparkyte 3y agoTeaching with AI sounds kinda meh. But I would like to see it in motion.
- mmcwilliams 3y agoSomething that's confused me about detector products is: presumably OpenAI stores all prompts and responses. Wouldn't it make sense for them to provide a detector service that checks against their own records? Sure, students could use another LLM but it seems like low-hanging fruit to at least offer that as a product to higher ed.
- userbinator 3y agoWho teaches the AI? I think that will become a mostly unnoticeable but persistent and growing problem in the coming decades. We currently have a huge amount of pre-AI knowledge to train them on, but in the future that could slowly get replaced by AI output with all its inaccuracies, which then gets used to train future AIs, and transitively the people they'll have taught, etc.
- thexumaker 3y agoHere’s my thing. It’s kind of like calculators and computers. It’s fine to use calculators later on because once you’ve mastered basic division you don’t need to do it 40 million more times. But if you gave calculators to first graders I struggle to believe they’re gonna be learning shit lol
- earth2mars 3y agoA better way to test whether student have learned or not is by looking at the question and answers that the student went through while interacting with the chat bot. Especially evaluating the follow-up questions that the student asked, which should depict whether how much he actually read and understood the information. So technically by feeding the q and a of the chat interaction on a subject that the student had with the chat bot should be fed back into chatbot to assess the student understanding and then probably conductive verbal and other forms of testing. Maybe writing on a paper. I have learned a great deal using these tools since they came. I think people are looking them the wrong way, and people just prompting and getting answers. Versus how much of the thought process that goes into it in understanding the concepts.
- nhinck2 3y agoChatGPT, write a believable interaction between a smart curious student and ChatGPT that demonstrates the student learning from the interaction.
- quadrifoliate 3y agoIt is reasonably trivial to weed out whether students are producing their own work, or are cheating using LLMs. Have the students produce their work in the classroom under examination-style conditions, with no electronic devices allowed in a classroom. Pens / pencils are allowed. Paper books can be allowed. No electronics. Let's see you write out that insightful essay about social hierarchy in Romeo and Juliet in longhand. Calculate the roots of that quadratic equation using pencil and paper, please. Explain the links between the Treaty of Versailles and World War II using what you learned in class, and a (paper) textbook for reference if you must. We have literally been doing this for hundreds of years. We were doing it when I was at school ~20 years ago. Any kid caught using a calculator in the classroom was told to put it away one time, or the teacher would confiscate it. Obviously, no laptops. The teachers used a blackboard and chalk; and the occasional slide projector. I don't completely understand why people act as if it's impossible to teach properly now that we have LLMs; but perhaps a general over-reliance on laptops and electronic devices in the classroom is the reason. As it is, kids (and adults) have a huge problem with screen time, so it would serve us well to get away from it.
- deleted 3y ago[deleted]
- euroderf 3y agoI see a general model here. In "learning mode" (information injection & assimilation) you can "gear up" all you want. In "testing mode" (info retrieval and presentation) you're on your own.
- tummycarter 3y agoGreat highly informative knowledge..!!
- TheCaptain4815 3y agoPersonally, using GPT4 has been absolutely invaluable to me when attempting to learn. I’m extremely jealous that kids these days have access to tools this powerful. The problem with these types of learning tools, same as with the Khan Academy one, is that they’re too “safe guarded” for general public release. Obviously needed for a public launch, but giving someone free reign of the models with some prompt teaching and an understanding of hallucinations could help kids learn so much more efficiently in the future.
- cfn 3y agoI don't get this whole commotion with LLMs and education. Isn't a Show & Tell and presencial classes enough to weed out the cheaters? LLM is a killer for homework but not when you are in a class with a teacher present.
- softbt 3y ago“Teaching students about critical thinking” This is an odd point. You’re already in school to critically think and why not just google it if you need to check its accuracy anyway
- hyperthesis 3y agoExams that allow calculators assess tasks harder than calculation. By analogy, if LLMs were allowed in an exam, what harder tasks would be assessed?
- truculent 3y agoLikewise, you learn how calculation works before being let loose on the calculator. Similarly, pilots must learn to fly under various regimes of automation, despite primarily using high-automation settings.
- iamleppert 3y agoThose who really desire to understand how things work will be undeterred by the temptation of AI. There are two types of people: those who care to know and really understand and those who don’t. Should we really force people, past a certain point, to care when it’s clear they don’t and are only doing something because they are forced to? I would argue that people should spend more time on the things they truly care about. That’s the critical difference; when you care about something and get enjoyment and satisfaction out of it, you want to understand all the fine details and have a thirst for knowledge and true insight. When you don’t care, you take the absolute shortest path so you can make time to do whatever it is that brings you true satisfaction. That’s perfectly okay with me because I do it all the time for things I couldn’t care less about. If someone who wants to be a software engineer can’t be bothered to learn and understand the fundamentals I’d argue that software engineering isn’t the discipline for them. The more you understand, the larger the surface area of the problem you have for which to explore further.
- carlotte 3y ago[dead]