4 ms·

The main issue with LLM powered learning is you need to have at least a surface level understanding of the topic to recognise if the output is wrong. That requ

The main issue with LLM powered learning is you need to have at least a surface level understanding of the topic to recognise if the output is wrong. That requires some basic knowledge, a specific mental model for interacting with LLM’s, and a modicum of good taste in code to be able to correct for the code smells that LLM’s fall victim to. You can never let the LLM take the wheel, you need to be the driver knowing what you want. This works if for example you know how to something in language x but need it in language Y, but it doesn’t work if you have never been able to solve the problem before. For something like leetcode however which is far from novel, there are that many correct answers in the training data that the risk is low.

creesch 2y ago
> you need to have at least a surface level understanding of the topic to recognise if the output is wrong. Often overlooked point. The only reason I even consider LLMs a useful tool for my work is because for the things I request from them I do have the prerequisites to validate their output.
dailykoder 2y ago
Very very good point and often understated imo. Yes, LLMs are okay-ish at javascript and python, but if I ask it to generate me some VHDL, it often just produces plain garbage. And without a basic understanding of hardware design you wouldn't even know what to look out for. It's just not possible to learn anything from it.
Joel_Mckay 2y ago
The classic "How Many R's in 'Strawberry'?" LLM test is amusing to students... Still useless for trying to understand subtle complex concepts not previously well-defined in the data sets. The numerous slop articles posted on YC that obviously were partially written with LLM help often makes the alleged authors sound like they had a stroke. lol =3
- sayonidroy 2y ago
  Agreed. But LLMs are pretty good at answering questions and helping students understand concepts from fixed datasets ie. courses. That is what we're trying to achieve at Edmigo
  - Joel_Mckay 2y ago
    Passively cramming information will not retain a lesson more than 3 months. Additionally, only 1/17 students have the discipline to succeed with self-directed studies. I remain skeptical the current well-formatted LLM nonsense will help students, and have concerns it may cause hindrance with long-term information recall. We shall soon see I guess... Best of luck, =3
    - blharr 2y ago
      Curious where you found the 1/17 statistic?
      - Joel_Mckay 2y ago
        It is an internal metric from a College that offers kids GED/remedial-high-school-level math prep classes for remote students. There are various reasons a student may be unable to physically attend classes, and remote 1-on-1 instruction is not in the budgets. The rate for curriculum completion is a very low 1:17 for self-directed study. The learning outcomes for students are often terrible for this path, but perhaps good if you charge for the tutorial service I guess. Attrition rates are not something institutions proudly promote in public. YC seems to down-vote anything that deals with a data-driven reality. lol =3 https://www.youtube.com/watch?v=GYh7smM6YpM https://www.youtube.com/watch?v=GYh7smM6YpM
        sayonidroy 2y ago
        Interesting points. but not very student can afford or has access to group studies or human tutors right?
ramraj07 2y ago
I am so glad I got my basics in before this AI era, I feel sorry for the folks who are just learning to code now. AI can do all the basic tasks and frameworks so it will probably feel so futile for them to try to learn it all by themselves.
- sayonidroy 2y ago
  I feel Programming is more about learning how to think and break problems down in smaller parts. That skill should always be useful
- kranner 2y ago
  I'm seeing this play out among young people I know who were trying to improve their English writing skills. Everything they post now in any formal or semi-formal context is via ChatGPT. Their self-confidence has taken a nosedive.
- dvngnt_ 2y ago
  i think textbooks are still good
- BoingBoomTschak 2y ago
  This. I don't know why, but I find it easy to imagine a future where real programmers who understand both the craft and the art from its lowest fundamentals to the highest levels of abstraction are the equivalent of https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel.
sayonidroy 2y ago
Very valid points. We've controlled these issues by providing content which we created ourselves and running other pipelines to validate the tutor's answers
HPsquared 2y ago
It's like the skill needed for calculator usage. You need to be able to quickly estimate what rough order of magnitude, first and last couple of digits, to check that the result "looks right". Using a calculator to do 100 quick sums is a different skillset to doing 10 by hand.

creesch 2y ago

> you need to have at least a surface level understanding of the topic to recognise if the output is wrong. Often overlooked point. The only reason I even consider LLMs a useful tool for my work is because for the things I request from them I do have the prerequisites to validate their output.

dailykoder 2y ago

Very very good point and often understated imo. Yes, LLMs are okay-ish at javascript and python, but if I ask it to generate me some VHDL, it often just produces plain garbage. And without a basic understanding of hardware design you wouldn't even know what to look out for. It's just not possible to learn anything from it.

Joel_Mckay 2y ago

The classic "How Many R's in 'Strawberry'?" LLM test is amusing to students... Still useless for trying to understand subtle complex concepts not previously well-defined in the data sets. The numerous slop articles posted on YC that obviously were partially written with LLM help often makes the alleged authors sound like they had a stroke. lol =3

sayonidroy 2y ago
Agreed. But LLMs are pretty good at answering questions and helping students understand concepts from fixed datasets ie. courses. That is what we're trying to achieve at Edmigo
- Joel_Mckay 2y ago
  Passively cramming information will not retain a lesson more than 3 months. Additionally, only 1/17 students have the discipline to succeed with self-directed studies. I remain skeptical the current well-formatted LLM nonsense will help students, and have concerns it may cause hindrance with long-term information recall. We shall soon see I guess... Best of luck, =3
  - blharr 2y ago
    Curious where you found the 1/17 statistic?
    - Joel_Mckay 2y ago
      It is an internal metric from a College that offers kids GED/remedial-high-school-level math prep classes for remote students. There are various reasons a student may be unable to physically attend classes, and remote 1-on-1 instruction is not in the budgets. The rate for curriculum completion is a very low 1:17 for self-directed study. The learning outcomes for students are often terrible for this path, but perhaps good if you charge for the tutorial service I guess. Attrition rates are not something institutions proudly promote in public. YC seems to down-vote anything that deals with a data-driven reality. lol =3 https://www.youtube.com/watch?v=GYh7smM6YpM https://www.youtube.com/watch?v=GYh7smM6YpM
      - sayonidroy 2y ago
        Interesting points. but not very student can afford or has access to group studies or human tutors right?

ramraj07 2y ago

I am so glad I got my basics in before this AI era, I feel sorry for the folks who are just learning to code now. AI can do all the basic tasks and frameworks so it will probably feel so futile for them to try to learn it all by themselves.

sayonidroy 2y ago
I feel Programming is more about learning how to think and break problems down in smaller parts. That skill should always be useful
kranner 2y ago
I'm seeing this play out among young people I know who were trying to improve their English writing skills. Everything they post now in any formal or semi-formal context is via ChatGPT. Their self-confidence has taken a nosedive.
dvngnt_ 2y ago
i think textbooks are still good
BoingBoomTschak 2y ago
This. I don't know why, but I find it easy to imagine a future where real programmers who understand both the craft and the art from its lowest fundamentals to the highest levels of abstraction are the equivalent of https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel.

sayonidroy 2y ago

Very valid points. We've controlled these issues by providing content which we created ourselves and running other pipelines to validate the tutor's answers

HPsquared 2y ago

It's like the skill needed for calculator usage. You need to be able to quickly estimate what rough order of magnitude, first and last couple of digits, to check that the result "looks right". Using a calculator to do 100 quick sums is a different skillset to doing 10 by hand.