6 ms·

As someone already said, parents used to be concerned that kids wouldn't be able to solve maths problems without a calculator, and it's the same problem, but th

by Stubbs 1y ago

As someone already said, parents used to be concerned that kids wouldn't be able to solve maths problems without a calculator, and it's the same problem, but there's a difference between solving problems _with_ LLMs, and having LLMs solve it _for you_. I don't see the former as that much of a problem.

9rx 1y ago
> there's a difference between solving problems _with_ LLMs, and having LLMs solve it _for you_. If there is a difference, then fundamentally LLMs cannot solve problems for you. They can only apply transformations using already known operators. No different than a calculator, except with exponentially more built-in functions. But I'm not sure that there is a difference. A problem is only a problem if you recognize it, and once you recognize a problem then anything else that is involved along the way towards finding a solution is merely helping you solve it. If a "problem" is solved for you, it was never a problem. So, for each statement to have any practical meaning, they must be interpreted with equivalency.
- kevindamm 1y ago
  There is a difference between thinking about the context of a problem and "critical thinking" about the problem or its possible solutions. There is a measurable decrease in critical thinking skills when people consistently offload the thinking about a problem to an LLM. This is where the primary difference is between solving problems with an LLM vs having it solved for you with an LLM. And, that is cause for concern. Two studies on impact of LLMs and generative AI on critical thinking: https://www.mdpi.com/2075-4698/15/1/6 https://www.mdpi.com/2075-4698/15/1/6 https://slejournal.springeropen.com/articles/10.1186/s40561-024-00316-7 https://slejournal.springeropen.com/articles/10.1186/s40561-...
shinycode 1y ago
Well the extent is much broader from a calculator vs an LLM. Why should I hire you if an agent can do it ? LLM is every job is a calculator and can be replaced. Spotify CEO stated on X that before asking for more headcount they have to justify not being able to do the job with an agent. So all the students who let the LLM do their assignment and learn basically nothing, what’s their value for a company to be hired ? The company will and is just using the agent as well …
- jpc0 1y ago
  > Why should I hire you if an agent can do it ? You as the employer are liable, a human has real reasoning abilities and real fears about messing up, the likely hood of them doing something absurd like telling a customer that a product is 70% off and them not losing their job is effectively nil. What are you going to do with the LLM, fire it? Data scientist and people deeply familiar with LLMs to the point that they could fine tune a model to your use case cost significantly more than a low skilled employee and depending on liability just running the LLM may be cheaper. As an accounting firm ( one example from above ) far as I know in most jurisdictions the accountant doing the work is personally liable, who would be liable in the case of the LLM? There is absolutely a market for LLM augmented workforces, I don't see any viable future even with SOTA models right now for flat out replacing a workforce with them.
  - shinycode 1y ago
    I fully agree with you about liability. I was advocating for the other point of view. Some people argue that it doesn’t matter if there is mistakes (it depends which actually) and with time it will cost nothing. I argue that if we give up learning and let LLM do the assignments then what is the extent of my knowledge and value to be hired in the first place ? We hired a developper and he did everything with chatGPT, all the code and documentation he wrote. First it was all bad because from the infinity of answers chatGPT is not pinpointing the best in every case. But does he have enough knowledge to understand what he did was bad ? And then we need people with experience that confronted themselves with hard problems and found their way out. How can we confront and critic an LLM answer otherwise ? I feel student’s value is diluted to be at the mercy of companies providing the LLM and we might loose some critical knowledge / critical thinking in the process from the students.
    - jpc0 1y ago
      I agree entirely on your take regarding education. I feel like there is a place where LLMs are useful but doesn't impact learning but it's definitely not in the "discovery" phase of learning. However I really don't need to implement some weird algorithms myself every time (ideally I am using a well tested Library) but the point is that you learn to be able to but also to be able to modify or compose the algorithm in ways the LLM couldn't easily do.
UncleMeat 1y ago
Almost none of the cheaters appear to be solving problems with LLMs. All my faculty friends are getting large portions of their class clearly turning in "just copied directly from ChatGPT" responses.
- pc86 1y ago
  It's an issue in grad school as well. You'll have an online discussion where someone submits 4 paragraphs of not-quite-eloquent prose with that AI "stink" on it. You can't be sure but it definitely makes your spidey sense tingle a bit. Then they're on a video call and their vocabulary is wildly different, or they're very clearly a recent immigrant and struggle with basic sentence structure such that there is absolutely zero change their discussion forum persona is actually who they are. This has happened at least once in every class, and invariably the best classes in terms of discussion and learning from other students are the ones where the people using AI to generate their answers are failed or drop the course.
SkyBelow 1y ago
>As someone already said, parents used to be concerned that kids wouldn't be able to solve maths problems without a calculator Were they wrong? People who rely too much on a calculator don't develop strong math muscles that can be used in more advanced math. Identifying patterns in numbers and seeing when certain tricks can be used to solve a problem (verses when they just make a problem worse) is a skill that ends up being beyond their ability to develop.
- Suppafly 1y ago
  >Were they wrong? People who rely too much on a calculator don't develop strong math muscles that can be used in more advanced math. Yes. People who rely too much on a calculator weren't going to be doing advanced math anyway.
- kobenni 1y ago
  Yes, they were wrong. Many young kids who are bad at mental calculations are later competent at higher mathematics and able to use it. I don't understand what patterns and tricks you're referring to, but if they are important for problems outside of mental calculations, then you can also learn about them by solving these problems directly.

by Stubbs 1y ago

9rx 1y ago

> there's a difference between solving problems _with_ LLMs, and having LLMs solve it _for you_. If there is a difference, then fundamentally LLMs cannot solve problems for you. They can only apply transformations using already known operators. No different than a calculator, except with exponentially more built-in functions. But I'm not sure that there is a difference. A problem is only a problem if you recognize it, and once you recognize a problem then anything else that is involved along the way towards finding a solution is merely helping you solve it. If a "problem" is solved for you, it was never a problem. So, for each statement to have any practical meaning, they must be interpreted with equivalency.

kevindamm 1y ago
There is a difference between thinking about the context of a problem and "critical thinking" about the problem or its possible solutions. There is a measurable decrease in critical thinking skills when people consistently offload the thinking about a problem to an LLM. This is where the primary difference is between solving problems with an LLM vs having it solved for you with an LLM. And, that is cause for concern. Two studies on impact of LLMs and generative AI on critical thinking: https://www.mdpi.com/2075-4698/15/1/6 https://www.mdpi.com/2075-4698/15/1/6 https://slejournal.springeropen.com/articles/10.1186/s40561-024-00316-7 https://slejournal.springeropen.com/articles/10.1186/s40561-...

shinycode 1y ago

Well the extent is much broader from a calculator vs an LLM. Why should I hire you if an agent can do it ? LLM is every job is a calculator and can be replaced. Spotify CEO stated on X that before asking for more headcount they have to justify not being able to do the job with an agent. So all the students who let the LLM do their assignment and learn basically nothing, what’s their value for a company to be hired ? The company will and is just using the agent as well …

jpc0 1y ago
> Why should I hire you if an agent can do it ? You as the employer are liable, a human has real reasoning abilities and real fears about messing up, the likely hood of them doing something absurd like telling a customer that a product is 70% off and them not losing their job is effectively nil. What are you going to do with the LLM, fire it? Data scientist and people deeply familiar with LLMs to the point that they could fine tune a model to your use case cost significantly more than a low skilled employee and depending on liability just running the LLM may be cheaper. As an accounting firm ( one example from above ) far as I know in most jurisdictions the accountant doing the work is personally liable, who would be liable in the case of the LLM? There is absolutely a market for LLM augmented workforces, I don't see any viable future even with SOTA models right now for flat out replacing a workforce with them.
- shinycode 1y ago
  I fully agree with you about liability. I was advocating for the other point of view. Some people argue that it doesn’t matter if there is mistakes (it depends which actually) and with time it will cost nothing. I argue that if we give up learning and let LLM do the assignments then what is the extent of my knowledge and value to be hired in the first place ? We hired a developper and he did everything with chatGPT, all the code and documentation he wrote. First it was all bad because from the infinity of answers chatGPT is not pinpointing the best in every case. But does he have enough knowledge to understand what he did was bad ? And then we need people with experience that confronted themselves with hard problems and found their way out. How can we confront and critic an LLM answer otherwise ? I feel student’s value is diluted to be at the mercy of companies providing the LLM and we might loose some critical knowledge / critical thinking in the process from the students.
  - jpc0 1y ago
    I agree entirely on your take regarding education. I feel like there is a place where LLMs are useful but doesn't impact learning but it's definitely not in the "discovery" phase of learning. However I really don't need to implement some weird algorithms myself every time (ideally I am using a well tested Library) but the point is that you learn to be able to but also to be able to modify or compose the algorithm in ways the LLM couldn't easily do.

UncleMeat 1y ago

Almost none of the cheaters appear to be solving problems with LLMs. All my faculty friends are getting large portions of their class clearly turning in "just copied directly from ChatGPT" responses.

pc86 1y ago
It's an issue in grad school as well. You'll have an online discussion where someone submits 4 paragraphs of not-quite-eloquent prose with that AI "stink" on it. You can't be sure but it definitely makes your spidey sense tingle a bit. Then they're on a video call and their vocabulary is wildly different, or they're very clearly a recent immigrant and struggle with basic sentence structure such that there is absolutely zero change their discussion forum persona is actually who they are. This has happened at least once in every class, and invariably the best classes in terms of discussion and learning from other students are the ones where the people using AI to generate their answers are failed or drop the course.

SkyBelow 1y ago

>As someone already said, parents used to be concerned that kids wouldn't be able to solve maths problems without a calculator Were they wrong? People who rely too much on a calculator don't develop strong math muscles that can be used in more advanced math. Identifying patterns in numbers and seeing when certain tricks can be used to solve a problem (verses when they just make a problem worse) is a skill that ends up being beyond their ability to develop.

Suppafly 1y ago
>Were they wrong? People who rely too much on a calculator don't develop strong math muscles that can be used in more advanced math. Yes. People who rely too much on a calculator weren't going to be doing advanced math anyway.
kobenni 1y ago
Yes, they were wrong. Many young kids who are bad at mental calculations are later competent at higher mathematics and able to use it. I don't understand what patterns and tricks you're referring to, but if they are important for problems outside of mental calculations, then you can also learn about them by solving these problems directly.