7 ms·

My daughter was accused of turning in an essay written by AI because the school software at her online school said so. Her mom watched her write the essay. I th

by jmugan 2y ago

My daughter was accused of turning in an essay written by AI because the school software at her online school said so. Her mom watched her write the essay. I thought it was common knowledge that it was impossible to tell whether text was generated by AI. Evidently, the software vendors are either ignorant or are lying, and school administrators are believing them.

clipsy 2y ago
> Evidently, the software vendors are either ignorant or are lying I’ll give you a hint: they’re not ignorant.
add-sub-mul-div 2y ago
Imagine how little common knowledge there will be one or two generations down the road after people decide they no longer need general thinking skills, just as they've already decided calculators free them from having to care about arithmetic skills.
- arkh 2y ago
  We don't learn directions now: we use GPS. We don't do calculations: computers do it for us. We don't accumulate knowledge: we trust Google to give us the information when needed. Everything in a small package everyone can wear all day long. We're at the second step of transhumanism.
  - hyperbrainer 2y ago
    At least the first 2 are far more accurate than humans ever could be. The third, i.e. trusting others to vet and find the correct information, is the problem.
    - hhh 2y ago
      Why? We've done it for ages, most trust in Wikipedia, and before most trusted in encyclopedias. Books written by others have been used forever. We just shift where we place the trust over time.
      - hyperbrainer 2y ago
        Agreed, but google hardly gives you those results. Sponsored Ads and AI generated seo crap is hardly an encylopedia.
      - ClumsyPilot 2y ago
        I just googled ‘do I need a license to drive a power boat in UK’ I got AI answer saying ‘no’, but actually you do. If I use a calculator it will be correct. If I open encyclopaedia it will mostly be correct, because someone with a brain did at least 5 minutes of thining. We are not talking about some minor detail, AI makes colossal errors with great confidence and conviction.
        pennaMan 2y ago
        Try that query in perplexity :) Spoilers: it gets it right and explains the nuances.
        fullstackchris 2y ago
        But you're comparing apples to oranges anyway... a mathematical problem is vastly different than a q&a problem - which of course involves language which is anyway a lossy form of communication.
        hyperbrainer 2y ago
        that is the point. google is not Multivac
    - ben_w 2y ago
      Almost. GPS is great at knowing where you are, but directions are much much harder, and the extra difficulty is why the first version of Apple Maps was widely ridiculed. Even now, I find it's a mistake to just assume Google Maps can direct me around Berlin public transport better than my own local knowledge — sometimes it can, sometimes it can't. (But yes, a single original Pi Zero beats all humans combined at arithmetic even if all of us were at the level of the world record holder).
      - DrammBA 2y ago
        When I visit a new city I trust google maps more than I trust myself with a paper map, it even knows all public transport routes and times, and can guide me through connecting different types of public transports (e.g.: bus + train) to get to my destination quicker/cheaper, that would take me and a paper map quite a bit longer to plan.
        ben_w 2y ago
        I trust it in new places for the same reason. After I moved here and learned the system, I realised it had on my first trip directed me through a series of unnecessary train routes for a 5 minute walk. Last summer, when trying to find a specific named cafe a friend was at, Google Maps tried to have me walk 5 minutes to the train station behind me to catch the train to the stop in front of me to walk back to… the other side of the street because I hadn't recognised the sign. It's a great tool, fantastic even, but it still doesn't beat local knowledge. And very occasionally, invisibly unless you hit the edge, the map isn't correctly joined at the nodes and you can spot the mistake even as a first time visitor.
    - arkh 2y ago
      > trusting others to vet and find the correct information, is the problem To be honest, we do for most things: I have not checked the speed of light. And I surely would not be able to implement a way to measure it from only my observations and experience.
      - hyperbrainer 2y ago
        Agreed, but google hardly gives you those results. Sponsored Ads and AI generated seo crap is hardly an encylopedia
- gosub100 2y ago
  It's more insidious than that. AI will be used as a liability shield/scapegoat, so will become more prevalent in the workplace. So in order to not be homeless, more people will be forced to turn their brains off.
- jampekka 2y ago
  Maybe not having to learn to write "properly" means more bandwidth for more general thinking? At least not having to care about arithmetic leaves more time to care about mathematics.
- Cthulhu_ 2y ago
  And yet, this fear is timeless; back when book printing was big, people were fearmongering that people would no longer memorize things but rely too much on books. But in hindsight it ended up becoming a force multiplier. I mean I'm skeptical about AI as well and don't like it, but I can see it becoming a force multiplier itself.
  - bigstrat2003 2y ago
    > people were fearmongering that people would no longer memorize things but rely too much on books... Posters here love to bring out this argument, but I think a major weakness is that those people wound up being right. People don't memorize things any more! I don't think it's fair to hold out as an example of fears which didn't come to pass, as they very much did come to pass.
    - slidehero 2y ago
      >People don't memorize things any more .... and it made no difference.
lithos 2y ago
AI does have things it does consistently wrong. Especially if you don't narrow down what it's allowed to grab from. The easiest for someone here to see is probably code generation. You can point at parts of it and go "this part is from a high-school level tutorial", "this looks like it was grabbed from college assignments", and "this is following 'clean code' rules in silly places"(like assuming a vector might need to be Nd, instead of just 3D).
newZWhoDis 2y ago
The education system in the US is broadly staffed by the dumbest people from every walk of life. If they could make it elsewhere, they would. I don’t expect this to be a popular take here, and most replies will be NAXALT fallacies, but in aggregate it’s the truth. Sorry, your retired CEO physics teacher who you loved was not a representative sample.
- deleted 2y ago
  [deleted]
- krick 2y ago
  It's not just USA, it's pretty much universal, as much as I've seen it. People like to pretend like it's some sort of noble profession, but I vividly remember having a conversation with recently graduated ex-classmates, where one of them was complaining that she failed to pass at every department she applied to, so she has no other choice than to apply for department of education (I guess? I don't know what is the name of the American equivalent of that thing: bachelor-level program for people who are going to be teachers). At that moment I felt suddenly validated in all my complaints about the system we just passed through.
  - Gud 2y ago
    In some countries teaching is a highly respected profession. Switzerland and Finland comes to mind.
    - llm_trw 2y ago
      You can't eat respect.
      - benjaminfh 2y ago
        In those places salary (and good public services) follows respect
        2y ago
Daz1 2y ago
>I thought it was common knowledge that it was impossible to tell whether text was generated by AI. Anyone who's been around AI generated content for more than five minutes can tell you what's legitimate and what isn't. For example this: https://www.maersk.com/logistics-explained/transportation-and-freight/2024/07/11/container-destuffing https://www.maersk.com/logistics-explained/transportation-an... is obviously an AI article.
- bryanrasmussen 2y ago
  >Anyone who's been around AI generated content for more than five minutes can tell you what's legitimate and what isn't. to some degree of accuracy.
- zeroonetwothree 2y ago
  It’s impossible to tell AI apart with 100% accuracy
- kreyenborgi 2y ago
  Obviously false, as LLMs parrot what they're trained on. Not that hard to get them to regurgitate Shakespeare or what have you.
  - Daz1 2y ago
    Sounds like a skill issue on your part
    - kreyenborgi 2y ago
      Let's test your skills as a plagiarism detector. Below are two paragraphs. One of them was written by an LLM, one by a human. I have only altered whitespace in order to make them scan the same. Can you tell which is which? How much would you bet that you are correct? A. The Pathfinder and The Deerslayer stand at the head of Cooper's novels as artistic creations. There are others of his works which contain parts as perfect as are to be found in these, and scenes even more thrilling. Not one can be compared with either of them as a finished whole. The defects in both of these tales are comparatively slight. They were pure works of art. B. The Pathfinder and The Deerslayer stand at the head of Cooper's novels as artistic creations. There are others of his works which contain parts as perfect as are to be found in these, and scenes even more thrilling. Not one can be compared with either of them as a finished whole. The defects in both of these tales are comparatively slight. They were pure works of art.
ffujdefvjg 2y ago
I expect there will be some legal disputes over this kind of thing pretty soon. As another comment pointed out: run the AI-detection software on essays from before ChatGPT was a thing to see how accurate these are. There's also the problem of autists having their essays flagged disproportionately, so you're potentially looking at some sort of civil rights violation.
teeray 2y ago
> I thought it was common knowledge that it was impossible to tell whether text was generated by AI. I think it is, however the dream among educators of an “AI detector” is so strong that they’re willing to believe “these guys are the ones that cracked the problem” over and over, when it’s not entirely true. They try it out themselves with some simple attempts and find that it mostly works and conclude the company’s claims are true. The problem though is that their tests are all trying to pass off AI-generated work as human-generated—not the other way around. Since these tools have a non-zero false positive rate, there will always exist some poor kid who slaved away on a 20-page term paper for weeks that gets popped for using AI. That kid has no recourse, no appeals—the school spent a lot of money on the AI detector, and you better believe that it’s right.
TuringNYC 2y ago
>> Evidently, the software vendors are either ignorant or are lying, and school administrators are believing them. This is what will eventually happen. Some component or provider deep in the stack will provide some answer and organizations will be sufficiently shrouded from hard decisions and be able to easily point to "the system." This happens all the time in the US. Addresses are changed randomly because some address verification system feedback was accepted w/o account owner approval -- call customer service and they say "the system said that your address isnt right", as if the system knows where i've been living for the past 5yrs better than me, better than the DMV, better than the deed on my house. If the error rate is low enough, people just accept it in the US. Then, it gets worse. Perhaps the error rate isnt low, just that it is high for a sub-group. Then you get to see how you rank in society. Ask brown people in 2003-2006 how fun it was to fly. If you have the wrong last name and zipcode combo in NYC suddenly you arent allowed to rent citibikes despite it operating on public land. The same will happen with this, unless there is some massive ACLU lawsuit which exposes and the damages will continue until there is a resolution. Quite possibly subtle features on language style will get used as triggers, probably unknowingly. People in the "in-group" who arent exposed will claim it is a fair system while others will be forced to defend themselves and have to provide the burden of proof on a blackbox.
chatmasta 2y ago
> Her mom watched her write the essay. I suspect there is a product opportunity here. It could be as simple as a chrome extension that records your sessions in google docs and generates a timelapse of your writing process. That’s the kind of thing that’s hard to fake and could convince an accuser that you really did write the essay. At the very least it could be useful insurance in case you’re accused.
- nathants 2y ago
  obs screen recording with laptop facecam.

by jmugan 2y ago

clipsy 2y ago

> Evidently, the software vendors are either ignorant or are lying I’ll give you a hint: they’re not ignorant.

add-sub-mul-div 2y ago

Imagine how little common knowledge there will be one or two generations down the road after people decide they no longer need general thinking skills, just as they've already decided calculators free them from having to care about arithmetic skills.

arkh 2y ago
We don't learn directions now: we use GPS. We don't do calculations: computers do it for us. We don't accumulate knowledge: we trust Google to give us the information when needed. Everything in a small package everyone can wear all day long. We're at the second step of transhumanism.
- hyperbrainer 2y ago
  At least the first 2 are far more accurate than humans ever could be. The third, i.e. trusting others to vet and find the correct information, is the problem.
  - hhh 2y ago
    Why? We've done it for ages, most trust in Wikipedia, and before most trusted in encyclopedias. Books written by others have been used forever. We just shift where we place the trust over time.
    - hyperbrainer 2y ago
      Agreed, but google hardly gives you those results. Sponsored Ads and AI generated seo crap is hardly an encylopedia.
    - ClumsyPilot 2y ago
      I just googled ‘do I need a license to drive a power boat in UK’ I got AI answer saying ‘no’, but actually you do. If I use a calculator it will be correct. If I open encyclopaedia it will mostly be correct, because someone with a brain did at least 5 minutes of thining. We are not talking about some minor detail, AI makes colossal errors with great confidence and conviction.
      - pennaMan 2y ago
        Try that query in perplexity :) Spoilers: it gets it right and explains the nuances.
      - fullstackchris 2y ago
        But you're comparing apples to oranges anyway... a mathematical problem is vastly different than a q&a problem - which of course involves language which is anyway a lossy form of communication.
        hyperbrainer 2y ago
        that is the point. google is not Multivac
  - ben_w 2y ago
    Almost. GPS is great at knowing where you are, but directions are much much harder, and the extra difficulty is why the first version of Apple Maps was widely ridiculed. Even now, I find it's a mistake to just assume Google Maps can direct me around Berlin public transport better than my own local knowledge — sometimes it can, sometimes it can't. (But yes, a single original Pi Zero beats all humans combined at arithmetic even if all of us were at the level of the world record holder).
    - DrammBA 2y ago
      When I visit a new city I trust google maps more than I trust myself with a paper map, it even knows all public transport routes and times, and can guide me through connecting different types of public transports (e.g.: bus + train) to get to my destination quicker/cheaper, that would take me and a paper map quite a bit longer to plan.
      - ben_w 2y ago
        I trust it in new places for the same reason. After I moved here and learned the system, I realised it had on my first trip directed me through a series of unnecessary train routes for a 5 minute walk. Last summer, when trying to find a specific named cafe a friend was at, Google Maps tried to have me walk 5 minutes to the train station behind me to catch the train to the stop in front of me to walk back to… the other side of the street because I hadn't recognised the sign. It's a great tool, fantastic even, but it still doesn't beat local knowledge. And very occasionally, invisibly unless you hit the edge, the map isn't correctly joined at the nodes and you can spot the mistake even as a first time visitor.
  - arkh 2y ago
    > trusting others to vet and find the correct information, is the problem To be honest, we do for most things: I have not checked the speed of light. And I surely would not be able to implement a way to measure it from only my observations and experience.
    - hyperbrainer 2y ago
      Agreed, but google hardly gives you those results. Sponsored Ads and AI generated seo crap is hardly an encylopedia
gosub100 2y ago
It's more insidious than that. AI will be used as a liability shield/scapegoat, so will become more prevalent in the workplace. So in order to not be homeless, more people will be forced to turn their brains off.
jampekka 2y ago
Maybe not having to learn to write "properly" means more bandwidth for more general thinking? At least not having to care about arithmetic leaves more time to care about mathematics.
Cthulhu_ 2y ago
And yet, this fear is timeless; back when book printing was big, people were fearmongering that people would no longer memorize things but rely too much on books. But in hindsight it ended up becoming a force multiplier. I mean I'm skeptical about AI as well and don't like it, but I can see it becoming a force multiplier itself.
- bigstrat2003 2y ago
  > people were fearmongering that people would no longer memorize things but rely too much on books... Posters here love to bring out this argument, but I think a major weakness is that those people wound up being right. People don't memorize things any more! I don't think it's fair to hold out as an example of fears which didn't come to pass, as they very much did come to pass.
  - slidehero 2y ago
    >People don't memorize things any more .... and it made no difference.

lithos 2y ago

AI does have things it does consistently wrong. Especially if you don't narrow down what it's allowed to grab from. The easiest for someone here to see is probably code generation. You can point at parts of it and go "this part is from a high-school level tutorial", "this looks like it was grabbed from college assignments", and "this is following 'clean code' rules in silly places"(like assuming a vector might need to be Nd, instead of just 3D).

newZWhoDis 2y ago

The education system in the US is broadly staffed by the dumbest people from every walk of life. If they could make it elsewhere, they would. I don’t expect this to be a popular take here, and most replies will be NAXALT fallacies, but in aggregate it’s the truth. Sorry, your retired CEO physics teacher who you loved was not a representative sample.

deleted 2y ago
[deleted]
krick 2y ago
It's not just USA, it's pretty much universal, as much as I've seen it. People like to pretend like it's some sort of noble profession, but I vividly remember having a conversation with recently graduated ex-classmates, where one of them was complaining that she failed to pass at every department she applied to, so she has no other choice than to apply for department of education (I guess? I don't know what is the name of the American equivalent of that thing: bachelor-level program for people who are going to be teachers). At that moment I felt suddenly validated in all my complaints about the system we just passed through.
- Gud 2y ago
  In some countries teaching is a highly respected profession. Switzerland and Finland comes to mind.
  - llm_trw 2y ago
    You can't eat respect.
    - benjaminfh 2y ago
      In those places salary (and good public services) follows respect
      - 2y ago

Daz1 2y ago

>I thought it was common knowledge that it was impossible to tell whether text was generated by AI. Anyone who's been around AI generated content for more than five minutes can tell you what's legitimate and what isn't. For example this: https://www.maersk.com/logistics-explained/transportation-and-freight/2024/07/11/container-destuffing https://www.maersk.com/logistics-explained/transportation-an... is obviously an AI article.

bryanrasmussen 2y ago
>Anyone who's been around AI generated content for more than five minutes can tell you what's legitimate and what isn't. to some degree of accuracy.
zeroonetwothree 2y ago
It’s impossible to tell AI apart with 100% accuracy
kreyenborgi 2y ago
Obviously false, as LLMs parrot what they're trained on. Not that hard to get them to regurgitate Shakespeare or what have you.
- Daz1 2y ago
  Sounds like a skill issue on your part
  - kreyenborgi 2y ago
    Let's test your skills as a plagiarism detector. Below are two paragraphs. One of them was written by an LLM, one by a human. I have only altered whitespace in order to make them scan the same. Can you tell which is which? How much would you bet that you are correct? A. The Pathfinder and The Deerslayer stand at the head of Cooper's novels as artistic creations. There are others of his works which contain parts as perfect as are to be found in these, and scenes even more thrilling. Not one can be compared with either of them as a finished whole. The defects in both of these tales are comparatively slight. They were pure works of art. B. The Pathfinder and The Deerslayer stand at the head of Cooper's novels as artistic creations. There are others of his works which contain parts as perfect as are to be found in these, and scenes even more thrilling. Not one can be compared with either of them as a finished whole. The defects in both of these tales are comparatively slight. They were pure works of art.

ffujdefvjg 2y ago

I expect there will be some legal disputes over this kind of thing pretty soon. As another comment pointed out: run the AI-detection software on essays from before ChatGPT was a thing to see how accurate these are. There's also the problem of autists having their essays flagged disproportionately, so you're potentially looking at some sort of civil rights violation.

teeray 2y ago

> I thought it was common knowledge that it was impossible to tell whether text was generated by AI. I think it is, however the dream among educators of an “AI detector” is so strong that they’re willing to believe “these guys are the ones that cracked the problem” over and over, when it’s not entirely true. They try it out themselves with some simple attempts and find that it mostly works and conclude the company’s claims are true. The problem though is that their tests are all trying to pass off AI-generated work as human-generated—not the other way around. Since these tools have a non-zero false positive rate, there will always exist some poor kid who slaved away on a 20-page term paper for weeks that gets popped for using AI. That kid has no recourse, no appeals—the school spent a lot of money on the AI detector, and you better believe that it’s right.

TuringNYC 2y ago

>> Evidently, the software vendors are either ignorant or are lying, and school administrators are believing them. This is what will eventually happen. Some component or provider deep in the stack will provide some answer and organizations will be sufficiently shrouded from hard decisions and be able to easily point to "the system." This happens all the time in the US. Addresses are changed randomly because some address verification system feedback was accepted w/o account owner approval -- call customer service and they say "the system said that your address isnt right", as if the system knows where i've been living for the past 5yrs better than me, better than the DMV, better than the deed on my house. If the error rate is low enough, people just accept it in the US. Then, it gets worse. Perhaps the error rate isnt low, just that it is high for a sub-group. Then you get to see how you rank in society. Ask brown people in 2003-2006 how fun it was to fly. If you have the wrong last name and zipcode combo in NYC suddenly you arent allowed to rent citibikes despite it operating on public land. The same will happen with this, unless there is some massive ACLU lawsuit which exposes and the damages will continue until there is a resolution. Quite possibly subtle features on language style will get used as triggers, probably unknowingly. People in the "in-group" who arent exposed will claim it is a fair system while others will be forced to defend themselves and have to provide the burden of proof on a blackbox.

chatmasta 2y ago

> Her mom watched her write the essay. I suspect there is a product opportunity here. It could be as simple as a chrome extension that records your sessions in google docs and generates a timelapse of your writing process. That’s the kind of thing that’s hard to fake and could convince an accuser that you really did write the essay. At the very least it could be useful insurance in case you’re accused.

nathants 2y ago
obs screen recording with laptop facecam.