3 ms·

Why do you think this rate is not acceptable? I would say it's more than acceptable, even as a single data point. If someone's submission comes up as positive o

Why do you think this rate is not acceptable? I would say it's more than acceptable, even as a single data point. If someone's submission comes up as positive on two separate occasions you've pretty much eliminated the chance of a false positive.

declaredapple 3y ago
I would agree with that it's usable as a single datapoint, used with others (other submissions, general performance, etc). However given that we already have professors literally failing people by just pasting and asking chatgpt, I'm not sure I'm comfortable with that.
FLT8 3y ago
Unless that person's writing style just happens to be very close to how the average human writes? If it's 0.01% for "any given human", I wonder what the numbers would look like for a human closer to average than usual?
wrs 3y ago
Only if the measures for an individual are uncorrelated, which there’s no reason to assume. It needs to be <= 0.01% false positive for each individual author. If it’s just that 0.01% of all tests are false positive, that leaves the possibility than a given individual might have anywhere up to 100% false positives.
Lerc 3y ago
I feel like 0.01% is one of the more dangerous levels of false positives. Widespread use would result in far more than 10,000 tests resulting in the near certainty that innocent people would be accused. The moderately low false positive rate would then be leaned on to imply guilt. You might find for every 10,000 tests you get around 151 targets. For each one individually you can say they probably did it, but cumulatively you can say that one of them is probably innocent. Consider using two positives. Do schools and universities generate 100,000,000 essays per year? Sure would suck for that innocent person tagged as having a one in a hundred million chance of not being guilty.
- google234123 3y ago
  Nothing is perfect, I wish more things had 1 in a hundred million certainty. Is our police this effective? Are the current plagiarism checks used in school this accurate?
  - int_19h 3y ago
    That kind of thing is easy to handwave away when it does not affect you personally. No, our justice system is probably not that effective either. But that is also a problem, so why should we create more problems like that instead of fixing the ones that we already have?
    - google234123 3y ago
      So we can’t replace current systems unless the replacement is absolutely perfect? How perfect is turn it in?
      - int_19h 3y ago
        If the current systems are even worse, we should throw those out. TurnItIn specifically is horrible and should have never been a thing
renewiltord 3y ago
If you ask people to write a program, maybe 5% of them write in Python. But if you ask them to give you two programs, then that's only 0.25%. Ask for seven programs and if they give you seven Python programs, they're one in a billion, the rarest of the rare.

declaredapple 3y ago

I would agree with that it's usable as a single datapoint, used with others (other submissions, general performance, etc). However given that we already have professors literally failing people by just pasting and asking chatgpt, I'm not sure I'm comfortable with that.

FLT8 3y ago

Unless that person's writing style just happens to be very close to how the average human writes? If it's 0.01% for "any given human", I wonder what the numbers would look like for a human closer to average than usual?

wrs 3y ago

Only if the measures for an individual are uncorrelated, which there’s no reason to assume. It needs to be <= 0.01% false positive for each individual author. If it’s just that 0.01% of all tests are false positive, that leaves the possibility than a given individual might have anywhere up to 100% false positives.

Lerc 3y ago

I feel like 0.01% is one of the more dangerous levels of false positives. Widespread use would result in far more than 10,000 tests resulting in the near certainty that innocent people would be accused. The moderately low false positive rate would then be leaned on to imply guilt. You might find for every 10,000 tests you get around 151 targets. For each one individually you can say they probably did it, but cumulatively you can say that one of them is probably innocent. Consider using two positives. Do schools and universities generate 100,000,000 essays per year? Sure would suck for that innocent person tagged as having a one in a hundred million chance of not being guilty.

google234123 3y ago
Nothing is perfect, I wish more things had 1 in a hundred million certainty. Is our police this effective? Are the current plagiarism checks used in school this accurate?
- int_19h 3y ago
  That kind of thing is easy to handwave away when it does not affect you personally. No, our justice system is probably not that effective either. But that is also a problem, so why should we create more problems like that instead of fixing the ones that we already have?
  - google234123 3y ago
    So we can’t replace current systems unless the replacement is absolutely perfect? How perfect is turn it in?
    - int_19h 3y ago
      If the current systems are even worse, we should throw those out. TurnItIn specifically is horrible and should have never been a thing

renewiltord 3y ago

If you ask people to write a program, maybe 5% of them write in Python. But if you ask them to give you two programs, then that's only 0.25%. Ask for seven programs and if they give you seven Python programs, they're one in a billion, the rarest of the rare.