4 ms·

We have 2 millions user generated forms. It is not possible for us to manually review all forms It's certainly possible. You could get reliable Mechanical Turk

by tlb 15y ago

We have 2 millions user generated forms. It is not possible for us to manually review all forms It's certainly possible. You could get reliable Mechanical Turk reviews of forms for $0.03 each, so $60k total. Or you could hire people to look at 500 forms / hour (phishing forms are instantly obvious) at $15 /hr, also $60k total. Compared to the cost of a site seizure, it might be a good investment.

pavel_lishin 15y ago
This seems obvious only in hindsight.
jrwoodruff 15y ago
Why is it everyone keeps making excuses for the secret service and godaddy? It DOESN'T MATTER whether it was possible or even financially viable for JotForm to review each and every form. The government shut down a legitimate business with no warning, court order or apparent process whatsoever. This is very, very wrong. Period.
- benatkin 15y ago
  It matters if your goal is simply to correct a statement that was stated as if it were a hard fact. That's all tlb seems to be doing in my reading of the comment. If your goal is only to figure out who's in the right I agree it doesn't matter.
- drusenko 15y ago
  GoDaddy shut down the website. The government complained about it. I see a very important distinction there.
cypherpunks01 15y ago
IMHO this isn't a good task for the Turks. If it's reasonable that a Turk might click 'Looks good to me!' most of the time, you'll gather a hoard of them that just click/script that same response 100% of the time and not look at the question. Of course you can include decoy questions to try and detect this behavior, but I just don't see the whole MTurk review process being terribly reliable overall. I just don't think the worker quality is that great, and they will do whatever possible to just churn through HITs (because they get paid zero for them).
- hessenwolf 15y ago
  Can't this be solved with different Turks reviewing the same form on occasion as quality control?

by tlb 15y ago

pavel_lishin 15y ago

This seems obvious only in hindsight.

jrwoodruff 15y ago

Why is it everyone keeps making excuses for the secret service and godaddy? It DOESN'T MATTER whether it was possible or even financially viable for JotForm to review each and every form. The government shut down a legitimate business with no warning, court order or apparent process whatsoever. This is very, very wrong. Period.

benatkin 15y ago
It matters if your goal is simply to correct a statement that was stated as if it were a hard fact. That's all tlb seems to be doing in my reading of the comment. If your goal is only to figure out who's in the right I agree it doesn't matter.
drusenko 15y ago
GoDaddy shut down the website. The government complained about it. I see a very important distinction there.

cypherpunks01 15y ago

IMHO this isn't a good task for the Turks. If it's reasonable that a Turk might click 'Looks good to me!' most of the time, you'll gather a hoard of them that just click/script that same response 100% of the time and not look at the question. Of course you can include decoy questions to try and detect this behavior, but I just don't see the whole MTurk review process being terribly reliable overall. I just don't think the worker quality is that great, and they will do whatever possible to just churn through HITs (because they get paid zero for them).

hessenwolf 15y ago
Can't this be solved with different Turks reviewing the same form on occasion as quality control?