28 ms·
It could be detected like so: 1. Give a sample of mechanical turk workers the same data to classify without telling them 2. Once submitted, look carefully at
by nomilk 3y ago
It could be detected like so:
1. Give a sample of mechanical turk workers the same data to classify without telling them
2. Once submitted, look carefully at the ones they get wrong
3. If the turk worker gets similar ones wrong to other turk workers, they're probably not cheating
4. If the turk worker is frequently wrong on questions other turk workers all got right (e.g. if 99.9% of other workers got it right) then they've likely used AI to generate their answers and then fiddled around with a few afterwards to make it look human
That presumes they're fiddling with a few responses for the human touch. If they're submitting raw AI output that should be easily discoverable by running similar models against the same data (like detection of chess cheating compares games against moves form several engines, not just one).
I think that would work, unless everybody uses AI. If that latter were suspected, get a group to label data in person (where you can be 100% sure they can't cheat), then use those answers as the basis for comparison.
- jvm___ 3y agoWhat's the word for maps? City map makers didn't want people to copy their hard work, so they'd add a short street or some other feature that doesn't actually exist in reality.
- ceejayoz 3y agohttps://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street
- nomilk 3y ago> Mapmakers deliberately put slight mistakes in their maps so they can tell when someone copies them. If another map has the same mistake, that's very convincing evidence. http://www.paulgraham.com/say.html http://www.paulgraham.com/say.html
- deleted 3y ago[deleted]
- mahathu 3y ago> What scares me is that there are moral fashions too. They're just as arbitrary, and just as invisible to most people. But they're much more dangerous. Fashion is mistaken for good design; moral fashion is mistaken for good. What dog whistling drivel. That dude's trying so hard to use fancy words and walls of texts to defend his racist views without stating them outright. This is coming from a person who previously said: "One quality that's a really bad indication is a CEO with a strong foreign accent. They must be clueless if they haven't gotten rid of their strong accent." Already going mask off in public and later doubling down on it in his blog. Just imagine what his views are in private if he goes to those lengths to try and fail to morally justify them.
- edgyquant 3y ago“That dude,” is the owner of this website and not everything is bad or racist just because it goes against your view of the world. “Dog whistle,” give me a break.
- sezycei 3y agoWhat's the source on that quote? I searched it and this thread is the only result. Edit: I found the context around it, at least. Seems perfectly reasonable. [1] "Recently Inc published an interview in which I said we'd noticed a correlation between founders having very strong foreign accents and their companies doing badly. Some interpreted this statement as xenophobic, or even racist—as if I'd said that having a foreign accent at all was a problem. But that's not what I said, or what I think. No one in Silicon Valley would think that. A lot of the most successful founders here speak with accents. The case I was talking about is when founders have accents so strong that people can't understand what they're saying. I.e. the problem is not the cultural signal accents send, but the practical difficulty of getting a startup off the ground when people can't understand you." [1]: http://www.paulgraham.com/accents.html http://www.paulgraham.com/accents.html
- mahathu 3y agohttps://www.inc.com/magazine/201309/issie-lapowsky/how-paul-graham-became-successful.html https://www.inc.com/magazine/201309/issie-lapowsky/how-paul-...
- z3ugma 3y ago"paper towns" ? https://en.wikipedia.org/wiki/Fictitious_entry#Maps https://en.wikipedia.org/wiki/Fictitious_entry#Maps
- coffeebeqn 3y agoThe thing is that the only reason mturk exists is that the work providers can get cheap as dirt “labor” to do very menial tasks. Having to build all those guardrails would defeat the purpose. The cost per task is some cents, having to pay even one engineer would 1000x the cost
- Izkata 3y ago> 3. If the turk worker gets similar ones wrong to other turk workers, they're probably not cheating Per a comment above, that for a long time now they've been sharing scripts to get the same answer so they can get paid, this indicates cheating.