I dislike bad faith questions or tests. I took part in a Facebook recruitment competition (this was many years ago when it was less obvious how evil they were

I dislike bad faith questions or tests. I took part in a Facebook recruitment competition (this was many years ago when it was less obvious how evil they were going to be). They gave us some data and some notional analytics task and your goal was to maximise a certain prediction metric. The catch was that they had errors/repetitions in the data such that you could exploit it to maximise the prediction score by methods largely unconnected to the notional analytics. This allowed some to get scores miles above accuracy reported in the literature, to the point where it wasn't physically possible to do that well. I'm still not convinced it wasn't an error on their part or that it wasn't an inside job. I discovered this and basically made the claim that some of the other scores others were getting weren't possible, and in the process of chasing down what the "cheaters" were doing, they came clean with the "oh, it's part of the job to assume/find such problems in the data and exploit it". So then everyone changed their models to include this little hack and the competition continued on the remainder of the data after wasting most of our time. but it left a really bad taste in my mouth. first of all, the metric is now meaningless in this case, so no, it's not an analysts job to exploit errors to maximise a meaningless metric. secondly, just do things in good faith: if you give me a data set and tell me it's a certain thing, I'm going to believe you. if it's dodgy or you're interested in OTHER things, then tell me so we can stop fucking around, because this isn't real life and I've got no idea what contrived thing it is you're looking for in this finite and time restricted exercise. if you want me to find some contrived thing, then at least tell me that's part of it. gotchas where they're fucking with you or dishonest about what they're actually doing in the process are the worst. especially if they're actually incompetent, made a mistake and they're too proud to admit that fact.