3 ms·
There's two important things here to consider when reading: 1) O3 cheated by using Google search. This is both against the rules of the game and OP didn't use
by godelski 2y ago
There's two important things here to consider when reading:
1) O3 cheated by using Google search. This is both against the rules of the game and OP didn't use search either
2) OP was much quicker. They didn't record their time but if their final summary is accurate then they were much faster.
It's an apples to oranges comparison. They're both fruit and round, but you're ignoring obvious differences. You're cherry picking.
The title is fraudulent as you can't make a claim like that when one party cheats.
I would find it surprising if OP didn't know these rules considering their credentials. Doing this kind of clickbait completely undermines a playful study like this.
Certainly O3 is impressive, but by over exaggerating its capabilities you taint any impressive feats with deception. It's far better to under sell than over sell. If it's better than expected people are happier, even if the thing is crap. But if you over sell people are angry and feel cheated, even if the thing is revolutionary. I don't know why we insist on doing this in tech, but if you're wondering why so many people hate "tech bros", this is one of the reasons. There's no reason to lie here either! Come on! We can't just normalize this behavior. It's just creating a reasonable expectation for people to be distrusting of technology and anything tech people say. It's pretty fucked up. And no, I don't think "it's just a blog post" makes it any better. It makes it worse, because it normalizes the behavior. There's other reasons to distrust big corporations, I don't want to live in a world where we should have our guards up all the time.
- SamPatt 2y ago>1) O3 cheated by using Google search. This is both against the rules of the game and OP didn't use search either I re-ran it without search, and it made no difference: https://news.ycombinator.com/item?id=43837832 https://news.ycombinator.com/item?id=43837832 >2) OP was much quicker. They didn't record their time but if their final summary is accurate then they were much faster. Correct. This was the second bullet point of my conclusion: >Humans still hold a big edge in decision time—most of my guesses were < 2 min, o3 often took > 4 min.” I genuinely don't believe that I'm exaggerating or this is clickbait. The o3 geolocation capability astounded me, and I wanted to share my awe with others.
- godelski 2y agoI don't think the time claim was exaggeration or clickbait. I do appreciate you re-running the experiments without search. I think it adds far more legitimacy to the claim. Though in that link I only see a single instance. Does O3 still beat you when it can't search? I'm still interested in that question. Or more specifically: After making O3's play constraints as comparable to a human's (in expected play settings), what is its performance? Truthfully, I think this is the underlying issue that people were bringing up when pointing out EXIF data. How it was cheating was less important than the fact that it was cheating. That's why allowing a different means to cheat undermines your claims.