4 ms·

This entire thing has been pretty disingenuous on both sides of the fence. All the anti-AI (or anti OpenAI) people are doing victory laps, but what GPT-5 Pro di

This entire thing has been pretty disingenuous on both sides of the fence. All the anti-AI (or anti OpenAI) people are doing victory laps, but what GPT-5 Pro did is still very valuable. 1) What good is your open problem set if really its a trivial "google search" away from being solved. Why are they not catching any blame here? 2) These answers still weren't perfectly laid out for the most part. GPT-5 was still doing some cognitive lifting to piece it together. If a human would have done this by hand it would have made news and instead the narrative would have been inverted to ask serious questions about the validity of some these style problem sets and/or ask the question how many other solutions are out there that just need pieced together from pre-existing research. But, you know, AI Bad.

nurettin 1y ago
AI great, but AI not creative, yet.
puttycat 1y ago
This is a strawman argument. No anti-AI sentiment was involved here. Simply the fact that finding and matching text on the Internet is several orders of magnitude easier than finding novel solutions to hard math problems.
- strangescript 1y ago
  You didn't read the X replies if you believe that
matsemann 1y ago
You're moving the goal post.
Topfi 1y ago
> What good is your open problem set if really its a trivial "google search" away from being solved. Why are they not catching any blame here? They are a community run database, not the sole arbiter and source of this information. We learned the most basic research back in highschool, I'd hope researchers from top institutions now working for one of the biggest frontier labs can do the same prior to making a claim, but microblogging has and continues to be a blight on any accurate information so nothing new there. > GPT-5 was still doing some cognitive lifting to piece it together. Cognitive lifting? It's a model, not a person, but besides that fact, this was already published literature. Handy that a LLM can be a slightly better search, but calling claims of "solving maths problems" out as irresponsible and inaccurate is the only right choice in this case. > If a human would have done this by hand it would have made news [...] "Researcher does basic literature review" isn't news in this or any other scenario. If we did a press release every journal club, there wouldn't be enough time to print a single page advert. > [...] how many other solutions are out there that just need pieced together from pre-existing research [...] I am not certain you actually looked into the model output or why this was such an embarrassment. > But, you know, AI Bad. AI hype very bad. AI anthropomorphism even worse.
- throwawayerdos 1y ago
  [dead]
andrepd 1y ago
> 1) What good is your open problem set if really its a trivial "google search" away from being solved. Why are they not catching any blame here? Please explain how this is in any way related to the matter at hand. What is the relation between the incompleteness of an math problem database, and AI hypesters lying about the capabilities of GPT5? I fail to see the relevance. > If a human would have done this by hand it would have made news If someone updated information on an obscure math problem aggregator database this would be news?? Again, I fail to see your point here.
lukev 1y ago
Framing this question as "AI good" OR "AI bad" is culture-war thinking. The real problem here is that there's clearly a strong incentive for the big labs to deceive the public (and/or themselves) about the actual scientific and technical capabilities of LLMs. As Karpathy pointed out on the recent Dwarkesh podcast, LLMs are quite terrible at novel problems, but this has become sort of an "Emperor's new clothes" situation where nobody with a financial stake will actually admit that, even though it's common knowledge if you actually work with these things. And this directly leads to the misallocation of billions of dollars and potentially trillions in economic damage as companies align their 5-year strategies towards capabilities that are (right now) still science fiction. The truth is at stake.
- strangescript 1y ago
  Except they weren't intentionally trying to deceive anyone. They made the faulty assumption that these problems were non-trivial to solve and didn't think it was simply GPT-5 aggregating solutions in the wild.
  - lukev 1y ago
    Knowing what I know about LLMs, from their internal architecture and from extensive experience working with them daily, I would find this kind of result highly surprising and in a clear violation of my mental model of how these things work. And I'm very far from an expert. If a purported expert in the field can is willing to credulously publish this kind of result, it's not unreasonable to assume that either they're acting in bad faith, or (at best) are high on their own supply regarding what these things can actually do.

nurettin 1y ago

AI great, but AI not creative, yet.

puttycat 1y ago

This is a strawman argument. No anti-AI sentiment was involved here. Simply the fact that finding and matching text on the Internet is several orders of magnitude easier than finding novel solutions to hard math problems.

strangescript 1y ago
You didn't read the X replies if you believe that

matsemann 1y ago

You're moving the goal post.

Topfi 1y ago

> What good is your open problem set if really its a trivial "google search" away from being solved. Why are they not catching any blame here? They are a community run database, not the sole arbiter and source of this information. We learned the most basic research back in highschool, I'd hope researchers from top institutions now working for one of the biggest frontier labs can do the same prior to making a claim, but microblogging has and continues to be a blight on any accurate information so nothing new there. > GPT-5 was still doing some cognitive lifting to piece it together. Cognitive lifting? It's a model, not a person, but besides that fact, this was already published literature. Handy that a LLM can be a slightly better search, but calling claims of "solving maths problems" out as irresponsible and inaccurate is the only right choice in this case. > If a human would have done this by hand it would have made news [...] "Researcher does basic literature review" isn't news in this or any other scenario. If we did a press release every journal club, there wouldn't be enough time to print a single page advert. > [...] how many other solutions are out there that just need pieced together from pre-existing research [...] I am not certain you actually looked into the model output or why this was such an embarrassment. > But, you know, AI Bad. AI hype very bad. AI anthropomorphism even worse.

throwawayerdos 1y ago
[dead]

andrepd 1y ago

> 1) What good is your open problem set if really its a trivial "google search" away from being solved. Why are they not catching any blame here? Please explain how this is in any way related to the matter at hand. What is the relation between the incompleteness of an math problem database, and AI hypesters lying about the capabilities of GPT5? I fail to see the relevance. > If a human would have done this by hand it would have made news If someone updated information on an obscure math problem aggregator database this would be news?? Again, I fail to see your point here.

lukev 1y ago

Framing this question as "AI good" OR "AI bad" is culture-war thinking. The real problem here is that there's clearly a strong incentive for the big labs to deceive the public (and/or themselves) about the actual scientific and technical capabilities of LLMs. As Karpathy pointed out on the recent Dwarkesh podcast, LLMs are quite terrible at novel problems, but this has become sort of an "Emperor's new clothes" situation where nobody with a financial stake will actually admit that, even though it's common knowledge if you actually work with these things. And this directly leads to the misallocation of billions of dollars and potentially trillions in economic damage as companies align their 5-year strategies towards capabilities that are (right now) still science fiction. The truth is at stake.

strangescript 1y ago
Except they weren't intentionally trying to deceive anyone. They made the faulty assumption that these problems were non-trivial to solve and didn't think it was simply GPT-5 aggregating solutions in the wild.
- lukev 1y ago
  Knowing what I know about LLMs, from their internal architecture and from extensive experience working with them daily, I would find this kind of result highly surprising and in a clear violation of my mental model of how these things work. And I'm very far from an expert. If a purported expert in the field can is willing to credulously publish this kind of result, it's not unreasonable to assume that either they're acting in bad faith, or (at best) are high on their own supply regarding what these things can actually do.