8 ms·

GPT-5 and GPT-5-Codex are already not clever enough for anything interesting.

by lostmsu 1y ago

asadm 1y ago
not correct. are you using high reasoning? i am using codex everyday.
- pogue 1y ago
  What sort of stuff are you using it for?
  - asadm 1y ago
    computer vision things these days.
- wahnfrieden 1y ago
  Medium reportedly uses even higher reasoning than high, when it chooses to. Whereas high is fixed at a high but not as high level.
jeswin 1y ago
What's your definition of interesting? I'm half way through writing a typescript to native code translator (via .Net) compiling a large enough subset of current code with a lot of help from GPT5 and Codex CLI. It has completely blown me away. I'd like to give you a concrete example which stood out (from by now, dozens). I wanted d.ts files from the .Net Standard Libs. One immediately obvious problem is that .Net would allow classes/interfaces to be redefined if the generic type arity is different. For example, there can be SomeClass<int> and SomeClass<int, int> which are completely separate. TypeScript of course, wouldn't allow this - you can have one with all types defined, but it'd obviously be a mess. I was stuck with (quite ugly): const users = new List_1<User>(...); instead of const users = new List<User>(...); So GPT comes up with this: declare const __unspecified: unique symbol; type __ = typeof __unspecified; // Your arity-anchored delegates exist elsewhere: // import("internal/System").Action_0 // import("internal/System").Action_1<T1> // import("internal/System").Action_2<T1, T2> // import("internal/System").Action_3<T1, T2, T3> // ... up to 17 export type Action< T1 = __, T2 = __, T3 = __, // ... continue through T17 = __ > = [T1] extends [__] ? import("internal/System").Action_0 : [T2] extends [__] ? import("internal/System").Action_1<T1> : [T3] extends [__] ? import("internal/System").Action_2<T1, T2> : /* next lines follow the same pattern … */ import("internal/System").Action_3<T1, T2, T3>; This lets me write: const a: Action<number> = (n) => {}; // OK (void) const f: Func<number, string> = (s) => 20; // OK (string -> number) A human could come up with this, of course. But doing this at scale (there are many such problems which crop up), would take a lot of effort. Btw I'm using Claude for the grunt work (because its faster), but GPT5 is doing all the architecture/thinking/planning/review.
- nawgz 1y ago
  Ternary chains are pretty common in TS, since it’s the main control flow. Are you a comfortable TS user normally?
  - jeswin 1y ago
    I'm a somewhat comfortable TS user. Are you saying ternary chains using sentinels for arity inference is pretty common? I would disagree. > since it’s the main control flow Perhaps you're saying ternery chains are common in TS code? That's a very different thing though - the code above is not for runtime behavior.
    - nawgz 1y ago
      For instance if you wrote a type to extend number to something like range<min,max> - obviously a toy - it would look very similar to what you’ve posted. So I’m struggling to see what the insight from the LLM is. Anytime one needs to iterate in TS, arity or otherwise, that is the technique to use…
      - jeswin 1y ago
        > For instance if you wrote a type to extend number to something like range<min,max> - obviously a toy - it would look very similar to what you’ve posted. Why would Range need a sentinel? My point is that using a sentinel to bridge TypeScript's lack of generic arity-based specialization is a non-trivial problem. After you mentioned it, I looked for examples on Google and couldn't find anything that matches precisely. I'm not claiming humans can't solve this, or that gpt5 invented something fundamentally new. My original point was about productivity at scale. Having a model apply the right solution across dozens of similar problems, rather than me manually figuring out each one.
        nawgz 1y ago
        What is a sentinel?
- 1y ago
EnPissant 1y ago
The GPT5 family is the most intelligent model IMO. It's the only model that I have found that can refactor complex code and have the result be better than what came before it.
RestartKernel 1y ago
I'd say this holds true for LLMs in general depending on your standards of interest, but Codex has blown Claude out of the water for me personally. It seems to have much better code "taste" than any other model.

GPT-5 and GPT-5-Codex are already not clever enough for anything interesting.

asadm 1y ago

not correct. are you using high reasoning? i am using codex everyday.

pogue 1y ago
What sort of stuff are you using it for?
- asadm 1y ago
  computer vision things these days.
wahnfrieden 1y ago
Medium reportedly uses even higher reasoning than high, when it chooses to. Whereas high is fixed at a high but not as high level.

jeswin 1y ago

What's your definition of interesting? I'm half way through writing a typescript to native code translator (via .Net) compiling a large enough subset of current code with a lot of help from GPT5 and Codex CLI. It has completely blown me away. I'd like to give you a concrete example which stood out (from by now, dozens). I wanted d.ts files from the .Net Standard Libs. One immediately obvious problem is that .Net would allow classes/interfaces to be redefined if the generic type arity is different. For example, there can be SomeClass<int> and SomeClass<int, int> which are completely separate. TypeScript of course, wouldn't allow this - you can have one with all types defined, but it'd obviously be a mess. I was stuck with (quite ugly): const users = new List_1<User>(...); instead of const users = new List<User>(...); So GPT comes up with this: declare const __unspecified: unique symbol; type __ = typeof __unspecified; // Your arity-anchored delegates exist elsewhere: // import("internal/System").Action_0 // import("internal/System").Action_1<T1> // import("internal/System").Action_2<T1, T2> // import("internal/System").Action_3<T1, T2, T3> // ... up to 17 export type Action< T1 = __, T2 = __, T3 = __, // ... continue through T17 = __ > = [T1] extends [__] ? import("internal/System").Action_0 : [T2] extends [__] ? import("internal/System").Action_1<T1> : [T3] extends [__] ? import("internal/System").Action_2<T1, T2> : /* next lines follow the same pattern … */ import("internal/System").Action_3<T1, T2, T3>; This lets me write: const a: Action<number> = (n) => {}; // OK (void) const f: Func<number, string> = (s) => 20; // OK (string -> number) A human could come up with this, of course. But doing this at scale (there are many such problems which crop up), would take a lot of effort. Btw I'm using Claude for the grunt work (because its faster), but GPT5 is doing all the architecture/thinking/planning/review.

nawgz 1y ago
Ternary chains are pretty common in TS, since it’s the main control flow. Are you a comfortable TS user normally?
- jeswin 1y ago
  I'm a somewhat comfortable TS user. Are you saying ternary chains using sentinels for arity inference is pretty common? I would disagree. > since it’s the main control flow Perhaps you're saying ternery chains are common in TS code? That's a very different thing though - the code above is not for runtime behavior.
  - nawgz 1y ago
    For instance if you wrote a type to extend number to something like range<min,max> - obviously a toy - it would look very similar to what you’ve posted. So I’m struggling to see what the insight from the LLM is. Anytime one needs to iterate in TS, arity or otherwise, that is the technique to use…
    - jeswin 1y ago
      > For instance if you wrote a type to extend number to something like range<min,max> - obviously a toy - it would look very similar to what you’ve posted. Why would Range need a sentinel? My point is that using a sentinel to bridge TypeScript's lack of generic arity-based specialization is a non-trivial problem. After you mentioned it, I looked for examples on Google and couldn't find anything that matches precisely. I'm not claiming humans can't solve this, or that gpt5 invented something fundamentally new. My original point was about productivity at scale. Having a model apply the right solution across dozens of similar problems, rather than me manually figuring out each one.
      - nawgz 1y ago
        What is a sentinel?
1y ago

EnPissant 1y ago

The GPT5 family is the most intelligent model IMO. It's the only model that I have found that can refactor complex code and have the result be better than what came before it.

RestartKernel 1y ago

I'd say this holds true for LLMs in general depending on your standards of interest, but Codex has blown Claude out of the water for me personally. It seems to have much better code "taste" than any other model.