4 ms·

While Claude Sonnet is superior than 4o for most my use cases, there are still occasionally some specific tasks where it performs slightly better.

antirez 2y ago
Probably. But statistically to work with 4o is a lose of time for me. LLMs is like an investment: you write the prompts, you "work" with them. If the LLM is too weak, this is a lose of time. You need to have a return on the investment that is positive. With ChatGPT 4o / o1 most of the times for me the investment of time has almost zero return. Before Claude Sonnet 3.5 I already had a ChatGPT PRO account but never used it for coding since it was most of the times useless if not for throw away scripts that I didn't want to do myself or as a stack overflow replacement for trivial stuff. Now it's different.
- airstrike 2y ago
  This mirrors my experience 100%. I'm not even sure why I still pay for OpenAI at this point. Claude 3.5 is just incredibly superior. And I totally agree on the point about dropping in context and asking very specific questions. I've had Claude pinpoint a bug in a 2k LOC module that I was struggling to find the cause for. After wasting a lot of time on it on my own, I thought "what the heck, maybe Claude can figure it out" and it did. It's objectively useful, even if flawed sometimes.
  - hanikesn 2y ago
    I'm curious. Can you go into more detail what kind of bug it found?
    - airstrike 2y ago
      I was writing a custom widget for iced (the Rust GUI library) and I was getting a panic due to some fancy logic I was trying to do. I guess the shortest description I can say is that it was a combination of what appeared to be a caching issue at first, but the real cause turned out to be some method shadowing where I was using a struct's method where I meant to use the trait's method. I had made the specific operation generic (moving it out of the struct and into a trait) but forgot to delete it from the struct, so I was calling the incorrect function. Claude pinpointed the cache issue immediately when I just dumped two files into the context and asked it: somewhere in my codebase I'm triggering a perform() on the editor but the next call on highlight() panics because `Line layout should be cached` what am I missing? do I need to do something after perform() to re-cache the layout? at first that seemed to fix the issue, but other errors persisted. so we kept debugging together until we found the root cause. either way I knew where to look thanks to its assistance
- d0mine 2y ago
  why "lose of time" instead of "loss of time" Is it a typo or fingerprinting?
  - fragmede 2y ago
    it's "proof" that it wasn't written by an LLM (but let me delve into this issue).
  - squirrel 2y ago
    Typo
tootie 2y ago
Like what? Claude has become my go-to, but I find that it's wrong enough often enough that I really can't trust it for anything. If it says something I have to go dig through it's citations very carefully.

antirez 2y ago

Probably. But statistically to work with 4o is a lose of time for me. LLMs is like an investment: you write the prompts, you "work" with them. If the LLM is too weak, this is a lose of time. You need to have a return on the investment that is positive. With ChatGPT 4o / o1 most of the times for me the investment of time has almost zero return. Before Claude Sonnet 3.5 I already had a ChatGPT PRO account but never used it for coding since it was most of the times useless if not for throw away scripts that I didn't want to do myself or as a stack overflow replacement for trivial stuff. Now it's different.

airstrike 2y ago
This mirrors my experience 100%. I'm not even sure why I still pay for OpenAI at this point. Claude 3.5 is just incredibly superior. And I totally agree on the point about dropping in context and asking very specific questions. I've had Claude pinpoint a bug in a 2k LOC module that I was struggling to find the cause for. After wasting a lot of time on it on my own, I thought "what the heck, maybe Claude can figure it out" and it did. It's objectively useful, even if flawed sometimes.
- hanikesn 2y ago
  I'm curious. Can you go into more detail what kind of bug it found?
  - airstrike 2y ago
    I was writing a custom widget for iced (the Rust GUI library) and I was getting a panic due to some fancy logic I was trying to do. I guess the shortest description I can say is that it was a combination of what appeared to be a caching issue at first, but the real cause turned out to be some method shadowing where I was using a struct's method where I meant to use the trait's method. I had made the specific operation generic (moving it out of the struct and into a trait) but forgot to delete it from the struct, so I was calling the incorrect function. Claude pinpointed the cache issue immediately when I just dumped two files into the context and asked it: somewhere in my codebase I'm triggering a perform() on the editor but the next call on highlight() panics because `Line layout should be cached` what am I missing? do I need to do something after perform() to re-cache the layout? at first that seemed to fix the issue, but other errors persisted. so we kept debugging together until we found the root cause. either way I knew where to look thanks to its assistance
d0mine 2y ago
why "lose of time" instead of "loss of time" Is it a typo or fingerprinting?
- fragmede 2y ago
  it's "proof" that it wasn't written by an LLM (but let me delve into this issue).
- squirrel 2y ago
  Typo

tootie 2y ago

Like what? Claude has become my go-to, but I find that it's wrong enough often enough that I really can't trust it for anything. If it says something I have to go dig through it's citations very carefully.