2 ms·

Crushing it on DeepSWE is a very big deal. Excited to give this a try.

• pietz 24d ago
  I know everyone is benchmaxxing but this one feels one step too far. Doesn't DeepSWE have both public and private tasks? I'd love to see the diff here. It looks more like Google execs losing their mind and pressuring researchers to put DeepSWE directly into the training set.
• re-thc 24d ago
  > DeepSWE is a very big deal It's clearly been "dealt with" already. When it launched we had interesting gaps and definitely differences. Now every new release is "crushing it".
  • ttul 24d ago
    Will look forward to the "feel" of the model in real testing. But I agree that these benchmarks do get "dealt with" rapidly. That's a shame, but I guess it's the times we live in.
• rickdg 23d ago
  Check DeepSWE for number of agent steps.