5 ms·

For coding assistance, I have tried OpenCode with several large open models through OpenRouter. All were fairly bad compared to Claude Opus. Could you provide

by smokel 5mo ago

For coding assistance, I have tried OpenCode with several large open models through OpenRouter. All were fairly bad compared to Claude Opus. Could you provide some hints on how I should be holding these open models so that I might get more value out of them? I agree with the common trope that open models lag behind by about a year, but something magical happened just around a year ago when the state of the art models became extremely useful. By this reasoning we're about to see open models perform well, but I'm afraid there is more to it than just waiting for another revolution around the sun. Note, my application is coding assistance. Open models can be great for other purposes.

tariky 5mo ago
I tried almost all OS models on opencode, none of them is on levels as opus 4.7. In latest experiment I used opus for implementation plan then used cursor composer 2.5 for execution. I must say that combo is really good. Main drawback of claude code is that is super slow. So when paired with composer that is super fast it flies.
- cainxinth 5mo ago
  No one is claiming that OS is as good. They are saying it isn't that far behind SOTA commercial products. So why pay exorbitantly just to get something only a few percent better than the free option? But there have been very good open source office apps for decades and few enterprises use them, so perhaps this is just the nature of B2B purchasing committees and 'nobody getting fired for buying IBM.'
  - Alex-Programs 5mo ago
    Because failures compound. My productivity has substantially improved since I switched from open models to a Codex subscription, because it doesn't need hand holding, and it doesn't pull stupid tricks occasionally.
slopinthebag 5mo ago
Do more planning yourself, be smart about the context, break down tasks into smaller components, give it more guidance. You can't just lazily prompt it to complete large features autonomously and expect good results.
- amilios 5mo ago
  But if the closed-source models can do this without the additional effort, that's a significant gap, no?
  - flexagoon 5mo ago
    Is it really when they are hundreds of times more expensive?
  - 10000truths 5mo ago
    The point is that the price gap is so much larger than the capability gap, that even with the extra compute needed to make up for the lack of capability, you can still come out ahead in terms of amortized $/work done.
  - bigfishrunning 5mo ago
    See that's the thing, they can't. Every model needs hand holding and guidance.
    - amilios 5mo ago
      some require less hand-holding than others though
      - myaccountonhn 5mo ago
        No one is trying to argue that OS models are better than Opus 4.7. It's simply that they're good enough and cheaper.
  - eikenberry 5mo ago

by smokel 5mo ago

tariky 5mo ago

I tried almost all OS models on opencode, none of them is on levels as opus 4.7. In latest experiment I used opus for implementation plan then used cursor composer 2.5 for execution. I must say that combo is really good. Main drawback of claude code is that is super slow. So when paired with composer that is super fast it flies.

cainxinth 5mo ago
No one is claiming that OS is as good. They are saying it isn't that far behind SOTA commercial products. So why pay exorbitantly just to get something only a few percent better than the free option? But there have been very good open source office apps for decades and few enterprises use them, so perhaps this is just the nature of B2B purchasing committees and 'nobody getting fired for buying IBM.'
- Alex-Programs 5mo ago
  Because failures compound. My productivity has substantially improved since I switched from open models to a Codex subscription, because it doesn't need hand holding, and it doesn't pull stupid tricks occasionally.

slopinthebag 5mo ago

Do more planning yourself, be smart about the context, break down tasks into smaller components, give it more guidance. You can't just lazily prompt it to complete large features autonomously and expect good results.

amilios 5mo ago
But if the closed-source models can do this without the additional effort, that's a significant gap, no?
- flexagoon 5mo ago
  Is it really when they are hundreds of times more expensive?
- 10000truths 5mo ago
  The point is that the price gap is so much larger than the capability gap, that even with the extra compute needed to make up for the lack of capability, you can still come out ahead in terms of amortized $/work done.
- bigfishrunning 5mo ago
  See that's the thing, they can't. Every model needs hand holding and guidance.
  - amilios 5mo ago
    some require less hand-holding than others though
    - myaccountonhn 5mo ago
      No one is trying to argue that OS models are better than Opus 4.7. It's simply that they're good enough and cheaper.
- eikenberry 5mo ago