8 ms·

“GitHub says Claude Sonnet 4 soars in agentic scenarios and will introduce it as the base model for the new coding agent in GitHub Copilot.” Maybe this model w

“GitHub says Claude Sonnet 4 soars in agentic scenarios and will introduce it as the base model for the new coding agent in GitHub Copilot.” Maybe this model will push the “Assign to CoPilot” closer to the dream of having package upgrades and other mostly-mechanical stuff handled automatically. This tech could lead to a huge revival of older projects as the maintenance burden falls.

ed_elliott_asc 1y ago
Until it pushes a severe vulnerability which takes a big service doen
BaculumMeumEst 1y ago
Anyone see news of when it’s planned to go live in copilot?
- vinhphm 1y ago
  The option just shown up in Copilot settings page for me
  - BaculumMeumEst 1y ago
    Same! Rock and roll!
    - denysvitali 1y ago
      I got rate-limited in like 5 seconds. Wow
  - bbor 1y ago
    Turns out Opus 4 starts at their $40/mo ("Pro+") plan which is sad, and they serve o4-mini and Gemini as well so it's a bit less exclusive than this announcement implies. That said, I have a random question for any Anthropic-heads out there: GitHub says "Claude Opus 4 is hosted by Anthropic PBC. Claude Sonnet 4 is hosted by Anthropic 1P."[1]. What's Anthropic 1P? Based on the only Kagi result being a deployment tutorial[2] and the fact that GitHub negotiated a "zero retention agreement" with the PBC but not whatever "1P" is, I'm assuming it's a spinoff cloud company that only serves Claude...? No mention on the Wikipedia or any business docs I could find, either. Anyway, off to see if I can access it from inside SublimeText via LSP! [1] https://docs.github.com/en/copilot/using-github-copilot/ai-models/using-claude-in-github-copilot https://docs.github.com/en/copilot/using-github-copilot/ai-m... [2] https://github.com/anthropics/prompt-eng-interactive-tutorial https://github.com/anthropics/prompt-eng-interactive-tutoria...
    - l1n 1y ago
      1P = Anthropic's first party API, e.g. not through Bedrock or Vertex
    - Workaccount2 1y ago
      Google launched Jules two days ago, which is the gemini coding agent[1]. I was pretty quickly accepted into the beta and you get 5 free tasks a day. So far I have found it pretty powerful, its also the first time an LLM has ever stopped while working to ask me a question or for clarification. [1]https://jules.google/ https://jules.google/
    - Barbing 1y ago
      Interesting, first link changed now: "Claude Opus 4 and Claude Sonnet 4 are hosted by Anthropic PBC and Google Cloud Platform." They also mention: "GitHub has provider agreements in place to ensure data is not used for training." They go on to elaborate. Perhaps this kind of offering instills confidence in some who might not trust model providers 1:1, but believe they will respect their contract with a large customer like Microsoft (GitHub).
- minimaxir 1y ago
  The keynote confirms it is available now.
  - jasonthorsness 1y ago
    Gotta love keynotes with concurrent immediate availability
    - brookst 1y ago
      Not if you work there
      - echelon 1y ago
        That's just a few weeks of DR + prep, a feature freeze, and oncall with bated breath. Nothing any rank and file hasn't been through before with a company that relies on keynotes and flashy releases for growth. Stressful, but part and parcel. And well-compensated.
        brookst 1y ago
        Sometimes. When things work great. Sometimes you just hear “BTW your previously-soft-released feature will be on stage day after tomorrow, probably don’t make any changes until after the event, and expect 10x traffic”
ModernMech 1y ago
That's kind of my benchmark for whether or not these models are useful. I've got a project that needs some extensive refactoring to get working again. Mostly upgrading packages, but also it will require updating the code to some new language semantics that didn't exist when it was written. So far, current AI models can make essentially zero progress on this task. I'll keep trying until they can!
- tmpz22 1y ago
  And IMO it has a long way to go. There is a lot of nuance when orchestrating dependencies that can cause subtle errors in an application that are not easily remedied. For example a lot of llms (I've seen it in Gemini 2.5, and Claude 3.7) will code non-existent methods in dynamic languages. While these runtime errors are often auto-fixable, sometimes they aren't, and breaking out of an agentic workflow to deep dive the problem is quite frustrating - if mostly because agentic coding entices us into being so lazy.
  - jasonthorsness 1y ago
    The agents will definitely need a way to evaluate their work just as well as a human would - whether that's a full test suite, tests + directions on some manual verification as well, or whatever. If they can't use the same tools as a human would they'll never be able to improve things safely.
  - mikepurvis 1y ago
    "... and breaking out of an agentic workflow to deep dive the problem is quite frustrating" Maybe that's the problem that needs solving then? The threshold doesn't have to be "bot capable of doing entire task end to end", like it could also be "bot does 90% of task, the worst and most boring part, human steps in at the end to help with the one bit that is more tricky". Or better yet, the bot is able to recognize its own limitations and proactively surface these instances, be like hey human I'm not sure what to do in this case; based on the docs I think it should be A or B, but I also feel like C should be possible yet I can't get any of them to work, what do you think? As humans, it's perfectly normal to put up a WIP PR and then solicit this type of feedback from our colleagues; why would a bot be any different?
rco8786 1y ago
It could be! But that's also what people said about all the models before it!
- kmacdough 1y ago
  And they might all be right! > This tech could lead to... I don't think he's saying this is the version that will suddenly trigger a Renaissance. Rather, it's one solid step that makes the path ever more promising. Sure, everyone gets a bit overexcited each release until they find the bounds. But the bounds are expanding, and the need for careful prompt engineering is diminishing. Ever since 3.7, Claude has been a regular part of my process for the mundane. And so far 4.0 seems to take less fighting for me. A good question would be when can AI take a basic prompt, gather its own requirements and build meaningful PRs off basic prompt. I suspect it's still at least a couple of paradigm shifts away. But those seem to be coming every year or faster.
  - sagarpatil 1y ago
    Did you not see the live stream? They took a feature request for excalidraw (table support) and Claude 4 worked on it for 90 minutes and the PR was working as expected. I’m not sure if they were using sonnet or opus.
    - andrepd 1y ago
      Pre-prepared demos don't impress me.
      - spiderfarmer 1y ago
        By that logic sport athletes don’t impress you. Movies don’t impress you. Theater doesn’t impress you. Your date won’t impress you. Becoming a parent won’t impress you. Most things in life take years of preparation.
        xaoz 1y ago
max_on_hn 1y ago
I am incredibly eager to see what affordable coding agents can do for open source :) in fact, I should really be giving away CheepCode[0] credits to open source projects. Pending any sort of formal structure, if you see this comment and want free coding agent runs, email me and I’ll set you up! [0] My headless coding agents product, similar to “assign to copilot” but works from your task board (Linear, Jira, etc) on multiple tasks in parallel. So far simple/routine features are already quite successful. In general the better the tests, the better the resulting code (and yes, it can and does write its own tests).
- dr_dshiv 1y ago
  Especially since the EU just made open source contributors liable for cybersecurity (Cyber Resilience Act). Just let AI contribute and ur good
  - hn111 1y ago
    Didn’t they make an exception for open-source projects? https://opensource.org/blog/the-european-regulators-listened-to-the-open-source-communities https://opensource.org/blog/the-european-regulators-listened...
    - andrepd 1y ago
      Yeah, just the usual hn FUD about the EU.
      - throwaway894345 1y ago
        Can you reconcile that with this sibling comment? https://news.ycombinator.com/item?id=44074070 https://news.ycombinator.com/item?id=44074070 I don’t have an opinion, just trying to make sense of contradictory claims.
    - dr_dshiv 1y ago
      “Anyone opensourcing anything while in the course of ‘commercial activity’ will be fully liable. Effectively they rugpulled the Apache2 / MIT licenses... all opensource released by small businesses is fucked. where the was no red tape now there is infinite liability” This is my current understanding, from a friend not a lawyer. Would appreciate any insight from folks here.
epolanski 1y ago
> having package upgrades and other mostly-mechanical stuff handled automatically Those are already non-issues mostly solved by bots. In any case, where I think AI could help here would be by summarizing changes, conflicts, impact on codebase and possibly also conduct security scans.
phito 1y ago
I don't see how a LLM could do better than a bot, eg renovate

ed_elliott_asc 1y ago

Until it pushes a severe vulnerability which takes a big service doen

BaculumMeumEst 1y ago

Anyone see news of when it’s planned to go live in copilot?

vinhphm 1y ago
The option just shown up in Copilot settings page for me
- BaculumMeumEst 1y ago
  Same! Rock and roll!
  - denysvitali 1y ago
    I got rate-limited in like 5 seconds. Wow
- bbor 1y ago
  Turns out Opus 4 starts at their $40/mo ("Pro+") plan which is sad, and they serve o4-mini and Gemini as well so it's a bit less exclusive than this announcement implies. That said, I have a random question for any Anthropic-heads out there: GitHub says "Claude Opus 4 is hosted by Anthropic PBC. Claude Sonnet 4 is hosted by Anthropic 1P."[1]. What's Anthropic 1P? Based on the only Kagi result being a deployment tutorial[2] and the fact that GitHub negotiated a "zero retention agreement" with the PBC but not whatever "1P" is, I'm assuming it's a spinoff cloud company that only serves Claude...? No mention on the Wikipedia or any business docs I could find, either. Anyway, off to see if I can access it from inside SublimeText via LSP! [1] https://docs.github.com/en/copilot/using-github-copilot/ai-models/using-claude-in-github-copilot https://docs.github.com/en/copilot/using-github-copilot/ai-m... [2] https://github.com/anthropics/prompt-eng-interactive-tutorial https://github.com/anthropics/prompt-eng-interactive-tutoria...
  - l1n 1y ago
    1P = Anthropic's first party API, e.g. not through Bedrock or Vertex
  - Workaccount2 1y ago
    Google launched Jules two days ago, which is the gemini coding agent[1]. I was pretty quickly accepted into the beta and you get 5 free tasks a day. So far I have found it pretty powerful, its also the first time an LLM has ever stopped while working to ask me a question or for clarification. [1]https://jules.google/ https://jules.google/
  - Barbing 1y ago
    Interesting, first link changed now: "Claude Opus 4 and Claude Sonnet 4 are hosted by Anthropic PBC and Google Cloud Platform." They also mention: "GitHub has provider agreements in place to ensure data is not used for training." They go on to elaborate. Perhaps this kind of offering instills confidence in some who might not trust model providers 1:1, but believe they will respect their contract with a large customer like Microsoft (GitHub).
minimaxir 1y ago
The keynote confirms it is available now.
- jasonthorsness 1y ago
  Gotta love keynotes with concurrent immediate availability
  - brookst 1y ago
    Not if you work there
    - echelon 1y ago
      That's just a few weeks of DR + prep, a feature freeze, and oncall with bated breath. Nothing any rank and file hasn't been through before with a company that relies on keynotes and flashy releases for growth. Stressful, but part and parcel. And well-compensated.
      - brookst 1y ago
        Sometimes. When things work great. Sometimes you just hear “BTW your previously-soft-released feature will be on stage day after tomorrow, probably don’t make any changes until after the event, and expect 10x traffic”

ModernMech 1y ago

That's kind of my benchmark for whether or not these models are useful. I've got a project that needs some extensive refactoring to get working again. Mostly upgrading packages, but also it will require updating the code to some new language semantics that didn't exist when it was written. So far, current AI models can make essentially zero progress on this task. I'll keep trying until they can!

tmpz22 1y ago
And IMO it has a long way to go. There is a lot of nuance when orchestrating dependencies that can cause subtle errors in an application that are not easily remedied. For example a lot of llms (I've seen it in Gemini 2.5, and Claude 3.7) will code non-existent methods in dynamic languages. While these runtime errors are often auto-fixable, sometimes they aren't, and breaking out of an agentic workflow to deep dive the problem is quite frustrating - if mostly because agentic coding entices us into being so lazy.
- jasonthorsness 1y ago
  The agents will definitely need a way to evaluate their work just as well as a human would - whether that's a full test suite, tests + directions on some manual verification as well, or whatever. If they can't use the same tools as a human would they'll never be able to improve things safely.
- mikepurvis 1y ago
  "... and breaking out of an agentic workflow to deep dive the problem is quite frustrating" Maybe that's the problem that needs solving then? The threshold doesn't have to be "bot capable of doing entire task end to end", like it could also be "bot does 90% of task, the worst and most boring part, human steps in at the end to help with the one bit that is more tricky". Or better yet, the bot is able to recognize its own limitations and proactively surface these instances, be like hey human I'm not sure what to do in this case; based on the docs I think it should be A or B, but I also feel like C should be possible yet I can't get any of them to work, what do you think? As humans, it's perfectly normal to put up a WIP PR and then solicit this type of feedback from our colleagues; why would a bot be any different?

rco8786 1y ago

It could be! But that's also what people said about all the models before it!

kmacdough 1y ago
And they might all be right! > This tech could lead to... I don't think he's saying this is the version that will suddenly trigger a Renaissance. Rather, it's one solid step that makes the path ever more promising. Sure, everyone gets a bit overexcited each release until they find the bounds. But the bounds are expanding, and the need for careful prompt engineering is diminishing. Ever since 3.7, Claude has been a regular part of my process for the mundane. And so far 4.0 seems to take less fighting for me. A good question would be when can AI take a basic prompt, gather its own requirements and build meaningful PRs off basic prompt. I suspect it's still at least a couple of paradigm shifts away. But those seem to be coming every year or faster.
- sagarpatil 1y ago
  Did you not see the live stream? They took a feature request for excalidraw (table support) and Claude 4 worked on it for 90 minutes and the PR was working as expected. I’m not sure if they were using sonnet or opus.
  - andrepd 1y ago
    Pre-prepared demos don't impress me.
    - spiderfarmer 1y ago
      By that logic sport athletes don’t impress you. Movies don’t impress you. Theater doesn’t impress you. Your date won’t impress you. Becoming a parent won’t impress you. Most things in life take years of preparation.
      - xaoz 1y ago

max_on_hn 1y ago

I am incredibly eager to see what affordable coding agents can do for open source :) in fact, I should really be giving away CheepCode[0] credits to open source projects. Pending any sort of formal structure, if you see this comment and want free coding agent runs, email me and I’ll set you up! [0] My headless coding agents product, similar to “assign to copilot” but works from your task board (Linear, Jira, etc) on multiple tasks in parallel. So far simple/routine features are already quite successful. In general the better the tests, the better the resulting code (and yes, it can and does write its own tests).

dr_dshiv 1y ago
Especially since the EU just made open source contributors liable for cybersecurity (Cyber Resilience Act). Just let AI contribute and ur good
- hn111 1y ago
  Didn’t they make an exception for open-source projects? https://opensource.org/blog/the-european-regulators-listened-to-the-open-source-communities https://opensource.org/blog/the-european-regulators-listened...
  - andrepd 1y ago
    Yeah, just the usual hn FUD about the EU.
    - throwaway894345 1y ago
      Can you reconcile that with this sibling comment? https://news.ycombinator.com/item?id=44074070 https://news.ycombinator.com/item?id=44074070 I don’t have an opinion, just trying to make sense of contradictory claims.
  - dr_dshiv 1y ago
    “Anyone opensourcing anything while in the course of ‘commercial activity’ will be fully liable. Effectively they rugpulled the Apache2 / MIT licenses... all opensource released by small businesses is fucked. where the was no red tape now there is infinite liability” This is my current understanding, from a friend not a lawyer. Would appreciate any insight from folks here.

epolanski 1y ago

> having package upgrades and other mostly-mechanical stuff handled automatically Those are already non-issues mostly solved by bots. In any case, where I think AI could help here would be by summarizing changes, conflicts, impact on codebase and possibly also conduct security scans.

phito 1y ago

I don't see how a LLM could do better than a bot, eg renovate