5 ms·

Opus 4.5 is writing code that Opus 5.0 will refactor and extend. And Opus 5.5 will take that code and rewrite it in C from the ground up. And Opus 6.0 will take

Opus 4.5 is writing code that Opus 5.0 will refactor and extend. And Opus 5.5 will take that code and rewrite it in C from the ground up. And Opus 6.0 will take that code and make it assembly. And Opus 7.0 will design its own CPU. And Opus 8.0 will make a factory for its own CPUs. And Opus 9.0 will populate mars. And Opus 10.0 will be able to achieve AGI. And Opus 11.0 will find God. And Opus 12.0 will make us a time machine. And so on.

latentsea 10mo ago
Can't wait to see what Opus 13.0 does with the multiverse.
- coldtea 9mo ago
  https://users.ece.cmu.edu/~gamvrosi/thelastq.html https://users.ece.cmu.edu/~gamvrosi/thelastq.html
zwnow 9mo ago
I also love how AI enthusiasts just ignore the issue of exhausted training data... You cant just magically create more training data. Also synthetic training data reduces the quality of models.
- puchatek 9mo ago
  That's been my main argument for why LLMs might be at their zenith. But I recently started wondering whether all those codebases we expose to them are maybe good enough training data for the next generation. It's not high quality like accepted stackoverflow answers but it's working software for the most part.
  - jacquesm 9mo ago
    If they'd be good enough you could rent them to put together closed source stuff you can hide behind a paywall, or maybe the AI owners would also own the paywall and rent you the software instead. The second that that is possible it will happen.
- TeMPOraL 9mo ago
  They don't ignore it, they just know it's not an actual problem. It saddens me to see AI detractors being stuck in 2022 and still thinking language models are just regurgitating bits of training data.
  - zwnow 9mo ago
    You are thankfully wrong. I watch lots of talks on the topic from actual experts. New models are just old models with more tooling. Training data is exhausted and its a real issue.
    - GrumpyGoblin 9mo ago
      Not saying it's not a problem, I actually don't know, but new CPU's are just old models with more improvements/tooling. Same with TV's. And cars. And clothes. Everything is. That's how improving things works. Running out of raw data doesn't mean running out of room for improvement. The data has been the same for the last 20 years, AI isn't new, things keep improving anyways.
      - zwnow 9mo ago
        Well from cars or CPUs its not expected for them to eventually reach AGI, they also don't eat a trillion dollar hole into us peasants pockets. Sure, improvements can be made. But on a fundamental level, agents/LLMs can not reason (even though they love to act like they can). They are parrots learning words, these parrots wont ever invent new words once the list of words is exhausted though.
    - TeMPOraL 9mo ago
      Well, my experts disagree with your experts :). Sure, the supply of available fresh data is running out, but at the same time, there's way more data than needed. Most of it is low-quality noise anyway. New models aren't just old models with more tooling - the entire training pipeline has been evolving, as researchers and model vendors focus on making better use of data they have, and refining training datasets themselves. There are more stages to LLM training than just the pre-training stage :).
- aspenmartin 9mo ago
  Youre mixing up several concepts. Synthetic data works for coding because coding is a verifiable domain. You train via reinforcement learning to reward code generation behavior that passes detailed specs and meets other deseridata. It’s literally how things are done today and how progress gets made.
  - zwnow 9mo ago
    Most code out there is a legacy security nightmare, surely its good to train on that.
    - dang 9mo ago
      Would you please stop posting cynical, dismissive comments? From a brief scroll through https://news.ycombinator.com/comments?id=zwnow https://news.ycombinator.com/comments?id=zwnow, it seems like your account has been doing nothing else, regardless of the topic that it's commenting on. This is not what HN is for, and destroys what it is for. If you keep this up, we're going to have to ban you, not because of your views on any particular topic but because you're going entirely against the intended spirit of the site by posting this way. There's plenty of room to express your views substantively and thoughtfully, but we don't want cynical flamebait and denunciation. HN needs a good deal less of this. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
      - zwnow 9mo ago
        Then ban me u loser, as I wrote HN is full of pretentious bullshitters. But its good that u wanna ban authentic views. Way to go. If i feel like it I'll just create a new account:-)
        aspenmartin 9mo ago
        dang is a saint, he wants your opinion, not the other toxic stuff.
lomase 9mo ago
Just one more OPUS bro.
- whynotminot 9mo ago
  Honestly the scary part is that we don’t really even need one more Opus. If all we had for the rest of our lives was Opus 4.5, the software engineering world would still radically change. But there’s no sign of them slowing down.
- deleted 9mo ago
  [deleted]
mfalcon 9mo ago
Wake me up at Opus 12
TheOtherHobbes 9mo ago
Objectively, we are talking about systems that have gone from being cute toys to outmatching most juniors using only rigid and slow batch training cycles. As soon as models have persistent memory for their own try/fail/succeed attempts, and can directly modify what's currently called their training data in real time, they're going to develop very, very quickly. We may even be underestimating how quickly this will happen. We're also underestimating how much more powerful they become if you give them analysis and documentation tasks referencing high quality software design principles before giving them code to write. This is very much 1.0 tech. It's already scary smart compared to the median industry skill level. The 2.0 version is going to be something else entirely.

latentsea 10mo ago

Can't wait to see what Opus 13.0 does with the multiverse.

coldtea 9mo ago
https://users.ece.cmu.edu/~gamvrosi/thelastq.html https://users.ece.cmu.edu/~gamvrosi/thelastq.html

zwnow 9mo ago

I also love how AI enthusiasts just ignore the issue of exhausted training data... You cant just magically create more training data. Also synthetic training data reduces the quality of models.

puchatek 9mo ago
That's been my main argument for why LLMs might be at their zenith. But I recently started wondering whether all those codebases we expose to them are maybe good enough training data for the next generation. It's not high quality like accepted stackoverflow answers but it's working software for the most part.
- jacquesm 9mo ago
  If they'd be good enough you could rent them to put together closed source stuff you can hide behind a paywall, or maybe the AI owners would also own the paywall and rent you the software instead. The second that that is possible it will happen.
TeMPOraL 9mo ago
They don't ignore it, they just know it's not an actual problem. It saddens me to see AI detractors being stuck in 2022 and still thinking language models are just regurgitating bits of training data.
- zwnow 9mo ago
  You are thankfully wrong. I watch lots of talks on the topic from actual experts. New models are just old models with more tooling. Training data is exhausted and its a real issue.
  - GrumpyGoblin 9mo ago
    Not saying it's not a problem, I actually don't know, but new CPU's are just old models with more improvements/tooling. Same with TV's. And cars. And clothes. Everything is. That's how improving things works. Running out of raw data doesn't mean running out of room for improvement. The data has been the same for the last 20 years, AI isn't new, things keep improving anyways.
    - zwnow 9mo ago
      Well from cars or CPUs its not expected for them to eventually reach AGI, they also don't eat a trillion dollar hole into us peasants pockets. Sure, improvements can be made. But on a fundamental level, agents/LLMs can not reason (even though they love to act like they can). They are parrots learning words, these parrots wont ever invent new words once the list of words is exhausted though.
  - TeMPOraL 9mo ago
    Well, my experts disagree with your experts :). Sure, the supply of available fresh data is running out, but at the same time, there's way more data than needed. Most of it is low-quality noise anyway. New models aren't just old models with more tooling - the entire training pipeline has been evolving, as researchers and model vendors focus on making better use of data they have, and refining training datasets themselves. There are more stages to LLM training than just the pre-training stage :).
aspenmartin 9mo ago
Youre mixing up several concepts. Synthetic data works for coding because coding is a verifiable domain. You train via reinforcement learning to reward code generation behavior that passes detailed specs and meets other deseridata. It’s literally how things are done today and how progress gets made.
- zwnow 9mo ago
  Most code out there is a legacy security nightmare, surely its good to train on that.
  - dang 9mo ago
    Would you please stop posting cynical, dismissive comments? From a brief scroll through https://news.ycombinator.com/comments?id=zwnow https://news.ycombinator.com/comments?id=zwnow, it seems like your account has been doing nothing else, regardless of the topic that it's commenting on. This is not what HN is for, and destroys what it is for. If you keep this up, we're going to have to ban you, not because of your views on any particular topic but because you're going entirely against the intended spirit of the site by posting this way. There's plenty of room to express your views substantively and thoughtfully, but we don't want cynical flamebait and denunciation. HN needs a good deal less of this. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
    - zwnow 9mo ago
      Then ban me u loser, as I wrote HN is full of pretentious bullshitters. But its good that u wanna ban authentic views. Way to go. If i feel like it I'll just create a new account:-)
      - aspenmartin 9mo ago
        dang is a saint, he wants your opinion, not the other toxic stuff.

lomase 9mo ago

Just one more OPUS bro.

whynotminot 9mo ago
Honestly the scary part is that we don’t really even need one more Opus. If all we had for the rest of our lives was Opus 4.5, the software engineering world would still radically change. But there’s no sign of them slowing down.
deleted 9mo ago
[deleted]

mfalcon 9mo ago

Wake me up at Opus 12

TheOtherHobbes 9mo ago

Objectively, we are talking about systems that have gone from being cute toys to outmatching most juniors using only rigid and slow batch training cycles. As soon as models have persistent memory for their own try/fail/succeed attempts, and can directly modify what's currently called their training data in real time, they're going to develop very, very quickly. We may even be underestimating how quickly this will happen. We're also underestimating how much more powerful they become if you give them analysis and documentation tasks referencing high quality software design principles before giving them code to write. This is very much 1.0 tech. It's already scary smart compared to the median industry skill level. The 2.0 version is going to be something else entirely.