6 ms·
I just used o3 to design a distributed scheduler that scales to 1M+ sxchedules a day. It was perfect, and did better than two weeks of thought around the best w
by codingwagie 1y ago
I just used o3 to design a distributed scheduler that scales to 1M+ sxchedules a day. It was perfect, and did better than two weeks of thought around the best way to build this.
- csto12 1y agoYou just asked it to design or implement? If o3 can design it, that means it’s using open source schedulers as reference. Did you think about opening up a few open source projects to see how they were doing things in those two weeks you were designing?
- mprast 1y agoyeah unless you have very specific requirements I think the baseline here is not building/designing it yourself but setting up an off-the-shelf commercial or OSS solution, which I doubt would take two weeks...
- torginus 1y agoDunno, in work we wanted to implement a task runner that we could use to periodically queue tasks through a web UI - it would then spin up resources on AWS and track the progress and archive the results. We looked at the existing solutions, and concluded that customizing them to meet all our requirements would be a giant effort. Meanwhile I fed the requirement doc into Claude Sonnet, and with about 3 days of prompting and debugging we had a bespoke solution that did exactly what we needed.
- codingwagie 1y agothe future is more custom software designed by ai, not less. alot of frameworks will disappear once you can build sophisticated systems yourself. people are missing this
- rsynnott 1y agoThat's a future with a _lot_ more bugs.
- codingwagie 1y agoyoure assuming humans built it. also, a ton of complexity in software engineering is really due to having to fit a business domain into a string of interfaces in different libraries and technical infrastructure
- 9rx 1y agoWhat else is going to build it? Lions? The only real complexity in software is describing it. There is no evidence that the tools are going to ever help with that. Maybe some kind of device attached directly to the brain that can sidestep the parts that get in the way, but that is assuming some part of the brain is more efficient than it seems through the pathways we experience it through. It could also be that the brain is just fatally flawed.
- deleted 1y ago[deleted]
- cmsj 1y agoThat's a future paid for by the effort of creating current frameworks, and it's a stagnant future where every "sophisticated system" is just re-hashing the last human frameworks ever created.
- namaria 1y agoBingo. LLMs are consuming data. They cannot generate new information, they can only give back what already exists or mangle it. It is inevitable that they will degrade the total sum of information.
- codingwagie 1y agowhy would I do that kind of research if it can identify the problem I am trying to solve, and spit out the exact solution. also, it was a rough implementation adapted to my exact tech stack
- kazinator 1y agoSo you could stick your own copyright notice on the result, for one thing.
- ben_w 1y agoWhat's the point holding copyright on a new technical solution, to a problem that can be solved by anyone asking an existing AI, trained on last year's internet, independently of your new copyright?
- kazinator 1y agoAll sorts of stuff containing no original ideas is copyrighted. It legally belongs to someone and they can license it to others, etc. E.g. pop songs with no original chord progressions or melodies, and hackneyed lyrics are still copyrighted. Plagiarized and uncopyrightable code is radioactive; it can't be pulled into FOSS or commercial codebases alike.
- alabastervlog 1y agoSomeone raised the point in another recent HN LLM thread that the primary productivity benefit of LLMs in programing is the copyright laundering. The argument went that the main reason the now-ancient push for code reuse failed to deliver anything close to its hypothetical maximum benefit was because copyright got in the way. Result: tons and tons of wheel-reinvention, like, to the point that most of what programmers do day to day is reinvent wheels. LLMs essentially provide fine-grained contextual search of existing code, while also stripping copyright from whatever they find. Ta-da! Problem solved.
- cmsj 1y agoThere is one very specific risk worth mentioning: AI code is a potentially existential crisis for Open Source. An ecosystem that depends on copyright can't exist if its codebase is overrun by un-copyrightable code.
- davidsainez 1y agoWhile impressive, I'm not convinced that improved performance on tasks of this nature are indicative of progress toward AGI. Building a scheduler is a well studied problem space. Something like the ARC benchmark is much more indicative of progress toward true AGI, but probably still insufficient.
- codingwagie 1y agothe other models failed at this miserably. There were also specific technical requirements I gave it related to my tech stack
- fragmede 1y agoThe point is that AGI is the wrong bar to be aiming for. LLMs are sufficiently useful at their current state that even if it does take us 30 years to get to AGI, even just incremental improvements from now until then, they'll still be useful enough to provide value to users/customers for some companies to win big. VC funding will run out and some companies won't make it, but some of them will, to the delight of their investors. AGI when? is an interesting question, but might just be academic. we have self driving cars, weight loss drugs that work, reusable rockets, and useful computer AI. We're living in the future, man, and robot maids are just around the corner.
- MisterSandman 1y agoDesigning a distributed scheduler is a solved problem, of course an LLM was able to spit out a solution.
- codingwagie 1y agoas noted elsewhere, all other frontier models failed miserably at this
- daveguy 1y agoThat doesn't mean the one what manages to spit it out of its latent space is close to AGI. I wonder how consistently that specific model could. If you tried 10 LLMs maybe all 10 of them could have spit out the answer 1 out of 10 times. Correct problem retrieval by one LLM and failure by the others isn't a great argument for near-AGI. But LLMs will be useful in limited domains for a long time.
- alabastervlog 1y agoIt is unsurprising that some lossily-compressed-database search programs might be worse for some tasks than other lossily-compressed-database search programs.
- littlestymaar 1y ago“It does something well” ≠ “it will become AGI”. Your anodectical example isn't more convincing than “This machine cracked Enigma's messages in less time than an army of cryptanalysts over a month, surely we're gonna reach AGI by the end of the decade” would have.
- timeon 1y agoI'm not sure what is your point in context of AGI topic.
- codingwagie 1y agoim a tenured engineer, spent a long time at faang. was casually beat this morning by a far superior design from an llm.
- darod 1y agois this because the LLM actually reasoned on a better design or because it found a better design in its "database" scoured from another tenured engineer.
- anthonypasq 1y agowho cares?
- awkwardpotato 1y agoIgnoring the copyright issues, credit issues, and any ethical concerns... this approach doesn't work for anything not in the "database", it's not AGI and the tangential experience is barely relevant to the article.
- ben_w 1y agoDoes it matter if the thing a submarine does counts as "swimming"? We get paid to solve problems, sometimes the solution is to know an existing pattern or open source implementation and use it. Aguably it usually is: we seldom have to invent new architectures, DSLs, protocols, or OSes from scratch, but even those are patterns one level up. Whatever the AI is inside, doesn't matter: this was it solving a problem.
- AJ007 1y agoI find now I quickly bucket people in to "have not/have barely used the latest AI models" or "trolls" when they express a belief current LLMs aren't intelligent.
- tumsfestival 1y agoCall me back when ChatGPT isn't hallucinating half the outputs it gives me.
- machomaster 1y agoWrite me when humans will achieve hallucination levels smaller than ChatGPT.
- burnte 1y agoYou can put me in that bucket then. It's not true, I've been working with AI almost daily for 18 months, and I KNOW it's no where close to being intelligent, but it doesn't look like your buckets are based on truth but appeal. I disagree with your assessment so you think I don't know what I'm talking about. I hope you can understand that other people who know just as much as you (or even more) can disagree without being wrong or uninformed. LLMs are amazing, but they're nowhere close to intelligent.
- dundarious 1y agoWow, 12 per second on average.
- mountainriver 1y agoI’ve had similar things over the last couple days with o3. It was one-shotting whole features into my Rust codebase. Very impressive. I remember before ChatGPT, smart people would come on podcasts and say we were 100 or 300 years away from AGI. Then we saw GPT shock them. The reality is these people have no idea, it’s just catchy to talk this way. With the amount of money going into the problem and the linear increases we see over time, it’s much more likely we see AGI sooner than later.