2 ms·

Sure people have written about it but the LLM is only returning text that’s already been written with zero reasoning. What solution architects, devops engineer

Sure people have written about it but the LLM is only returning text that’s already been written with zero reasoning. What solution architects, devops engineers, etc get paid for is their experience. There is a saying that goes “the devil is in the detail” and that applies here. Everytime someone says “I created x using AI” it’s because they’ve been able to feed the LLM pointed questions and have been able to understand when the LLM was rattling off nonsense. And your counter argument there still falls back to the “writing every bit of code” which I’ve already said is the easy part of software development. I’m sure at some point AI will be able to reason about problems but that’s when they shift from being specialised AI to AGI. And I don’t for one second believe we are as nearly as close to AGI as Sam Altman wants his investors to believe.

Philpax 2y ago
I would encourage you to try out a recent reasoning model like Claude 3.7 Sonnet / o3-mini / R1. No, it's not perfect, but it can very much architect and design things at a higher level than you might think. This is already taken advantage of by Aider, which has a mode that splits the tasks: https://aider.chat/2024/09/26/architect.html https://aider.chat/2024/09/26/architect.html My point isn't that it's perfect today, but it's already further along the trajectory than one might think.
- hnlmorg 2y ago
  I use LLMs for brainstorming all the time. It’s probably my favourite use of them. But I always fallback to the same problem: with an open ended question there’s a higher probability of hallucinations. So I still have to spend the time separating out the good responses from that rubbish. You could liken it to a good solutions architecture vs a bad one. A bad one will cost your business time, effort and money. A good one will not. If I were to trust the LLMs output verbatim then I’d have made so many bad decisions. But I have the experience to identify what’s correct and what is not. > No, it's not perfect, but it can very much architect and design things at a higher level Exactly. I keep making the point that the real test of a competent engineer is understanding the details. And there are a lot of details which are highly context specific in any given software project. Where LLMs will thrive in software projects is solving easy problems or problems that have already been solved a thousand times over (eg how to architect a blog) and solving highly specific and narrowly defined problems (eg write code that does X). Maybe I’ve been blessed in my career, but most software projects I’ve worked on have brought something innovative and new to the table (and I don’t mean the “we are changing the world” meme that’s often cited by SV startups — I do literally mean something new to the industry). So LLMs would have a pretty poor success rate under those conditions.

Philpax 2y ago

I would encourage you to try out a recent reasoning model like Claude 3.7 Sonnet / o3-mini / R1. No, it's not perfect, but it can very much architect and design things at a higher level than you might think. This is already taken advantage of by Aider, which has a mode that splits the tasks: https://aider.chat/2024/09/26/architect.html https://aider.chat/2024/09/26/architect.html My point isn't that it's perfect today, but it's already further along the trajectory than one might think.

hnlmorg 2y ago
I use LLMs for brainstorming all the time. It’s probably my favourite use of them. But I always fallback to the same problem: with an open ended question there’s a higher probability of hallucinations. So I still have to spend the time separating out the good responses from that rubbish. You could liken it to a good solutions architecture vs a bad one. A bad one will cost your business time, effort and money. A good one will not. If I were to trust the LLMs output verbatim then I’d have made so many bad decisions. But I have the experience to identify what’s correct and what is not. > No, it's not perfect, but it can very much architect and design things at a higher level Exactly. I keep making the point that the real test of a competent engineer is understanding the details. And there are a lot of details which are highly context specific in any given software project. Where LLMs will thrive in software projects is solving easy problems or problems that have already been solved a thousand times over (eg how to architect a blog) and solving highly specific and narrowly defined problems (eg write code that does X). Maybe I’ve been blessed in my career, but most software projects I’ve worked on have brought something innovative and new to the table (and I don’t mean the “we are changing the world” meme that’s often cited by SV startups — I do literally mean something new to the industry). So LLMs would have a pretty poor success rate under those conditions.