4 ms·

I would encourage you to try out a recent reasoning model like Claude 3.7 Sonnet / o3-mini / R1. No, it's not perfect, but it can very much architect and design

I would encourage you to try out a recent reasoning model like Claude 3.7 Sonnet / o3-mini / R1. No, it's not perfect, but it can very much architect and design things at a higher level than you might think. This is already taken advantage of by Aider, which has a mode that splits the tasks: https://aider.chat/2024/09/26/architect.html https://aider.chat/2024/09/26/architect.html My point isn't that it's perfect today, but it's already further along the trajectory than one might think.

hnlmorg 2y ago
I use LLMs for brainstorming all the time. It’s probably my favourite use of them. But I always fallback to the same problem: with an open ended question there’s a higher probability of hallucinations. So I still have to spend the time separating out the good responses from that rubbish. You could liken it to a good solutions architecture vs a bad one. A bad one will cost your business time, effort and money. A good one will not. If I were to trust the LLMs output verbatim then I’d have made so many bad decisions. But I have the experience to identify what’s correct and what is not. > No, it's not perfect, but it can very much architect and design things at a higher level Exactly. I keep making the point that the real test of a competent engineer is understanding the details. And there are a lot of details which are highly context specific in any given software project. Where LLMs will thrive in software projects is solving easy problems or problems that have already been solved a thousand times over (eg how to architect a blog) and solving highly specific and narrowly defined problems (eg write code that does X). Maybe I’ve been blessed in my career, but most software projects I’ve worked on have brought something innovative and new to the table (and I don’t mean the “we are changing the world” meme that’s often cited by SV startups — I do literally mean something new to the industry). So LLMs would have a pretty poor success rate under those conditions.

hnlmorg 2y ago

I use LLMs for brainstorming all the time. It’s probably my favourite use of them. But I always fallback to the same problem: with an open ended question there’s a higher probability of hallucinations. So I still have to spend the time separating out the good responses from that rubbish. You could liken it to a good solutions architecture vs a bad one. A bad one will cost your business time, effort and money. A good one will not. If I were to trust the LLMs output verbatim then I’d have made so many bad decisions. But I have the experience to identify what’s correct and what is not. > No, it's not perfect, but it can very much architect and design things at a higher level Exactly. I keep making the point that the real test of a competent engineer is understanding the details. And there are a lot of details which are highly context specific in any given software project. Where LLMs will thrive in software projects is solving easy problems or problems that have already been solved a thousand times over (eg how to architect a blog) and solving highly specific and narrowly defined problems (eg write code that does X). Maybe I’ve been blessed in my career, but most software projects I’ve worked on have brought something innovative and new to the table (and I don’t mean the “we are changing the world” meme that’s often cited by SV startups — I do literally mean something new to the industry). So LLMs would have a pretty poor success rate under those conditions.