4 ms·

There's no reason AI couldn't do that too. It will continue moving up the cognitive ladder: there's nothing special about our ability to ask those questions. F

There's no reason AI couldn't do that too. It will continue moving up the cognitive ladder: there's nothing special about our ability to ask those questions. For example, try asking Claude "what decisions do I have to make when building a mobile app?" It gave me a list that looked quite a lot like yours, which I then asked it to compact into a single paragraph: > When building a mobile app, you'll need to decide between native or cross-platform development (React Native, Flutter); which platforms to prioritize; backend infrastructure; offline capabilities; data storage approach; UI/UX design system; monetization strategy (free, freemium, subscription); launch approach; ASO strategy; budget allocation across development, design, and marketing; timeline with clear milestones; team composition requirements; and legal compliance needs including privacy policy and relevant regulations like GDPR. and of course, I can ask it to answer those questions for a given app. People have written about this, but the ratio of "input amplification" is only getting larger: in the future, you will be able to ask for a multiplayer 3D chess game with native Android and iOS clients with branching timelines, and it'll write every bit of code for you, while asking you about important decisions. (See also Claude Code and similar agents.)

hnlmorg 2y ago
Sure people have written about it but the LLM is only returning text that’s already been written with zero reasoning. What solution architects, devops engineers, etc get paid for is their experience. There is a saying that goes “the devil is in the detail” and that applies here. Everytime someone says “I created x using AI” it’s because they’ve been able to feed the LLM pointed questions and have been able to understand when the LLM was rattling off nonsense. And your counter argument there still falls back to the “writing every bit of code” which I’ve already said is the easy part of software development. I’m sure at some point AI will be able to reason about problems but that’s when they shift from being specialised AI to AGI. And I don’t for one second believe we are as nearly as close to AGI as Sam Altman wants his investors to believe.
- Philpax 2y ago
  I would encourage you to try out a recent reasoning model like Claude 3.7 Sonnet / o3-mini / R1. No, it's not perfect, but it can very much architect and design things at a higher level than you might think. This is already taken advantage of by Aider, which has a mode that splits the tasks: https://aider.chat/2024/09/26/architect.html https://aider.chat/2024/09/26/architect.html My point isn't that it's perfect today, but it's already further along the trajectory than one might think.
  - hnlmorg 2y ago
    I use LLMs for brainstorming all the time. It’s probably my favourite use of them. But I always fallback to the same problem: with an open ended question there’s a higher probability of hallucinations. So I still have to spend the time separating out the good responses from that rubbish. You could liken it to a good solutions architecture vs a bad one. A bad one will cost your business time, effort and money. A good one will not. If I were to trust the LLMs output verbatim then I’d have made so many bad decisions. But I have the experience to identify what’s correct and what is not. > No, it's not perfect, but it can very much architect and design things at a higher level Exactly. I keep making the point that the real test of a competent engineer is understanding the details. And there are a lot of details which are highly context specific in any given software project. Where LLMs will thrive in software projects is solving easy problems or problems that have already been solved a thousand times over (eg how to architect a blog) and solving highly specific and narrowly defined problems (eg write code that does X). Maybe I’ve been blessed in my career, but most software projects I’ve worked on have brought something innovative and new to the table (and I don’t mean the “we are changing the world” meme that’s often cited by SV startups — I do literally mean something new to the industry). So LLMs would have a pretty poor success rate under those conditions.

hnlmorg 2y ago

Sure people have written about it but the LLM is only returning text that’s already been written with zero reasoning. What solution architects, devops engineers, etc get paid for is their experience. There is a saying that goes “the devil is in the detail” and that applies here. Everytime someone says “I created x using AI” it’s because they’ve been able to feed the LLM pointed questions and have been able to understand when the LLM was rattling off nonsense. And your counter argument there still falls back to the “writing every bit of code” which I’ve already said is the easy part of software development. I’m sure at some point AI will be able to reason about problems but that’s when they shift from being specialised AI to AGI. And I don’t for one second believe we are as nearly as close to AGI as Sam Altman wants his investors to believe.

Philpax 2y ago
I would encourage you to try out a recent reasoning model like Claude 3.7 Sonnet / o3-mini / R1. No, it's not perfect, but it can very much architect and design things at a higher level than you might think. This is already taken advantage of by Aider, which has a mode that splits the tasks: https://aider.chat/2024/09/26/architect.html https://aider.chat/2024/09/26/architect.html My point isn't that it's perfect today, but it's already further along the trajectory than one might think.
- hnlmorg 2y ago
  I use LLMs for brainstorming all the time. It’s probably my favourite use of them. But I always fallback to the same problem: with an open ended question there’s a higher probability of hallucinations. So I still have to spend the time separating out the good responses from that rubbish. You could liken it to a good solutions architecture vs a bad one. A bad one will cost your business time, effort and money. A good one will not. If I were to trust the LLMs output verbatim then I’d have made so many bad decisions. But I have the experience to identify what’s correct and what is not. > No, it's not perfect, but it can very much architect and design things at a higher level Exactly. I keep making the point that the real test of a competent engineer is understanding the details. And there are a lot of details which are highly context specific in any given software project. Where LLMs will thrive in software projects is solving easy problems or problems that have already been solved a thousand times over (eg how to architect a blog) and solving highly specific and narrowly defined problems (eg write code that does X). Maybe I’ve been blessed in my career, but most software projects I’ve worked on have brought something innovative and new to the table (and I don’t mean the “we are changing the world” meme that’s often cited by SV startups — I do literally mean something new to the industry). So LLMs would have a pretty poor success rate under those conditions.