3 ms·

These articles are a little annoying. I love ChatGPT [1]. I use it all the time. I use it for coding, I use it to generate stuff like form letters, I use it f

These articles are a little annoying. I love ChatGPT [1]. I use it all the time. I use it for coding, I use it to generate stuff like form letters, I use it for parsing out information from PDFs. Point is, I'm not a luddite with this stuff; I'm perfectly happy to play with and use new tech, including but not limited to AI. Which makes me confident when I say this: Anyone who thinks that AI in its current state is "blurring the line between PMs and Engineers" doesn't know what they are talking about. ChatGPT is definitely very useful, but it's nowhere near a replacement for an engineer. ChatGPT is really only useful if you already kind of know what you want. Like, if I asked it "I have a table with the columns name (a string), age (an integer), location (string), can you write me an upsert statement for Postgres for the values 'tom', 34, 'new york'?". This will likely give you exactly what you want, will give you the proper "ON CONFLICT" command, and it's cool and useful. If I ask it "I want to put a value into a table. I also want to make sure that if there's a value in there, we don't just put the value in there, but instead we get the value, update it, and then put the new value back in", it's not as guaranteed to be correct. It might give you the upsert command, but it also might fetch the value from the database, check if it exists, and if it doesn't do an "insert" and if it does do an "update", which is likely incorrect because you risk race conditions. My point is, the first example required knowing what an upsert is, and how to word it in a technical and precise way. It certainly doesn't "blur the line" between PM and engineer for me. I have to pretty heavily modify and babysit its outputs, even when it is giving me useful stuff. You might be saying "well that's what a PM does!!", but not really; project managers aren't typically involved in the technical minutia of a project in my experience, they're not going to correct me for using the wrong kind of upsert. These kinds of articles always seem to be operating on a theoretical "what if AIs could do this??" plane of existence. [1] Deepseek is cool too, what I'm saying applies to that as well. ETA: Even if I wasn't a fan, this article definitely shouldn't have been flagged.

razcle 2y ago
Hey Tombert, wrt did you read the article? I was quite specific about the ways I think LLMs are blurring the lines. I don't think its true for general engineering but I do think its true for applications being built with LLMs. Also its still very early
- tombert 2y ago
  I'm just saying that I don't think I agree with some of this; even if PMs are writing the prompts (and calling that "prompt engineering"), it's not equivalent because they don't know how to audit the code given to them. A PM might generate that SQL thing I mentioned and just blindly cut and paste it. For any application with more than one user, that is a bug, it's incorrect, and it's not like this is some deep cut: upserts happen all over the place. I didn't finish the entire article, I disagreed with the line, "Prompting Is Here To Stay and PMs—Not Engineers—Are Going to Do It", because I fundamentally do not think that is true unless AI models get considerably better. It's possible they will, maybe OpenAI will crack AGI or maybe these models will just get a lot better at figuring out your intent or maybe there's another variable that I'm not thinking of that will fix it. I hate the term "prompt engineer" because I don't think it's engineering, at least not really. I will agree that there's a skill to getting ChatGPT to give you what you want, I think I'm pretty good at it even, but I hesitate to call it engineering because it lacks a lot of "objectivity". I can come up with a "good" prompt that will 90% of the time give me a good answer, but 10% of the time give me utter rubbish, which doesn't really feel like engineering to me. I saw the line: `As AI models become able to write complex applications end-to-end, the work of an engineer will increasingly start to resemble that of a product manager.`, and while I don't completely disagree, I also don't completely agree either. Even when I heavily abuse ChatGPT for code generation, it doesn't feel at all like I'm barking orders to a human. It might superficially resemble it but I'm not convinced that it's actually that similar. I hope I'm not coming off as too much of a dick here, I apologize if I am, and obviously a blog post in which you wax philosophical about the implications of new technology is perfectly fine. I think I'm just a bit on edge with this stuff because you get morons like Zuckerberg claiming they'll be able to replace all their junior and mid level engineers with AI soon, and I think that's ridiculous unless they have access to considerably better models than I do.
  - arcsincosin 2y ago
    My read–which may be wrong–is that much of the article is discussing applications where the end user is interacting with an interface that queries an LLM using baked-in prompts (in one case, a marketing content generation tool). These prompts are being written by the PM. The PM is not writing prompts LLMs to generate code, the PM is writing prompts which are hidden behind a web form or button or something in an interface, hence the prompts being part of the codebase. The author argues that when a PM is editing these prompts they are delivering an artifact that looks more like an engineer's artifact than a PM's artifact, traditionally.
apercu 2y ago
I feel like I agree with you - AI is great if you're a seasoned, experienced person within the domain you are asking help for. If you're not, there is no way to know if what you are getting as feedback is accurate or not. AI is like a lot of "Startup Founders", great at presenting all sorts of things as facts, but if you really dig and drill, they don't really have any domain expertise.

razcle 2y ago

Hey Tombert, wrt did you read the article? I was quite specific about the ways I think LLMs are blurring the lines. I don't think its true for general engineering but I do think its true for applications being built with LLMs. Also its still very early

tombert 2y ago
I'm just saying that I don't think I agree with some of this; even if PMs are writing the prompts (and calling that "prompt engineering"), it's not equivalent because they don't know how to audit the code given to them. A PM might generate that SQL thing I mentioned and just blindly cut and paste it. For any application with more than one user, that is a bug, it's incorrect, and it's not like this is some deep cut: upserts happen all over the place. I didn't finish the entire article, I disagreed with the line, "Prompting Is Here To Stay and PMs—Not Engineers—Are Going to Do It", because I fundamentally do not think that is true unless AI models get considerably better. It's possible they will, maybe OpenAI will crack AGI or maybe these models will just get a lot better at figuring out your intent or maybe there's another variable that I'm not thinking of that will fix it. I hate the term "prompt engineer" because I don't think it's engineering, at least not really. I will agree that there's a skill to getting ChatGPT to give you what you want, I think I'm pretty good at it even, but I hesitate to call it engineering because it lacks a lot of "objectivity". I can come up with a "good" prompt that will 90% of the time give me a good answer, but 10% of the time give me utter rubbish, which doesn't really feel like engineering to me. I saw the line: `As AI models become able to write complex applications end-to-end, the work of an engineer will increasingly start to resemble that of a product manager.`, and while I don't completely disagree, I also don't completely agree either. Even when I heavily abuse ChatGPT for code generation, it doesn't feel at all like I'm barking orders to a human. It might superficially resemble it but I'm not convinced that it's actually that similar. I hope I'm not coming off as too much of a dick here, I apologize if I am, and obviously a blog post in which you wax philosophical about the implications of new technology is perfectly fine. I think I'm just a bit on edge with this stuff because you get morons like Zuckerberg claiming they'll be able to replace all their junior and mid level engineers with AI soon, and I think that's ridiculous unless they have access to considerably better models than I do.
- arcsincosin 2y ago
  My read–which may be wrong–is that much of the article is discussing applications where the end user is interacting with an interface that queries an LLM using baked-in prompts (in one case, a marketing content generation tool). These prompts are being written by the PM. The PM is not writing prompts LLMs to generate code, the PM is writing prompts which are hidden behind a web form or button or something in an interface, hence the prompts being part of the codebase. The author argues that when a PM is editing these prompts they are delivering an artifact that looks more like an engineer's artifact than a PM's artifact, traditionally.

apercu 2y ago

I feel like I agree with you - AI is great if you're a seasoned, experienced person within the domain you are asking help for. If you're not, there is no way to know if what you are getting as feedback is accurate or not. AI is like a lot of "Startup Founders", great at presenting all sorts of things as facts, but if you really dig and drill, they don't really have any domain expertise.