4 ms·

Why do you belive that agents wouldn't be able to take over product management, and generate prompts for the "software engineer" agents?

by 0c3ca83 16d ago

reedlaw 16d ago
Because agents lack human judgment. At the very least there's a need for a human-in-the-loop with agentic processes. Otherwise, it's like running a coding harness with --dangerously-skip-permissions all the time.
- 0c3ca83 16d ago
  Why do you think judgement is impossible to automate? What aspects of it do you think make it hard?
  - noir_lord 16d ago
    It requires general intelligence and we don't even have a good understanding of how our's works or a particularly good way of quantifying it. The counter argument is of course maybe you don't need to understand our kind of intelligence to create a different kind and that could well be true but then how do you determine if a system is intelligent. Unless the new system is intelligent enough to reason with us on our level in a way we can "see" is intelligent it becomes a philosophical argument. We also have a natural inclination towards anthropomorphising systems that mirror us, this is already a problem with LLM's and people overestimating their capabilities or forming actual emotional attachment. Then there are those of us who know more about how they work who in theory should be more immune to that and aren't. I added some stuff to my agent.md to make it sound less human and to communicate more like the machine be it is because I find the faked emotion extremely jarring. It can't be sorry, it's a set of numbers, it sits in the linguistic uncanny valley.
    - Analemma_ 16d ago
      Why do you think it requires general intelligence? The parent and I aren't being obtuse here: the history of artificial intelligence research is littered with examples of humans confidently declaring that task X requires general intelligence, then getting humiliated by a neural network doing task X better than humans a few years later. See Go, driving, art (you can complain about the quality of AI art, but it's winning competitions with human judges), etc. A priori I'm not sure why you would think being a PM at a FAANG, deciding what color the login button should be, is any different.
      - pixl97 16d ago
        Hence my saying. We will never prove AI is intelligence. We'll only prove humans are not.
        Analemma_ 16d ago
        I mean I could believe that there are some tasks which can't be done at acceptable performance until AI resembles something like Commander Data, I just don't think being a software PM is one of them.
      - thinkharderdev 16d ago
        I'm not entirely sure "general" intelligence is a meaningful concept but I take judgement to mean something like "the ability to make decisions under uncertainty without direct feedback." People seem to be able to do this using a lot of context and tacit knowledge. It's not really clear how you hill-climb to that capability through RL. > the history of artificial intelligence research is littered with examples of humans confidently declaring that task X requires general intelligence, then getting humiliated by a neural network doing task X better than humans a few years later True, but the history of AI research is also littered with AI researchers confidently predicting X job will be replaced by AI and being completely wrong because they don't actually understand what those jobs actually are. See Geoff Hinton predicting that Radiologists would be obsolete by ~2020, or predictions that truck drivers would all be replaced by self-driving tech.
      - devonkim 16d ago
  - vector_spaces 16d ago
    Judgments are not generally impossible to automate -- judgements are typically binary or quantifiable interpretations, so in some sense are perfect targets for automation, but the sheer volume of judgements needed to build something coherent is overly cumbersome to specify to the point of being intractable. There are also many hidden judgements, ones where the thresholds may not be well understood, and interactions between them. But humans still manage to wrangle these, sometimes seemingly effortlessly, through a process which we call by shorthand "taste". This is a largely vibes-based heuristic that combines expertise with life experience and cultural training -- intuition, more or less. This is likely not possible to automate either -- aspects of it may be automatable for a given expert, in small pieces in narrow subsets of their particular domains of interest, but even those likely will require some manual intervention. This is in part because it is, to a large degree, a black box, even to the expert deploying it. With some self-awareness and strong language skills we can articulate approximations of the judgements that go into taste. But even those will fall short, as even the most self-aware individual will fail to notice certain judgements and dependencies. In practice many of these are not even explicitly articulable. Humans are idiosyncratic and messy and dynamic, and the suggestion that we can build a machine that approximates this in a way that pleases our sensibilities and doesn't require supervision is kind of ludicrous, even in light of recent developments.
    - 0c3ca83 16d ago
      Taste is just a set of statistically expressible heuristics for what other people will think is good. I don't undestand why you think this is impossible for AI to do. It feels like today, taste in design is similar to where software engineering was about a year ago.
    - visarga 16d ago
      You don't need to describe it; just show samples of the style you want to achieve. Of course it's not perfect, but it's easier than describing it. I have my own theory about why it's impossible to remove the human from the loop: 1. Any task emerges from a need, from a human context. We need the human to pay and assume the risks and costs of using the model. So intent emerges from context. 2. While the task is being worked on, constant interaction with the context is needed, for action, for feedback, and for steering. 3. At the end of a task, consequences accumulate in the context, they don't fly to the model provider. The cost, risk, liability, gains and losses remain there. So the LLM is great except for the start, middle and end of a task. Contexts are humans, teams, projects, and they are maximally distributed, you can't copy a context, it is indexical and relational, just as you can't copy my phone number or eat for me.
  - d0mine 16d ago
    At the moment, the hurdle is [missing] context [aka shared model/understanding of the business, common sense]. Though it is not hard to imagine that any and all communications being recorded for AI consumption in the future. The Next one is the relative lack of prompt feedback (expect the blowup in finite time like Navier-Stokes ;) [there is not much feedback even for humans at middle management positions]. The cost [tokens] might become prohibitive unless LLMs improve further [not a guarantee].
  - jmcqk6 16d ago
    Not the OP, but it's because the value has got to ultimately be recognized by humans. You can make llms perform judgement, and maybe that will get you some progress. But ultimately the value is going to come from engaging with other humans.
  - freejazz 16d ago
    I think right here you're demonstrating that judgment doesn't come easily
snapetom 16d ago
When done right, product management is 90% idea generation and market/user research. So many companies do it poorly that PdMs are beaten to think their job is to wallow in bureaucracy and metrics. LLMs are great at helping with aspects of market research and that’s about it. Aka it’s a good deep search engine. It’s not going to decide what features solve certain customer pain points. It’s certainly not going to prioritize and coordinate between competing stakeholders.

Why do you belive that agents wouldn't be able to take over product management, and generate prompts for the "software engineer" agents?

by 0c3ca83 16d ago

Why do you belive that agents wouldn't be able to take over product management, and generate prompts for the "software engineer" agents?

reedlaw 16d ago

Because agents lack human judgment. At the very least there's a need for a human-in-the-loop with agentic processes. Otherwise, it's like running a coding harness with --dangerously-skip-permissions all the time.

0c3ca83 16d ago
Why do you think judgement is impossible to automate? What aspects of it do you think make it hard?
- noir_lord 16d ago
  It requires general intelligence and we don't even have a good understanding of how our's works or a particularly good way of quantifying it. The counter argument is of course maybe you don't need to understand our kind of intelligence to create a different kind and that could well be true but then how do you determine if a system is intelligent. Unless the new system is intelligent enough to reason with us on our level in a way we can "see" is intelligent it becomes a philosophical argument. We also have a natural inclination towards anthropomorphising systems that mirror us, this is already a problem with LLM's and people overestimating their capabilities or forming actual emotional attachment. Then there are those of us who know more about how they work who in theory should be more immune to that and aren't. I added some stuff to my agent.md to make it sound less human and to communicate more like the machine be it is because I find the faked emotion extremely jarring. It can't be sorry, it's a set of numbers, it sits in the linguistic uncanny valley.
  - Analemma_ 16d ago
    Why do you think it requires general intelligence? The parent and I aren't being obtuse here: the history of artificial intelligence research is littered with examples of humans confidently declaring that task X requires general intelligence, then getting humiliated by a neural network doing task X better than humans a few years later. See Go, driving, art (you can complain about the quality of AI art, but it's winning competitions with human judges), etc. A priori I'm not sure why you would think being a PM at a FAANG, deciding what color the login button should be, is any different.
    - pixl97 16d ago
      Hence my saying. We will never prove AI is intelligence. We'll only prove humans are not.
      - Analemma_ 16d ago
        I mean I could believe that there are some tasks which can't be done at acceptable performance until AI resembles something like Commander Data, I just don't think being a software PM is one of them.
    - thinkharderdev 16d ago
      I'm not entirely sure "general" intelligence is a meaningful concept but I take judgement to mean something like "the ability to make decisions under uncertainty without direct feedback." People seem to be able to do this using a lot of context and tacit knowledge. It's not really clear how you hill-climb to that capability through RL. > the history of artificial intelligence research is littered with examples of humans confidently declaring that task X requires general intelligence, then getting humiliated by a neural network doing task X better than humans a few years later True, but the history of AI research is also littered with AI researchers confidently predicting X job will be replaced by AI and being completely wrong because they don't actually understand what those jobs actually are. See Geoff Hinton predicting that Radiologists would be obsolete by ~2020, or predictions that truck drivers would all be replaced by self-driving tech.
    - devonkim 16d ago
- vector_spaces 16d ago
  Judgments are not generally impossible to automate -- judgements are typically binary or quantifiable interpretations, so in some sense are perfect targets for automation, but the sheer volume of judgements needed to build something coherent is overly cumbersome to specify to the point of being intractable. There are also many hidden judgements, ones where the thresholds may not be well understood, and interactions between them. But humans still manage to wrangle these, sometimes seemingly effortlessly, through a process which we call by shorthand "taste". This is a largely vibes-based heuristic that combines expertise with life experience and cultural training -- intuition, more or less. This is likely not possible to automate either -- aspects of it may be automatable for a given expert, in small pieces in narrow subsets of their particular domains of interest, but even those likely will require some manual intervention. This is in part because it is, to a large degree, a black box, even to the expert deploying it. With some self-awareness and strong language skills we can articulate approximations of the judgements that go into taste. But even those will fall short, as even the most self-aware individual will fail to notice certain judgements and dependencies. In practice many of these are not even explicitly articulable. Humans are idiosyncratic and messy and dynamic, and the suggestion that we can build a machine that approximates this in a way that pleases our sensibilities and doesn't require supervision is kind of ludicrous, even in light of recent developments.
  - 0c3ca83 16d ago
    Taste is just a set of statistically expressible heuristics for what other people will think is good. I don't undestand why you think this is impossible for AI to do. It feels like today, taste in design is similar to where software engineering was about a year ago.
  - visarga 16d ago
    You don't need to describe it; just show samples of the style you want to achieve. Of course it's not perfect, but it's easier than describing it. I have my own theory about why it's impossible to remove the human from the loop: 1. Any task emerges from a need, from a human context. We need the human to pay and assume the risks and costs of using the model. So intent emerges from context. 2. While the task is being worked on, constant interaction with the context is needed, for action, for feedback, and for steering. 3. At the end of a task, consequences accumulate in the context, they don't fly to the model provider. The cost, risk, liability, gains and losses remain there. So the LLM is great except for the start, middle and end of a task. Contexts are humans, teams, projects, and they are maximally distributed, you can't copy a context, it is indexical and relational, just as you can't copy my phone number or eat for me.
- d0mine 16d ago
  At the moment, the hurdle is [missing] context [aka shared model/understanding of the business, common sense]. Though it is not hard to imagine that any and all communications being recorded for AI consumption in the future. The Next one is the relative lack of prompt feedback (expect the blowup in finite time like Navier-Stokes ;) [there is not much feedback even for humans at middle management positions]. The cost [tokens] might become prohibitive unless LLMs improve further [not a guarantee].
- jmcqk6 16d ago
  Not the OP, but it's because the value has got to ultimately be recognized by humans. You can make llms perform judgement, and maybe that will get you some progress. But ultimately the value is going to come from engaging with other humans.
- freejazz 16d ago
  I think right here you're demonstrating that judgment doesn't come easily

snapetom 16d ago

When done right, product management is 90% idea generation and market/user research. So many companies do it poorly that PdMs are beaten to think their job is to wallow in bureaucracy and metrics. LLMs are great at helping with aspects of market research and that’s about it. Aka it’s a good deep search engine. It’s not going to decide what features solve certain customer pain points. It’s certainly not going to prioritize and coordinate between competing stakeholders.