4 ms·

I'm a product manager, and a lot of the things I see people do wrong is because they don't have any product management experience. It takes quite a bit of work

I'm a product manager, and a lot of the things I see people do wrong is because they don't have any product management experience. It takes quite a bit of work to develop a really good theory of what should be in your functional spec. Edge cases come up all the time in real software engineering, and often handling all those cases is spread across multiple engineers. A good product manager has a view of all of it, expects many of those issues from the agent, and plans for coaching it through them.

nevertoolate 7mo ago
Poe’s law is strong with this one
- Schiendelman 7mo ago
  Tell me more! I'm trying to figure out how you got that.
  - causal 7mo ago
    I think it's just sarcasm coming from the stereotypical HN attitude that Product Managers only get in the way of the real work of engineering. Ignore it; they're basically proving your point.
  - nevertoolate 7mo ago
    I understand that you are serious. I am also serious here. Have you built anything purely with LLM which is novel and is used by people who expect that their data is managed securely, and the application is well maintained so they can trust it? I have been writing specifications, rfcs, adrs, conducting architecture reviews, code reviews and what not for quite a bit of time now. Also I’ve driven cross organisational product initiatives etc. I’m experimenting with openspec with my team now on a brownfield project and have some good results. Having said all that I seriously doubt that if you treat the english language spec and your pm oversight as the sole QA pillars of a stochastic model transformer you are making a mistake.
    - Schiendelman 7mo ago
      Well, I was a QA engineer and an SDET before being a product manager, how does that change your opinion?
      - nevertoolate 7mo ago
        I think it is great! The issue is that validation needs presence and it is the limiting factor - common knowledge, but is part of the “physics”. Also maintenance gets really tricky if the codebase has warts in it - which it will have. I get much more easy to understand architecture out of an LLM driven code generation process if I follow it and course correct / update the spec process based on learnings. Example: yesterday I’ve introduced a batch job and realized during the implementation phase that some refactoring is needed so the error boundary can be reused in the batch application from the main backend. This was unplanned and definitely not a functional requirement - could be documented as non-functional. There was a gap between the agent’s knowledge and mine even though the error handling pattern is well documented in the repository. Of course this can be documented better next time if we update the process of openspec writing but having these gaps is inevitable unless formal and half-formal definitions are introduced - but still there needs to be someone with “fresh eyes” in the loop.
- tristor 7mo ago
  I think that's an incredibly reductionist and sarcastic take. I'm also in Product, but was an engineer for over a decade prior. I find that having strong structured functional specifications and a good holistic understanding of the solution you're trying to build goes a long way with AI tooling. Just like any software project, eliminating false starts and getting a clear set of requirements up front can minimize engineering time required to complete something, as long as things don't change in the middle. When your cycle time is an afternoon instead of two quarters, that type of up front investment pays off much better. I still think AI tooling is lacking, but you can get significantly better results by structuring your plans appropriately.
causal 7mo ago
I'm an engineer and I totally agree. Engineers + LLMs exacerbate the timeless problem of not understanding the reality behind the problem. Validating solutions against reality is hard and LLMs just hallucinate their way around unknowns.

nevertoolate 7mo ago

Poe’s law is strong with this one

Schiendelman 7mo ago
Tell me more! I'm trying to figure out how you got that.
- causal 7mo ago
  I think it's just sarcasm coming from the stereotypical HN attitude that Product Managers only get in the way of the real work of engineering. Ignore it; they're basically proving your point.
- nevertoolate 7mo ago
  I understand that you are serious. I am also serious here. Have you built anything purely with LLM which is novel and is used by people who expect that their data is managed securely, and the application is well maintained so they can trust it? I have been writing specifications, rfcs, adrs, conducting architecture reviews, code reviews and what not for quite a bit of time now. Also I’ve driven cross organisational product initiatives etc. I’m experimenting with openspec with my team now on a brownfield project and have some good results. Having said all that I seriously doubt that if you treat the english language spec and your pm oversight as the sole QA pillars of a stochastic model transformer you are making a mistake.
  - Schiendelman 7mo ago
    Well, I was a QA engineer and an SDET before being a product manager, how does that change your opinion?
    - nevertoolate 7mo ago
      I think it is great! The issue is that validation needs presence and it is the limiting factor - common knowledge, but is part of the “physics”. Also maintenance gets really tricky if the codebase has warts in it - which it will have. I get much more easy to understand architecture out of an LLM driven code generation process if I follow it and course correct / update the spec process based on learnings. Example: yesterday I’ve introduced a batch job and realized during the implementation phase that some refactoring is needed so the error boundary can be reused in the batch application from the main backend. This was unplanned and definitely not a functional requirement - could be documented as non-functional. There was a gap between the agent’s knowledge and mine even though the error handling pattern is well documented in the repository. Of course this can be documented better next time if we update the process of openspec writing but having these gaps is inevitable unless formal and half-formal definitions are introduced - but still there needs to be someone with “fresh eyes” in the loop.
tristor 7mo ago
I think that's an incredibly reductionist and sarcastic take. I'm also in Product, but was an engineer for over a decade prior. I find that having strong structured functional specifications and a good holistic understanding of the solution you're trying to build goes a long way with AI tooling. Just like any software project, eliminating false starts and getting a clear set of requirements up front can minimize engineering time required to complete something, as long as things don't change in the middle. When your cycle time is an afternoon instead of two quarters, that type of up front investment pays off much better. I still think AI tooling is lacking, but you can get significantly better results by structuring your plans appropriately.

causal 7mo ago

I'm an engineer and I totally agree. Engineers + LLMs exacerbate the timeless problem of not understanding the reality behind the problem. Validating solutions against reality is hard and LLMs just hallucinate their way around unknowns.