4 ms·

My company has a group of younger product managers who are very obviously using ChatGPT to write proposals, e-mails, and other documents. It's obvious because

My company has a group of younger product managers who are very obviously using ChatGPT to write proposals, e-mails, and other documents. It's obvious because their writings in Confluence and late night e-mails suddenly became very verbose and used the sometimes obvious ChatGPT voice. There's a stark contrast between content they write/share on the spot or discuss in meetings. Then you get them into a meeting to explain further and they some times can't even remember what they were thinking when they wrote various bullet points. It's eye opening to see how much the ChatGPT produced content can fool managers at the top who catch glimpses of neatly composed and verbose documents. Then you get the people into meetings and they don't really know where they were going with their "own" documents. It wouldn't be a problem if they could take the documents as some sort of next step for their learning or research, but it's really becoming useless to have all of this ChatGPT-created content out there where none of them can really execute on it without going back to ChatGPT over and over again to come up with more content.

wongarsu 4y ago
> It's eye opening to see how much the ChatGPT produced content can fool managers at the top who catch glimpses of neatly composed and verbose documents. Goodhart's law in action. If "neatly composed and verbose documents" is the metric higher ups use to judge product managers, then documents will become neater and more verbose to the detriment of everything else. In the end, the alignment problems that plague AI safety research are all playing out in company structures every day. It's hard to give a metric that measures your intended outcome, and any proxy leads to unintended consequences once there's an easier way to reach that proxy.
- 13years 4y ago
  In the end, the alignment problems that plague AI safety research are all playing out in company structures every day. It's hard to give a metric that measures your intended outcome, and any proxy leads to unintended consequences once there's an easier way to reach that proxy. I've actually pointed this out as a fallacy of the premise of current alignment theory. That being we simply need to apply humanities values for the AI to be properly aligned. However, every alignment problem can also be seen as an existing human problem for which we have never resolved.
  - jacquesm 4y ago
    The scale is the key. Things that have impact at scale are harmless initially and then in the long run the damage becomes visible and by then we'll be as dependent on them as a heroin junk is on their fix. Oil, plastic, electricity, natural gas, mines, wood, poultry, meat & fish, transportation etc. They're nice and super useful and when used in moderation or when there aren't that many humans to begin with and you can easily absorb the consequences. But if you multiply the number of people while at the same time you increase the consumption/production on some or even all of these in tandem the effects are massively compounding.
    - 13years 4y ago
      Yes, but that is just one facet. Alignment theory affects everything regardless of scale. It is about whether or not you can reliably get the intended result given a particular goal or task. We want AI to reason like humans and have humanities values. According alignment theory. Yet humans will also cheat to get a result. Our values also are what put us in conflict with each other. FYI, I've elaborated in much more detail here on such issues with containment/alignment - https://dakara.substack.com/p/ai-singularity-the-hubris-trap https://dakara.substack.com/p/ai-singularity-the-hubris-trap
- dTal 4y ago
  >the alignment problems that plague AI safety research are all playing out in company structures every day. Yes yes yes holy crap yes. Corporations are AI, and we are already in the AI apocalypse. Corporations exhibit emergent behavior that any individual human involved with them is powerless to stop, and they self-optimize towards inhuman goals. Coca Cola inc is a paperclip maximizer, except it maximizes the consumption of flavored sugar water. We are completely failing as a society to adequately rein them in.
  - jacquesm 4y ago
    There is this movie called Idiocracy. When I saw it the first time I thought it was hilarious. But over time I've stopped laughing and I now more or less realize that the makers may well have intended something funny but accidentally ended up making a prophesy. Or maybe they intended that from the get-go but they dressed it up as something funny for budget reasons. It's scary how many of the things that are in that movie as caricatures you can now find IRL.
    - deleted 4y ago
      [deleted]
    - saulpw 4y ago
      Mike Judge is a genius. He also did Office Space and Silicon Valley, among other things (I personally love King of the Hill). He has a sharp eye for both society and humor.
  - RangerScience 4y ago
    Fav Dune quote (it's from book 4): > The problem isn't thinking machines, it's letting machines do the thinking for you ...and then I like to point out that many of our social structures (including corporations) are, essentially, machines. They're just machines running on meat instead of silicon.
bluescrn 4y ago
> There's a stark contrast between content they write/share on the spot or discuss in meetings. I'd be worried about false accusations of this. Some people are much better at communicating in writing, when they have time to think about how to word things, and not so good when put on the spot in a meeting. Others might be better at communicating verbally, and perhaps too verbose in written communication.
- travisjungroth 4y ago
  Shouldn’t fire off on someone for seeming smarter in writing than in meetings. But if they don’t even seem to understand the things “they” wrote about and they finish JIRA tickets with “in conclusion”…
- PragmaticPulp 4y ago
  You're exactly right: This isn't a situation to jump to conclusions. However, when the junior product managers are proposing things with terms they don't even understand then something is clearly wrong. Imagine joining a meeting where a product manager is trying to explain their own document but can't understand what "they" wrote when they get to various bullet points. Or the document is correct, but their understanding of their own document is incorrect. Have you ever seen a student caught cheating? Someone who has the right answers on the paper, but can't explain why or remember why they wrote it down? It's exactly that.

wongarsu 4y ago

> It's eye opening to see how much the ChatGPT produced content can fool managers at the top who catch glimpses of neatly composed and verbose documents. Goodhart's law in action. If "neatly composed and verbose documents" is the metric higher ups use to judge product managers, then documents will become neater and more verbose to the detriment of everything else. In the end, the alignment problems that plague AI safety research are all playing out in company structures every day. It's hard to give a metric that measures your intended outcome, and any proxy leads to unintended consequences once there's an easier way to reach that proxy.

13years 4y ago
In the end, the alignment problems that plague AI safety research are all playing out in company structures every day. It's hard to give a metric that measures your intended outcome, and any proxy leads to unintended consequences once there's an easier way to reach that proxy. I've actually pointed this out as a fallacy of the premise of current alignment theory. That being we simply need to apply humanities values for the AI to be properly aligned. However, every alignment problem can also be seen as an existing human problem for which we have never resolved.
- jacquesm 4y ago
  The scale is the key. Things that have impact at scale are harmless initially and then in the long run the damage becomes visible and by then we'll be as dependent on them as a heroin junk is on their fix. Oil, plastic, electricity, natural gas, mines, wood, poultry, meat & fish, transportation etc. They're nice and super useful and when used in moderation or when there aren't that many humans to begin with and you can easily absorb the consequences. But if you multiply the number of people while at the same time you increase the consumption/production on some or even all of these in tandem the effects are massively compounding.
  - 13years 4y ago
    Yes, but that is just one facet. Alignment theory affects everything regardless of scale. It is about whether or not you can reliably get the intended result given a particular goal or task. We want AI to reason like humans and have humanities values. According alignment theory. Yet humans will also cheat to get a result. Our values also are what put us in conflict with each other. FYI, I've elaborated in much more detail here on such issues with containment/alignment - https://dakara.substack.com/p/ai-singularity-the-hubris-trap https://dakara.substack.com/p/ai-singularity-the-hubris-trap
dTal 4y ago
>the alignment problems that plague AI safety research are all playing out in company structures every day. Yes yes yes holy crap yes. Corporations are AI, and we are already in the AI apocalypse. Corporations exhibit emergent behavior that any individual human involved with them is powerless to stop, and they self-optimize towards inhuman goals. Coca Cola inc is a paperclip maximizer, except it maximizes the consumption of flavored sugar water. We are completely failing as a society to adequately rein them in.
- jacquesm 4y ago
  There is this movie called Idiocracy. When I saw it the first time I thought it was hilarious. But over time I've stopped laughing and I now more or less realize that the makers may well have intended something funny but accidentally ended up making a prophesy. Or maybe they intended that from the get-go but they dressed it up as something funny for budget reasons. It's scary how many of the things that are in that movie as caricatures you can now find IRL.
  - deleted 4y ago
    [deleted]
  - saulpw 4y ago
    Mike Judge is a genius. He also did Office Space and Silicon Valley, among other things (I personally love King of the Hill). He has a sharp eye for both society and humor.
- RangerScience 4y ago
  Fav Dune quote (it's from book 4): > The problem isn't thinking machines, it's letting machines do the thinking for you ...and then I like to point out that many of our social structures (including corporations) are, essentially, machines. They're just machines running on meat instead of silicon.

bluescrn 4y ago

> There's a stark contrast between content they write/share on the spot or discuss in meetings. I'd be worried about false accusations of this. Some people are much better at communicating in writing, when they have time to think about how to word things, and not so good when put on the spot in a meeting. Others might be better at communicating verbally, and perhaps too verbose in written communication.

travisjungroth 4y ago
Shouldn’t fire off on someone for seeming smarter in writing than in meetings. But if they don’t even seem to understand the things “they” wrote about and they finish JIRA tickets with “in conclusion”…
PragmaticPulp 4y ago
You're exactly right: This isn't a situation to jump to conclusions. However, when the junior product managers are proposing things with terms they don't even understand then something is clearly wrong. Imagine joining a meeting where a product manager is trying to explain their own document but can't understand what "they" wrote when they get to various bullet points. Or the document is correct, but their understanding of their own document is incorrect. Have you ever seen a student caught cheating? Someone who has the right answers on the paper, but can't explain why or remember why they wrote it down? It's exactly that.