37 ms·

> It's eye opening to see how much the ChatGPT produced content can fool managers at the top who catch glimpses of neatly composed and verbose documents. Goodh

> It's eye opening to see how much the ChatGPT produced content can fool managers at the top who catch glimpses of neatly composed and verbose documents. Goodhart's law in action. If "neatly composed and verbose documents" is the metric higher ups use to judge product managers, then documents will become neater and more verbose to the detriment of everything else. In the end, the alignment problems that plague AI safety research are all playing out in company structures every day. It's hard to give a metric that measures your intended outcome, and any proxy leads to unintended consequences once there's an easier way to reach that proxy.

13years 4y ago
In the end, the alignment problems that plague AI safety research are all playing out in company structures every day. It's hard to give a metric that measures your intended outcome, and any proxy leads to unintended consequences once there's an easier way to reach that proxy. I've actually pointed this out as a fallacy of the premise of current alignment theory. That being we simply need to apply humanities values for the AI to be properly aligned. However, every alignment problem can also be seen as an existing human problem for which we have never resolved.
- jacquesm 4y ago
  The scale is the key. Things that have impact at scale are harmless initially and then in the long run the damage becomes visible and by then we'll be as dependent on them as a heroin junk is on their fix. Oil, plastic, electricity, natural gas, mines, wood, poultry, meat & fish, transportation etc. They're nice and super useful and when used in moderation or when there aren't that many humans to begin with and you can easily absorb the consequences. But if you multiply the number of people while at the same time you increase the consumption/production on some or even all of these in tandem the effects are massively compounding.
  - 13years 4y ago
    Yes, but that is just one facet. Alignment theory affects everything regardless of scale. It is about whether or not you can reliably get the intended result given a particular goal or task. We want AI to reason like humans and have humanities values. According alignment theory. Yet humans will also cheat to get a result. Our values also are what put us in conflict with each other. FYI, I've elaborated in much more detail here on such issues with containment/alignment - https://dakara.substack.com/p/ai-singularity-the-hubris-trap https://dakara.substack.com/p/ai-singularity-the-hubris-trap
dTal 4y ago
>the alignment problems that plague AI safety research are all playing out in company structures every day. Yes yes yes holy crap yes. Corporations are AI, and we are already in the AI apocalypse. Corporations exhibit emergent behavior that any individual human involved with them is powerless to stop, and they self-optimize towards inhuman goals. Coca Cola inc is a paperclip maximizer, except it maximizes the consumption of flavored sugar water. We are completely failing as a society to adequately rein them in.
- jacquesm 4y ago
  There is this movie called Idiocracy. When I saw it the first time I thought it was hilarious. But over time I've stopped laughing and I now more or less realize that the makers may well have intended something funny but accidentally ended up making a prophesy. Or maybe they intended that from the get-go but they dressed it up as something funny for budget reasons. It's scary how many of the things that are in that movie as caricatures you can now find IRL.
  - deleted 4y ago
    [deleted]
  - saulpw 4y ago
    Mike Judge is a genius. He also did Office Space and Silicon Valley, among other things (I personally love King of the Hill). He has a sharp eye for both society and humor.
- RangerScience 4y ago
  Fav Dune quote (it's from book 4): > The problem isn't thinking machines, it's letting machines do the thinking for you ...and then I like to point out that many of our social structures (including corporations) are, essentially, machines. They're just machines running on meat instead of silicon.

13years 4y ago

In the end, the alignment problems that plague AI safety research are all playing out in company structures every day. It's hard to give a metric that measures your intended outcome, and any proxy leads to unintended consequences once there's an easier way to reach that proxy. I've actually pointed this out as a fallacy of the premise of current alignment theory. That being we simply need to apply humanities values for the AI to be properly aligned. However, every alignment problem can also be seen as an existing human problem for which we have never resolved.

jacquesm 4y ago
The scale is the key. Things that have impact at scale are harmless initially and then in the long run the damage becomes visible and by then we'll be as dependent on them as a heroin junk is on their fix. Oil, plastic, electricity, natural gas, mines, wood, poultry, meat & fish, transportation etc. They're nice and super useful and when used in moderation or when there aren't that many humans to begin with and you can easily absorb the consequences. But if you multiply the number of people while at the same time you increase the consumption/production on some or even all of these in tandem the effects are massively compounding.
- 13years 4y ago
  Yes, but that is just one facet. Alignment theory affects everything regardless of scale. It is about whether or not you can reliably get the intended result given a particular goal or task. We want AI to reason like humans and have humanities values. According alignment theory. Yet humans will also cheat to get a result. Our values also are what put us in conflict with each other. FYI, I've elaborated in much more detail here on such issues with containment/alignment - https://dakara.substack.com/p/ai-singularity-the-hubris-trap https://dakara.substack.com/p/ai-singularity-the-hubris-trap

dTal 4y ago

>the alignment problems that plague AI safety research are all playing out in company structures every day. Yes yes yes holy crap yes. Corporations are AI, and we are already in the AI apocalypse. Corporations exhibit emergent behavior that any individual human involved with them is powerless to stop, and they self-optimize towards inhuman goals. Coca Cola inc is a paperclip maximizer, except it maximizes the consumption of flavored sugar water. We are completely failing as a society to adequately rein them in.

jacquesm 4y ago
There is this movie called Idiocracy. When I saw it the first time I thought it was hilarious. But over time I've stopped laughing and I now more or less realize that the makers may well have intended something funny but accidentally ended up making a prophesy. Or maybe they intended that from the get-go but they dressed it up as something funny for budget reasons. It's scary how many of the things that are in that movie as caricatures you can now find IRL.
- deleted 4y ago
  [deleted]
- saulpw 4y ago
  Mike Judge is a genius. He also did Office Space and Silicon Valley, among other things (I personally love King of the Hill). He has a sharp eye for both society and humor.
RangerScience 4y ago
Fav Dune quote (it's from book 4): > The problem isn't thinking machines, it's letting machines do the thinking for you ...and then I like to point out that many of our social structures (including corporations) are, essentially, machines. They're just machines running on meat instead of silicon.