4 ms·

Sounds like an example of Goodhart's Law: "When a metric becomes a target, it ceases to be a good metric." Optimizing for something as abstruse as "engagement"

Sounds like an example of Goodhart's Law: "When a metric becomes a target, it ceases to be a good metric." Optimizing for something as abstruse as "engagement" seems like a great way to completely ignore everything that actually makes social networks compelling. https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law

dredmorbius 8y ago
I'd give Gresham some due as well.
- dredmorbius 8y ago
  Many possible examples, but Dwight MacDonald's "Mass Culture" (1953) is a good start. https://is.muni.cz/el/1421/jaro2008/ESB032/um/5136660/MacDonald_-_A_Theory_of_Mass_Culture.pdf https://is.muni.cz/el/1421/jaro2008/ESB032/um/5136660/MacDon...
mygo 8y ago
they’re just optimizing for the wrong metric. If the metric was “user satisfaction” they could target it day and night and nobody would be complaining.
- tomerico 8y ago
  Any metric can be abused if you try to optimize it at all costs. You can easily increase average user satisfaction by reducing the usage by less satisfied users, it doesn't mean you improved things.
- mehwoot 8y ago
  The reason goodharts law exists is because its hard to measure things like user satisfaction, so we're left with poor proxies like user engagement.
  - Uberphallus 8y ago
    Most metrics would imply prison is great: low bounce rate, average session length of a couple of years, very flat retention curve.
  - deleted 8y ago
    [deleted]
- sullyj3 8y ago
  The issue is that something like this difficult to operationalize. User Engagement conflated with user satisfaction, with poor results. Maybe you could force your users to fill out Likert scales all the time, I bet they'd love that.
  - mygo 8y ago
    —————————————————————— Are you satisfied? [yes] [no] ——————————————————————
- megablast 8y ago
  Does high user satisfaction make more money?
lloeki 8y ago
> Optimizing for something as abstruse as "engagement" seems like a great way to completely ignore everything that actually makes social networks compelling. Or anything else, for that matter. The most shocking part of Android Pie release[0] was the repetition of the "drive engagement" mantra (in the post as well as surrounding articles). Fundamentally, phones are tools; how good is any tool to me when it's purposely designed to make me use it more? A tool should make me more efficient, hence, for the same task, make me use it less! Everything about this is spiralling to artificially inflate dubious metrics such as eyeball time or number of interactions. [0]: https://android-developers.googleblog.com/2018/08/introducing-android-9-pie.html https://android-developers.googleblog.com/2018/08/introducin...
- xf00 8y ago
  > A tool should make me more efficient, hence, for the same task, make me use it less! It makes sense when you consider Android phones as a set of tools used by advertisers to reach consumers. The more eyeballs the more efficient the tool becomes.
  - Klover 8y ago
    It shows in both app stores on either platform. The play store gives me lots of suggestions on apps to download based on activity, and the iOS store has these write ups to promote apps. Both get them money. Does Android have something like Siri suggestions with P? Because that seems intended to do the exact opposite of what you’re saying. It is the feature that suggests your notes app if you often use your notes app in the grocery store location. It would be a good use for Android instead of the constant asking to review the store.
    - ryanbrunner 8y ago
      It does have a suggested apps section at the top of the apps list. It's supposedly time / location aware, but I haven't dug into how effective that is (I tend to use 3-4 apps almost exclusively on my phone, so that's all I tend to ever get suggested).
- 8y ago
kristiandupont 8y ago
When I hear startups declare that they are data-driven I get the image of a person driving a car by focusing stritcly on the dashboard.
- kasey_junk 8y ago
  There are situations where the only safe way to fly a plane is to ignore everything but the instruments... Though I suppose in this stretched analogy those are not great positions to be in.
  - crzwdjk 8y ago
    There are also situations where a sticky note can be a vital piece of safety equipment because an instrument is malfunctioning. And even though the pilot knows the instrument is showing bogus data, they're so used to looking at it that they will instinctively do so and react to its bogus indications, so the safest thing is to hide it from view entirely. Fortunately there's enough redundancy in the information from the basic instruments that even if one of them fails, you can still fly the plane by extrapolating the missing information from the other ones.
Pigo 8y ago
I'm forced to interact with certain people in my life through certain social media tools, but I lost interest in most if it a long time ago. I've tried to quantify what it is that made it so exciting at first, and the best thing I can think of is that I used to meet new people or engage with people I wanted to get to know better when things were first kicking off. I never wanted it to become the way I talk to my folks, I wanted to talk to that cute girl in my class or some guy in Russia that happened to like the Smashing Pumpkins too. Maybe I'm not using it right, but it seems more difficult to find new people while wading through ads and political posts from peripheral friends.
- Nickersf 8y ago
  I couldn't agree more with this. I'm in the same boat. 32 years old, so I had a social life before social media. There is one exception for me and that's Instagram. I'm a visual person and it satisfies that for me. Three minutes on Twitter is enough to make me depressed for the rest of the day. Facebook isn't much different... I think humans will naturally shift to actually hanging out in real life again.

dredmorbius 8y ago

I'd give Gresham some due as well.

dredmorbius 8y ago
Many possible examples, but Dwight MacDonald's "Mass Culture" (1953) is a good start. https://is.muni.cz/el/1421/jaro2008/ESB032/um/5136660/MacDonald_-_A_Theory_of_Mass_Culture.pdf https://is.muni.cz/el/1421/jaro2008/ESB032/um/5136660/MacDon...

mygo 8y ago

they’re just optimizing for the wrong metric. If the metric was “user satisfaction” they could target it day and night and nobody would be complaining.

tomerico 8y ago
Any metric can be abused if you try to optimize it at all costs. You can easily increase average user satisfaction by reducing the usage by less satisfied users, it doesn't mean you improved things.
mehwoot 8y ago
The reason goodharts law exists is because its hard to measure things like user satisfaction, so we're left with poor proxies like user engagement.
- Uberphallus 8y ago
  Most metrics would imply prison is great: low bounce rate, average session length of a couple of years, very flat retention curve.
- deleted 8y ago
  [deleted]
sullyj3 8y ago
The issue is that something like this difficult to operationalize. User Engagement conflated with user satisfaction, with poor results. Maybe you could force your users to fill out Likert scales all the time, I bet they'd love that.
- mygo 8y ago
  —————————————————————— Are you satisfied? [yes] [no] ——————————————————————
megablast 8y ago
Does high user satisfaction make more money?

lloeki 8y ago

> Optimizing for something as abstruse as "engagement" seems like a great way to completely ignore everything that actually makes social networks compelling. Or anything else, for that matter. The most shocking part of Android Pie release[0] was the repetition of the "drive engagement" mantra (in the post as well as surrounding articles). Fundamentally, phones are tools; how good is any tool to me when it's purposely designed to make me use it more? A tool should make me more efficient, hence, for the same task, make me use it less! Everything about this is spiralling to artificially inflate dubious metrics such as eyeball time or number of interactions. [0]: https://android-developers.googleblog.com/2018/08/introducing-android-9-pie.html https://android-developers.googleblog.com/2018/08/introducin...

xf00 8y ago
> A tool should make me more efficient, hence, for the same task, make me use it less! It makes sense when you consider Android phones as a set of tools used by advertisers to reach consumers. The more eyeballs the more efficient the tool becomes.
- Klover 8y ago
  It shows in both app stores on either platform. The play store gives me lots of suggestions on apps to download based on activity, and the iOS store has these write ups to promote apps. Both get them money. Does Android have something like Siri suggestions with P? Because that seems intended to do the exact opposite of what you’re saying. It is the feature that suggests your notes app if you often use your notes app in the grocery store location. It would be a good use for Android instead of the constant asking to review the store.
  - ryanbrunner 8y ago
    It does have a suggested apps section at the top of the apps list. It's supposedly time / location aware, but I haven't dug into how effective that is (I tend to use 3-4 apps almost exclusively on my phone, so that's all I tend to ever get suggested).
8y ago

kristiandupont 8y ago

When I hear startups declare that they are data-driven I get the image of a person driving a car by focusing stritcly on the dashboard.

kasey_junk 8y ago
There are situations where the only safe way to fly a plane is to ignore everything but the instruments... Though I suppose in this stretched analogy those are not great positions to be in.
- crzwdjk 8y ago
  There are also situations where a sticky note can be a vital piece of safety equipment because an instrument is malfunctioning. And even though the pilot knows the instrument is showing bogus data, they're so used to looking at it that they will instinctively do so and react to its bogus indications, so the safest thing is to hide it from view entirely. Fortunately there's enough redundancy in the information from the basic instruments that even if one of them fails, you can still fly the plane by extrapolating the missing information from the other ones.

Pigo 8y ago

I'm forced to interact with certain people in my life through certain social media tools, but I lost interest in most if it a long time ago. I've tried to quantify what it is that made it so exciting at first, and the best thing I can think of is that I used to meet new people or engage with people I wanted to get to know better when things were first kicking off. I never wanted it to become the way I talk to my folks, I wanted to talk to that cute girl in my class or some guy in Russia that happened to like the Smashing Pumpkins too. Maybe I'm not using it right, but it seems more difficult to find new people while wading through ads and political posts from peripheral friends.

Nickersf 8y ago
I couldn't agree more with this. I'm in the same boat. 32 years old, so I had a social life before social media. There is one exception for me and that's Instagram. I'm a visual person and it satisfies that for me. Three minutes on Twitter is enough to make me depressed for the rest of the day. Facebook isn't much different... I think humans will naturally shift to actually hanging out in real life again.