11 ms·

Stop measuring community engagement

296 points by rosiesherry 4y ago

DoreenMichele 4y ago
Even though it’s the energy they’re after, not the screams, it’s the screaming that was their primary metric, and everything about their company was organized around increasing that. Missed opportunity to bring it full circle back to this. Otherwise, not bad.
burlesona 4y ago
Good article, though I think it missed explicitly making the point that the reason "engagement" is the metric is because that's what platforms monetize. Engagement is a euphemism for attention, and social platforms exist to sell your to advertisers. Since the article is written for "community builders," understanding that engagement is a metric to quantify the advertising potential of a platform should make it obvious that people who aren't in the business of selling attention to advertisers don't need to copy social media and optimize for engagement; instead they should target better measurements of value creation for their community. Of course Goodhart's law[1] still applies. 1: https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law (edited for readability)
- cyborgx7 4y ago
  Exactly. It took me a while to understand that this article was written for community managers of brands, not people developing social networks. They keep saying engagement has no value, but to social networks and to "influencers" who both make their money off of ads, that is exactly what has value.
- phkahler 4y ago
  >> Good article, though I think it missed explicitly making the point that the reason "engagement" is the metric is because that's what platforms monetize. In the case of "community" around a product, the purpose of the community is usually to provide free support. That's not so much monetization as it is cost savings. For that to work, people need to be "engaged" or kept around so they are present to field those support issues.
  - sfink 4y ago
    Engagement is still the wrong metric for this purpose, because flamewars rate much higher than single comprehensive answers. In fact, partial or incorrect answer rate better, because the questioner will have to come back or someone will need to post corrections or clarifications. Engagement is a better proxy for value destruction, if you define people's time and sanity as valuable. (It's not perfect; a thriving community will rate highly.)
- throw10920 4y ago
  > the reason "engagement" is the metric is because that's what platforms monetize ...and the implication of that is that if we change the monetization model, then the metrics also change. For a company selling a paid product, metrics become things like the effectiveness of the sales funnel, how much support the average user requires, the rating of the product on app stores, etc - which are not ideal, but still far better than "engagement".
  - crazygringo 4y ago
    You might be surprised to discover that even for paid products, engagement is often still the #1 metric. Simply because people who are engaged with a product/app/whatever are by far the most likely to be repeat customers -- to renew their subscription, to buy the new version, to adopt the next product. Once engagement drops, future revenue dries up. A drop in engagement is the canary in the coal mine -- it shows where your business is going wrong months/years before it shows up in sales stats or renewal rates. So actually, the metric doesn't change at all. E.g. Netflix is obsessed with engagement even though it's subscription-based rather than ad-based. Because guess who winds up cancelling their subscriptions?
    - mhall119 4y ago
      s/surprised/disappointed/ A user's engagement with your product is a great metric, because what you value is users using your product. But for a COMMUNITY it's the wrong metric, because you don't value negative comments in your Slack no matter how many there are (in fact, the more there are the worse it is).
- hinkley 4y ago
  Engagement is just a more sophisticated version of 'eyeballs'. For community building you want successful events, you want interesting questions to be asked, and interesting answers to be offered. Someone could write a book on how to make communities work. Probably several books.
  - mhall119 4y ago
    There are several books on precisely this.
- Theodores 4y ago
  You are completely right on Goodhart's Law. I know this from experience. I put the customer first in the belief that 'the rain follows the plough' and, if you put the customer first, then all else follows. I see metrics as there to be cracked. If you are measuring the right things then you can see where the problems are, fix the problems and move on to measuring different metrics that are more specific and focused on the remaining problem areas. For me it is a campaign where initial reactive firefighting gives way to calm, proactive problem solving, fixing problems before they are of consequence. Along the way many unexpected things are learned about the customer. I do all this with a focus on the customer, meanwhile my colleagues in marketing are out for themselves. They are the dead weight keeping progress back as they have to do things like buy traffic and hold on to keyword optimised URLs that are cargo cult SEO. Their reports for their micromanagers are always measuring the same stuff, e.g. 'engagement', yet they could be engaging customers solely for the purpose of pissing them off for them to never come back. My favourite is the pop up shown as someone leaves a website begging them to stay, or the sign up to the newsletter popup. These could get one extra sale at the cost of pissing off a million people but if you only measure the former then it all seems good. In the capitalist world only one metric matters to the shareholders. Once I worked for a highly successful company that sent out sales newsletters with no measurement whatsoever of open rates or click throughs. None of that was needed. The problem was emptying the warehouse too quickly to have no stock left. Customers would be waiting for our email, not ignoring them. We had plenty of engagement with the customers as they actually bought stuff. Subsequently I have worked for agencies where they spend a fortune on some email service and they get these fancy graphs to show the clients of these open rates the world over. Which had wow value at the time and 'engagement' stats. But I had come from somewhere where we had no need to measure that stuff, we weren't looking at these charts, we were coping with a deluge of orders and had no time for that.
bluGill 4y ago
Unfortunately what is really needed here is a suggestion of something else to measure that is better. I fight this a lot with bad metrics (code test coverage), nobody wants to give them up unless I can suggest a replacement to measure.
- mhall119 4y ago
  Here's a couple: Connections between people: https://www.savannahhq.com/2022/09/06/measuring-your-community-connections/ https://www.savannahhq.com/2022/09/06/measuring-your-communi... Support: https://www.savannahhq.com/2022/09/08/measuring-your-community-support/ https://www.savannahhq.com/2022/09/08/measuring-your-communi...
jpster 4y ago
This sounds great, but I see a challenge. The author recommends measuring e.g. user happiness instead of engagement. How to measure user happiness? IMO, some type of survey could be used. But the “best” metrics are based on observable behavior. Not what users self-report, because they may not know how they actually feel about the experience. In general it’s better to see what users do rather than what they say.
- Bjartr 4y ago
  Observables are better, but self reported metrics do still have value. It's an aggregate version of the problem devs encounter when doing usability testing. Users will tell you what their experience is, but it's up to you as a software designer to distill that feedback into actual needs. Which may not at all be what they actually ask for. Survey responses are similar in that they don't tell you exactly what to do, but they do contain actionable information if you have the skill to interpret it.
- ItsMonkk 4y ago
  You must have an expert or experts. That expert will then make decisions, and use metrics and surveys and intuition to drive their new world view, but it's the expert that is always at the top of the stack. If you then take that model and try to use them in a new domain, that is never going to work out. The environment adapts to the incentives that the current system produces, and will instantly make your pre-created model worthless[0]. The only way to work out long-term is by having an expert stay ahead of the pack creating what they think are systems with better signal to the noise. [0]: https://dilbert.com/strip/1995-11-13 https://dilbert.com/strip/1995-11-13
  - mhall119 4y ago
    I'd recommend reading The Community Engagement Trap[0] by Rosie Sherry (which should also be shared here on HN), she talks about Community Discovery as the process for getting these deeper understandings. [0] https://rosie.land/posts/the-community-engagement-trap/ https://rosie.land/posts/the-community-engagement-trap/
fullshark 4y ago
I enjoyed the Monsters Inc reference and the turning of it on its head central to this piece. But we've said/argued this for years and at the end of the day engagement = ad reach = profits and until the economics there change any for-profit social media will gravitate towards engagement. Hell ANY media gravitates toward engagement and outrage bait. An actual community built around Mastodon seems possible, and there's community discussions around private group chats but they will always remain niche and not profitable. I don't get what https://savannahhq.com https://savannahhq.com is and what the value add is there but that seems to be what he's interested in, niche + small communities.
- snarf21 4y ago
  Well said! Outrage sells and it is easy to keep it stoked. Everyone is addicted to free. If we get rid of ads, how does (even the barest bones) Twitter keep the lights on? I wonder if there is a cross-over between Patreon, Substack and Twitter. This would probably work for niches but not writ large. But at the end of the day, we need to find a way to pay for what we value because that is how we get more of it. Interested in new ideas in this space....
  - ethbr0 4y ago
    The Medium/Spotify model -- per-user constant subscription fee, apportioned to content creators on the basis of their views + some time-viewing / read-to-end adjustments to discourage clickbaiting -- seems fairly decent, but really requires a global platform to work. But product companies don't like it because they can't scale revenue user-transparently by repackaging data (e.g. building new / better ad products), and raising prices for users sucks. The original sin is that the web was free, and the consequence of that, once post-content-scarcity in a free market system, has been continual sleight of hand to make money from "free." If everyone has gotten in the habit of paying for quality content... we'd have a very different world now. (And this said as a hard-charging "information wants to be free" type. But the end commercial result nowadays is terrible)
    - politician 4y ago
lucideer 4y ago
Nicely written post but can't help noticing the elephant in the room being tiptoed around, which really makes me question the sincerity of the author. The monsters need energy to power their civilisation. Why do social media companies measure engagement? What are they extracting? There is not one single mention of the "p word" in the article. The last two sections are delivered with such heavy doses of naivety that this ends up just coming across as Orwellian in tone. > “We should measure what we value, not value what we measure” Who is "we"? Is "we" the shareholders? What do "they" value do you think?
- deleted 4y ago
  [deleted]
alexb_ 4y ago
>Now imagine if this played out the other way around. Imagine that the monsters originally focused on maximizing energy by making children laugh as much as possible. Then one day, in a horrible twist, someone discovers that a child’s scream was a much easier and more powerful way of meeting their goals. Their company, which built a platform that gave them instant access to children around the world, suddenly realizes that they could reach their goals by spreading fear rather than joy. Then they would instantly start using screams? Nobody making business decisions in social media companies will prioritize "user happiness" over increased production of their main product (your attention) for their customers (advertisers).
- egypturnash 4y ago
  Congratulation, you grasped the intent of that analogy!
  - alexb_ 4y ago
    Maybe I'm reading the intent wrong, but no social media company will ever take this suggestion. This line in particular: >Let’s bring this back to community building. We still want our members to be happy, right? If it doesn't make them more money... no they don't!
    - mhall119 4y ago
      That's why it also says "Community isn’t social media." No social media company is going to do this, you are correct. But community managers and community-led companies absolutely should.
- catherd 4y ago
  Some will and some won't. In the absence of rules that penalize that type of behavior, usually the ones that will end up winning. It has nothing to do with morals, those just seem to be the rules of the game as we currently play it.
forbiddenvoid 4y ago
I like the premise of the post. I think it paints a rather overly idealistic view of social networks, who exist to make money first, not to create value (if they could make money without creating value, they certainly would do that).
efitz 4y ago
The article dances around a topic that is very relevant across technology companies- toxic metrics. I worked for many years at a company that prides itself on being metrics driven. To the point that they celebrated having many significant figures after the decimal point when measuring length of negative events in seconds. The good side of metrics driven culture is that having a metric gives you a concrete goal. The downside is that really good metrics are often very hard. By really good, I mean metrics that measure the desired outcome. It’s much easier to measure, for example, how many operation X your service performed, rather than the value the service delivered to your business or your customers. So we assume that each operation X contributes a uniform, positive quantum to those outcomes and we count X’s.
- hinkley 4y ago
  Toxic metrics are a subset of perverse incentives. Necessary, but not sufficient to stay out of trouble.
- godelski 4y ago
  I don't think they dance around the topic so much as outright state that it is bad. They mostly talk about engagement because it is the primary metric but even state that we should not value what we measure but measure what we value. Which it is harder to do that, but hey, you got billions of dollars to work with, it seems like the resources are there.
- JohnFen 4y ago
  The software industry has made me very aware that being metric-driven can be a terrible thing as easily as a good thing. Software has been getting substantially worse over time (in my opinion) in large part because of the use of metrics.
  - FalconSensei 4y ago
    Goodhart's law basically also. I've seen ppl on Twitter posting 'unpopular opinions' that they don't even agree, just to get replies. And then admit on the conversation that they don't agree with that.
slackerIII 4y ago
Early on in a community, there often aren't enough data points to cover other possible metrics. With that in mind, starting somewhere is better than not starting at all — measuring engagement is a useful way to begin understanding what's resonating in your community. As you continue building your community, there are many different angles that you should use to evaluate its health. Measuring engagement then becomes one piece of data that's valuable, but shouldn't be the only piece. When engagement data is combined with qualitative community surveys, enthusiasm from members to contribute to a community, clear and timely responsiveness to community needs, membership growth over time and geographies, depth of member interactions (whether across community channels or within specific channels), what's topically important to members and why, and overall sentiment and change in sentiment over time - that's when community builders can begin to better understand the health of their communities and their impact on their business. Engagement is an important piece, but just one of the pieces, that helps paint the full picture. Disclaimer: I’m a co-founder of Common Room [1] and a we’ve invested a lot of energy in solving for this exact problem. You’re welcome to check out the product (it’s free to sign up) and would love to hear of ideas or feedback. 1: https://www.commonroom.io/ https://www.commonroom.io/
nmilo 4y ago
I don't get it. What is a "community manager"? Is it someone that runs a company's social media page? And do they think that people care about their brand so much that they would join a community of people whose only common trait is liking the same brand? And in the first place, how do you even build a "community" manually and artificially? It's almost a bastardization of the word, as if a community is not a set of people with similar goals but instead yet another way to convert people's time and desire for social interaction into profit. The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of "meaningful relationships" created? But building an artificial community in the first place can never be altruistic because the end goal of it all is to guide people to your product or conference or whatever. If you were an actual altruistic community builder, you would be telling people to get off the Internet and go make "meaningful relationships" with people in real life.
- deleted 4y ago
  [deleted]
- mhall119 4y ago
  > I don't get it. What is a "community manager"? Is it someone that runs a company's social media page? No, that's a social media manager, completely different role. > And do they think that people care about their brand so much that they would join a community of people whose only common trait is liking the same brand? Yes! It happens all the time. Most of the biggest brands have a community of people who share that common trait. > And in the first place, how do you even build a "community" manually and artificially? Again yes, just like you build a garden. The gardener doesn't make the plants grow, but they do make sure they have the right environment and resources to grow in. > The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of "meaningful relationships" created? That's not altruism. Those meaningful relationships help the company/project the community was built around. They provide feedback, help improve processes, make connections outside the company/project, provide support to other users, there are too many things that communities do to list them all here.
  - 4y ago
wsb_mod2 4y ago
The profit motive is being blamed here for optimizing for engagement. But how do you assess progress when you remove all desire to monetize? I run r/WallStreetBets and "quality" is an extremely nebulous term. I look for things like "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research". But these are human assessed metrics. Some more easily measurable metrics might include, "length of submissions", or "number of outbound links excluding blacklisted domains". Or even "number of tickers or quality-correlated keywords mentioned". All these metrics have very clear downsides, and if generally well-known, become useless. Interestingly, a score too high can also result in something being unlikely to be authentic. Another challenge is your relationship with users. Surprisingly, moderators are not innately adversarial to users, they can also promote content through other channels (discord, twitter) or sticky threads for a viewership boost. So, even without a profit motive, what do you do?
- hammock 4y ago
  Hard to scale, but farming out content samples to a panel of your target market for them to rate on the attributes of "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research" might be an approach
- empathy_m 4y ago
  Historically, the immortals on MUDs used to optimize for thing like "fun" and "clout" - how much do you enjoy interacting with the players and being seen as the person in charge of their game? How much do you enjoy telling people you volunteer your time to work on it? If WSB members were doing bad stuff like, making mean jokes about the leadership of companies Ryan Cohen divests from in; or coordinating real-life harassment of family members of employees of the Depository Trust Company or something; you'd probably want to put a stop to that and focus them somewhere else. Because it wouldn't be fun. If Fox News interviewed you and you sounded silly and all the community members got mad at you, you'd probably also quit the volunteer job. But as long as you make choices that make the community a fun and engaging place to hang out and feel like you're part of a big secret treasure hunt, that seems awesome, and you would probably want to make choices that maximize that. TBH most MUDs were not data-driven and kinda govern by feeling -- they'd sort of add steering they thought was interesting and keep it in place unless there's a backlash. The tedious administrative stuff (bans, moderation, player requests for item reimbursement) would always have a backlog and you'd recruit junior imms to help out and they would feel like they were part of the fun too.
jamesknelson 4y ago
This is an issue with more than just the tech industry. It’s an issue with modern society itself. Take the GDP, the number that when it’s first derivative goes negative, we shout “recession” and use it as an excuse for all manner of harmful (and sometimes beneficial) behaviors. What is GDP? It’s not a measure of wealth; it’s a measure of production. Take the boots paradox, for example. A society making $50 boots that need replacing after a few months, will have a higher GDP than a society making $100 boots that last for years. And the shitty boots society, despite spending more on boots - to paraphrase Terry Pratchett - will still have soggy feat. It’s a hard problem, and I can’t see any obvious way to solve it. Both in social networks, and in economics. For whatever reason, consumption seems to be much easier to measure than wealth.
mikkergp 4y ago
I think people enjoy this argument because it's counter intuitive, but I also think it's counter intuitive because it's wrong. If you build a playground and everyone avoids the teeter-totter, there's probably something wrong with the teeter totter. Oddly the monsters inc example seems counter to his point since it seems like they were measuring something to specific and the proper solution would be to something more general (like engagement) instead. It seems to me like trying to start with something other than engagement would be premature optimization. Of course engagement is insufficient and of course if you want to know something else, measure that other thing, and of course you should interrogate your metrics, and maybe it's just this thing that bugs me about modern writing where you have to take this ridiculous extreme stance to get eyeballs but yeah.
JAA1337 4y ago
Couldn't agree more. I believe this falls into the 'Results Oriented' mindset of delivering value. This means figuring out your desired outcome ahead of time, then measure it over time. In addition, overtime, you may determine that your desired outcome has changed and you need to measure differently. Yes, iterating is not just for software. IMHO pure quantitative metrics are of little value for customer experience. I believe metrics should be a little squishy and subjective. The value in the subjectivity is that it spurns conversation and actual thought instead of simple counting.
felipelalli 4y ago
But... How? It's missing how to do it (measure value) in the article.
- mhall119 4y ago
  I've started writing those answers out as separate blog posts: Measuring Connections: https://www.savannahhq.com/2022/09/06/measuring-your-community-connections/ https://www.savannahhq.com/2022/09/06/measuring-your-communi... Measuring Support: https://www.savannahhq.com/2022/09/08/measuring-your-community-support/ https://www.savannahhq.com/2022/09/08/measuring-your-communi...
robust-cactus 4y ago
Lately - it seems we're all caught between 2 worlds: 1. If you over focus on impact the quality of your product gets torn to shreds. You've built great direction but do people really use/like your product? 2. You build for engagement, people use the thing, but what impact did it drive? This article is great, but it's unbalanced potentially. I think directionally, some average in the middle is likely the way - so why not both? And while we're add it, add in other indicators. Success metrics that balance all of that, plus just a vibe metric for good measure.
photochemsyn 4y ago
'Stop making decisions based on engagment measurement' might be a better approach, if what you care about is maintaining the quality of your online social group. Some social media sites have tons of high-quality original content submitted by people with expertise in their domains, and some are 90% fluff. If you want to attract people who'll make high-quality contributions, getting rid of the fluff is the most important goal.
swyx 4y ago
when community engagement is measured badly, its because community is being treated as a product like any app where MAU and time on site is being prioritized, yet companies dont dedicate even 1/100 of the resources to community “product” that they do their software product (perhaps because it is not viewed as a real job, as perceived in other comments here) that said, you do need a way to justify to others why you are a better community manager than the next, and whether you are doinng a good job. that part went unanswered here and the article would be better if it had a constructive replacement instead of just arguing to be unaccountable to the rest of the org.
ChrisMarshallNY 4y ago
That's a great post, and I plan to share it with others. Sadly, it's pretty difficult to measure for some of the "values" that my communities run on, but "engagement" isn't even on the map. I suspect that "engagement" does, indeed, provide real monetary value for advertising-based sites, as clicks == money. Since that isn't a factor in my communities, we don't worry about that. In fact, we try to minimize "engagement."
tamsaraas 4y ago
> Originally they were there to connect people and let them share in each other’s joy. No, they looking for a ways how to easily and faster to fuck. > They couldn’t measure joy, but they could measure engagement, and so they did. > That measurement became their goal, and they focused on maximizing it. Because this is social networks profit. Ads, and more people attached to some topic where can be shown an ads to these people. That's why and only that's why it is works like that. Nobody cares about principles. Only money. Power in money, a money is a power. > What they didn’t understand was that negative, divisive, hate-inspiring posts get more engagement than positive, supportive, kind posts do. For what do your country has intelligence and different defense structures? Because it is very easy manipulate by people and do wars. Extremely easy to organize that and involve a lot of people to mass destruction events. Extremely easy. And i do not know (the same as you too) who is behind some kind of negative behind. > . The algorithm didn’t care whether the posts brought joy or anger, it only cared about whether they brought engagement. Because this is about ads and money. More people will watch ads more money social network will get. Easy. > So in order to maximize engagement, the algorithm actively encourages and elevates posts that cause unhappiness among the platform’s users. You said that? Is a bots with fake votes, SEO thing is not about manipulating data, opinions? Its easy to make fake accounts and promote hate as a wide interested topic, while it's not. Only social network know that. > They join looking for support, or connections, to give back, or find meaningful relationships. No, they looking for themselves reflected in the community. > Measuring engagement doesn’t tell you if your community is happy, or healthy, or even accomplishing any of the things you were hoping it would accomplish. Depends on a case. Some measuring engagement can prevent negative business processes. > Knowing how much engagement you have doesn’t tell you how much value you’re getting For that you have another metrics. Depends on a case. Registration, profit, grow, etc. Final countable results. > Let’s bring this back to community building. We still want our members to be happy, right? Who the fuck am i to let people happy? Happiness is subjective thing. I have no clue, no ability to make someone happy. The same as you, the same as anybody else. This is very agile thing depends on many factors and the most important wish to be happy and objective situation. > So what should we value? Does your community member even read or know about "values" at all? Is it some kind of cult, or what? ---- I have a well known secret. To get maximum engagement need to say that the white color is black color. Water - is not a liquide, and planet earth is flat. And then your "engagement" will go above the sky. Just always need to split people, and say that one thing is not the thing what you used to think, it's another thing. totally vise versa. Oh... Darling, exactly this you have done by saying: "stop measuring engagement". And by this title invited to engagement a lot of people who forced to read all of your nonsense to one more time confirm that white is white, black is black.
pdntspa 4y ago
Hot take: stop building communities around everything. People are highly tribalist and banding together like that produces a lot of ugly outcomes. Communities form and likely won't go away, particularly the more people internalize them emotionally and integrate them into their identity. It is exhausting to see a community for literally everything, even the smallest products. Perhaps we should let products simply exist.
- BrianOnHN 4y ago
  The other day I was think, it's actually the platforms where most of these communities that benefit directly from their existence. And we know those platforms are tools for manipulation. How much of the desire for these communities might be manufactured in the first place?
  - pdntspa 4y ago
    That's the problem... so much of both the communities and desire itself are manufactured. When they grow organically... cool, for the most part. But the minute money becomes a focus things go to hell.
- neilk 4y ago
  You're raising an interesting point, but you're coming close to saying that communities are bad, and in my experience they don't have to be. Even ones based around a product. Near where I used to live in the Valley, there was a real-world club for enthusiasts of a particular weird antique car. Not toxic, just eccentric. They just liked getting together and drinking in a nearby pub and showing off how they'd restored this or that aspect of the car. I think our goals should be to replicate that kind of success. I think the other posters in this thread have it right, that the reason why we usually can't is because our metrics are all about selling audiences to advertisers.
yieldcrv 4y ago
It’s because investors aren’t discerning either They chose to pay higher prices for shares, divorced from actual revenue The companies, in-turn, found non revenue metrics to show quarter over quarter and thousands of startups copied the model Just don’t rely on the A/B test, be aware of it, but don’t rely on it. Put the human control back into the process. Users aren't staying on your platform 5 seconds longer after you hid the escape hatch because they love it, they are frustrated and lost! The A/B test doesn't tell you why the engagement is longer, only that it is. It takes an empathic human to say “okay, B got us that result but not for the reason we want”
tikkun 4y ago
- tough 4y ago
  damn yeah that is good, should build a bot we can summon by mentioning or something for the lazy: like @tldrbot
- torbTurret 4y ago
  You preface all of your posts with, “people seemed to like…” No. HN does not need the spam. Save that for Reddit. Ironically given the topic, it’s posters like you that ruin the quality of online communities and make engagement hard to measure.
dfabulich 4y ago
This article completely undermines itself by arguing in the conclusion ("Measure what you value") that we can and should measure "meaningful relationships" instead. > They’re not as easy to measure as engagement, sure, but they can be measured. I'm afraid that a citation is sorely needed here. There are no real measurements for "meaningful relationships." In fact, the only proxy metric we have for "meaningful relationships" is engagement! And, yes, engagement is not a very good proxy metric for meaningful relationships, but since there's no alternative, it's all we've got, so this article is pointless.
scanr 4y ago
I wonder if measuring engagement and optimising for engagement should be separated. I’m glad that there is lots of engagement on HN but I’m also glad that HN chooses not to implement a number of engagement optimisation features as well as downranking topics that result in high engagement flame wars.
mikejulietbravo 4y ago
engagement takes many forms. If you value people connecting with each other, measure how many messages they're sending or connections they're making. Saying stop measuring community engagement is a good clickbait headline, but it's absurd advice for community managers that even your own article doesn't agree with.
RicoElectrico 4y ago
So please tell me, what should I optimize for when moderating a local OpenStreetMap community. I personally feel it's crucial people internalize that's a project with a bottom-up organization. A "civil society", if you will. Alas, despite 200-250 daily active mappers in Poland [1] only a fraction ever posts on the forum, even if just to ask something. Meeting IRL is also futile except 2-3 largest cities. We have a forum, Facebook group and Discord. Whatever I'd come up with, I'd like it to be a self-healing organism that would continue to thrive even if some important members become inactive. And to feel it's not a Sisyphean task. [1] https://osmstats.neis-one.org/?item=countries&country=Poland https://osmstats.neis-one.org/?item=countries&country=Poland
- scoofy 4y ago
  I'm confused. I see OSM as a huge success. Firms like Mapbox and Strava have an incentive to update that wiki with you. The maps are amazing. Any local sub-community will be challenging to organize, but maybe the metric should be making it fun for you, and if you're having a good time others will join. I am also starting a wiki, a golf course wiki: https://golfcourse.wiki https://golfcourse.wiki It's still very early for me, but I see myself in this post. It's going to be extremely slow going, as the main consumers of the content created have no desire to contribute, but since golf courses are mostly static, as long as i plug away, adding a little bit all the time, the site will be a success. It costs almost nothing to operate. Generating traffic is challenging, and i've been trying to find more people willing to contribute, but again, I see my product as best placed if i do exactly the opposite of what most of the golf course info sites are doing. Don't base my success on engagement, no aggregation, no overly-prodding to contribute. Just one day at a time, with the assumption that the task is endless anyway, having people share their passion on their own time. If i can make that profitable at all, and it shouldn't be too difficult as the site grows, i'll see it as a total win.
MoscMob 4y ago
Sophistifunk 4y ago
Twitter isn't a community, it's a platform. There's nothing at all wrong with community engangement. Platform engagement and the advertising model that requires it are the problems.
swayvil 4y ago
Maximizing community engagement, trolling and advertising are related. They're different heads of the same multiheaded demon. All 3 are concerned with attention. Are hungry for it. If social media could be optimized to deliver engaging conversation to everybody then the demon would be slain. Because the hunger would be satisfied. Does that sound right?
- actionfromafar 4y ago
  Engaging conversation is not as intense as raging. The hunger can never be satisfied.
- sfink 4y ago
  Seems reasonable to me, if you reduce it to putting the demon to sleep. The demon is always there. A forum filled with engaging, productive conversation can and will slide back to toxic crap if there isn't something keeping it in check. It may be "optimized" for engaging conversation, but what does that mean in practice? If it's fully or mostly automated, then it's relying on metrics, and humans will always always always find a way to maximize the score :: effort ratio of any metric. Low effort is incompatible with engaging productive conversation. If it's done fully or mostly manually, then that means actively keeping that demon snoozing.
civilized 4y ago
Here's the little discussion I have in my head when this issue comes up. "Engagement isn't a good metric. Optimize for having a good community." "But engagement is easy to measure AND it means people are using the product. If people are choosing to use the product, doesn't that mean the product is giving them value?" "Not necessarily. Drugs like tobacco and heroin have fantastic 'engagement' but we consider them bad for us." "But how do you tell the difference between something that engages because it's good and something that engages because it's some kind of drug?" "I guess one important signal would be when people know something is bad for them but they're doing it anyway?" "Is that the case for social media? Do people feel addicted - feel that it's bad for them but they can't stop?" I think we've heard a lot of anecdata on this last question, but I can't recall seeing any big survey on it. EDIT: according to this, people will pay to have their social media restricted: https://www.washingtonpost.com/outlook/2021/07/19/social-media-addiction-social-science/ https://www.washingtonpost.com/outlook/2021/07/19/social-med...
bawolff 4y ago
Is engagement really value neutral? In the extreme, if you spend all your time "engaging", something toxic is happening
swayvil 4y ago
Advertising is a kind of trolling. I think that's an important connection to ponder. It could lead to something.
nitwit005 4y ago
If you can find a way to easily quantify the quality of human social interactions, you're probably on the short list for a Nobel prize. It'd have applications in every field related to humans.
- joelthelion 4y ago
  I don't think that's completely unachievable with the current state of NLP tools. You could (roughly) measure the diversity of discussion topics, the frequency of positive and negative emotions, etc. It'd be work to do it well, but I think it could work decently.
  - nitwit005 4y ago
    You're suggesting that you can find things you assume correlate with what people want out of a community, but several people here commenting they can't define what they want. There's low odds of it somehow matching what they want, but struggle to express.
Sohcahtoa82 4y ago
As an end user, fighting back against using anger to drive engagement is easy. Stop engaging with content that angers you. Don't follow politicians and news outlets you despise. If someone you follow retweets the latest garbage being spewed by Fox News, don't interact with it. No retweets, no replies, no likes, no matter how well they refute the bullshit or how well they "own" the original messenger. The algorithms feed you content they think you'll engage with. Train the algorithm to feed you better content.

296 points by rosiesherry 4y ago

Even though it’s the energy they’re after, not the screams, it’s the screaming that was their primary metric, and everything about their company was organized around increasing that. Missed opportunity to bring it full circle back to this. Otherwise, not bad.

burlesona 4y ago

Good article, though I think it missed explicitly making the point that the reason "engagement" is the metric is because that's what platforms monetize. Engagement is a euphemism for attention, and social platforms exist to sell your to advertisers. Since the article is written for "community builders," understanding that engagement is a metric to quantify the advertising potential of a platform should make it obvious that people who aren't in the business of selling attention to advertisers don't need to copy social media and optimize for engagement; instead they should target better measurements of value creation for their community. Of course Goodhart's law[1] still applies. 1: https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law (edited for readability)

cyborgx7 4y ago
Exactly. It took me a while to understand that this article was written for community managers of brands, not people developing social networks. They keep saying engagement has no value, but to social networks and to "influencers" who both make their money off of ads, that is exactly what has value.
phkahler 4y ago
>> Good article, though I think it missed explicitly making the point that the reason "engagement" is the metric is because that's what platforms monetize. In the case of "community" around a product, the purpose of the community is usually to provide free support. That's not so much monetization as it is cost savings. For that to work, people need to be "engaged" or kept around so they are present to field those support issues.
- sfink 4y ago
  Engagement is still the wrong metric for this purpose, because flamewars rate much higher than single comprehensive answers. In fact, partial or incorrect answer rate better, because the questioner will have to come back or someone will need to post corrections or clarifications. Engagement is a better proxy for value destruction, if you define people's time and sanity as valuable. (It's not perfect; a thriving community will rate highly.)
throw10920 4y ago
> the reason "engagement" is the metric is because that's what platforms monetize ...and the implication of that is that if we change the monetization model, then the metrics also change. For a company selling a paid product, metrics become things like the effectiveness of the sales funnel, how much support the average user requires, the rating of the product on app stores, etc - which are not ideal, but still far better than "engagement".
- crazygringo 4y ago
  You might be surprised to discover that even for paid products, engagement is often still the #1 metric. Simply because people who are engaged with a product/app/whatever are by far the most likely to be repeat customers -- to renew their subscription, to buy the new version, to adopt the next product. Once engagement drops, future revenue dries up. A drop in engagement is the canary in the coal mine -- it shows where your business is going wrong months/years before it shows up in sales stats or renewal rates. So actually, the metric doesn't change at all. E.g. Netflix is obsessed with engagement even though it's subscription-based rather than ad-based. Because guess who winds up cancelling their subscriptions?
  - mhall119 4y ago
    s/surprised/disappointed/ A user's engagement with your product is a great metric, because what you value is users using your product. But for a COMMUNITY it's the wrong metric, because you don't value negative comments in your Slack no matter how many there are (in fact, the more there are the worse it is).
hinkley 4y ago
Engagement is just a more sophisticated version of 'eyeballs'. For community building you want successful events, you want interesting questions to be asked, and interesting answers to be offered. Someone could write a book on how to make communities work. Probably several books.
- mhall119 4y ago
  There are several books on precisely this.
Theodores 4y ago
You are completely right on Goodhart's Law. I know this from experience. I put the customer first in the belief that 'the rain follows the plough' and, if you put the customer first, then all else follows. I see metrics as there to be cracked. If you are measuring the right things then you can see where the problems are, fix the problems and move on to measuring different metrics that are more specific and focused on the remaining problem areas. For me it is a campaign where initial reactive firefighting gives way to calm, proactive problem solving, fixing problems before they are of consequence. Along the way many unexpected things are learned about the customer. I do all this with a focus on the customer, meanwhile my colleagues in marketing are out for themselves. They are the dead weight keeping progress back as they have to do things like buy traffic and hold on to keyword optimised URLs that are cargo cult SEO. Their reports for their micromanagers are always measuring the same stuff, e.g. 'engagement', yet they could be engaging customers solely for the purpose of pissing them off for them to never come back. My favourite is the pop up shown as someone leaves a website begging them to stay, or the sign up to the newsletter popup. These could get one extra sale at the cost of pissing off a million people but if you only measure the former then it all seems good. In the capitalist world only one metric matters to the shareholders. Once I worked for a highly successful company that sent out sales newsletters with no measurement whatsoever of open rates or click throughs. None of that was needed. The problem was emptying the warehouse too quickly to have no stock left. Customers would be waiting for our email, not ignoring them. We had plenty of engagement with the customers as they actually bought stuff. Subsequently I have worked for agencies where they spend a fortune on some email service and they get these fancy graphs to show the clients of these open rates the world over. Which had wow value at the time and 'engagement' stats. But I had come from somewhere where we had no need to measure that stuff, we weren't looking at these charts, we were coping with a deluge of orders and had no time for that.

bluGill 4y ago

Unfortunately what is really needed here is a suggestion of something else to measure that is better. I fight this a lot with bad metrics (code test coverage), nobody wants to give them up unless I can suggest a replacement to measure.

mhall119 4y ago
Here's a couple: Connections between people: https://www.savannahhq.com/2022/09/06/measuring-your-community-connections/ https://www.savannahhq.com/2022/09/06/measuring-your-communi... Support: https://www.savannahhq.com/2022/09/08/measuring-your-community-support/ https://www.savannahhq.com/2022/09/08/measuring-your-communi...

jpster 4y ago

This sounds great, but I see a challenge. The author recommends measuring e.g. user happiness instead of engagement. How to measure user happiness? IMO, some type of survey could be used. But the “best” metrics are based on observable behavior. Not what users self-report, because they may not know how they actually feel about the experience. In general it’s better to see what users do rather than what they say.

Bjartr 4y ago
Observables are better, but self reported metrics do still have value. It's an aggregate version of the problem devs encounter when doing usability testing. Users will tell you what their experience is, but it's up to you as a software designer to distill that feedback into actual needs. Which may not at all be what they actually ask for. Survey responses are similar in that they don't tell you exactly what to do, but they do contain actionable information if you have the skill to interpret it.
ItsMonkk 4y ago
You must have an expert or experts. That expert will then make decisions, and use metrics and surveys and intuition to drive their new world view, but it's the expert that is always at the top of the stack. If you then take that model and try to use them in a new domain, that is never going to work out. The environment adapts to the incentives that the current system produces, and will instantly make your pre-created model worthless[0]. The only way to work out long-term is by having an expert stay ahead of the pack creating what they think are systems with better signal to the noise. [0]: https://dilbert.com/strip/1995-11-13 https://dilbert.com/strip/1995-11-13
- mhall119 4y ago
  I'd recommend reading The Community Engagement Trap[0] by Rosie Sherry (which should also be shared here on HN), she talks about Community Discovery as the process for getting these deeper understandings. [0] https://rosie.land/posts/the-community-engagement-trap/ https://rosie.land/posts/the-community-engagement-trap/

fullshark 4y ago

I enjoyed the Monsters Inc reference and the turning of it on its head central to this piece. But we've said/argued this for years and at the end of the day engagement = ad reach = profits and until the economics there change any for-profit social media will gravitate towards engagement. Hell ANY media gravitates toward engagement and outrage bait. An actual community built around Mastodon seems possible, and there's community discussions around private group chats but they will always remain niche and not profitable. I don't get what https://savannahhq.com https://savannahhq.com is and what the value add is there but that seems to be what he's interested in, niche + small communities.

snarf21 4y ago
Well said! Outrage sells and it is easy to keep it stoked. Everyone is addicted to free. If we get rid of ads, how does (even the barest bones) Twitter keep the lights on? I wonder if there is a cross-over between Patreon, Substack and Twitter. This would probably work for niches but not writ large. But at the end of the day, we need to find a way to pay for what we value because that is how we get more of it. Interested in new ideas in this space....
- ethbr0 4y ago
  The Medium/Spotify model -- per-user constant subscription fee, apportioned to content creators on the basis of their views + some time-viewing / read-to-end adjustments to discourage clickbaiting -- seems fairly decent, but really requires a global platform to work. But product companies don't like it because they can't scale revenue user-transparently by repackaging data (e.g. building new / better ad products), and raising prices for users sucks. The original sin is that the web was free, and the consequence of that, once post-content-scarcity in a free market system, has been continual sleight of hand to make money from "free." If everyone has gotten in the habit of paying for quality content... we'd have a very different world now. (And this said as a hard-charging "information wants to be free" type. But the end commercial result nowadays is terrible)
  - politician 4y ago

lucideer 4y ago

Nicely written post but can't help noticing the elephant in the room being tiptoed around, which really makes me question the sincerity of the author. The monsters need energy to power their civilisation. Why do social media companies measure engagement? What are they extracting? There is not one single mention of the "p word" in the article. The last two sections are delivered with such heavy doses of naivety that this ends up just coming across as Orwellian in tone. > “We should measure what we value, not value what we measure” Who is "we"? Is "we" the shareholders? What do "they" value do you think?

deleted 4y ago
[deleted]

alexb_ 4y ago

>Now imagine if this played out the other way around. Imagine that the monsters originally focused on maximizing energy by making children laugh as much as possible. Then one day, in a horrible twist, someone discovers that a child’s scream was a much easier and more powerful way of meeting their goals. Their company, which built a platform that gave them instant access to children around the world, suddenly realizes that they could reach their goals by spreading fear rather than joy. Then they would instantly start using screams? Nobody making business decisions in social media companies will prioritize "user happiness" over increased production of their main product (your attention) for their customers (advertisers).

egypturnash 4y ago
Congratulation, you grasped the intent of that analogy!
- alexb_ 4y ago
  Maybe I'm reading the intent wrong, but no social media company will ever take this suggestion. This line in particular: >Let’s bring this back to community building. We still want our members to be happy, right? If it doesn't make them more money... no they don't!
  - mhall119 4y ago
    That's why it also says "Community isn’t social media." No social media company is going to do this, you are correct. But community managers and community-led companies absolutely should.
catherd 4y ago
Some will and some won't. In the absence of rules that penalize that type of behavior, usually the ones that will end up winning. It has nothing to do with morals, those just seem to be the rules of the game as we currently play it.

forbiddenvoid 4y ago

I like the premise of the post. I think it paints a rather overly idealistic view of social networks, who exist to make money first, not to create value (if they could make money without creating value, they certainly would do that).

efitz 4y ago

The article dances around a topic that is very relevant across technology companies- toxic metrics. I worked for many years at a company that prides itself on being metrics driven. To the point that they celebrated having many significant figures after the decimal point when measuring length of negative events in seconds. The good side of metrics driven culture is that having a metric gives you a concrete goal. The downside is that really good metrics are often very hard. By really good, I mean metrics that measure the desired outcome. It’s much easier to measure, for example, how many operation X your service performed, rather than the value the service delivered to your business or your customers. So we assume that each operation X contributes a uniform, positive quantum to those outcomes and we count X’s.

hinkley 4y ago
Toxic metrics are a subset of perverse incentives. Necessary, but not sufficient to stay out of trouble.
godelski 4y ago
I don't think they dance around the topic so much as outright state that it is bad. They mostly talk about engagement because it is the primary metric but even state that we should not value what we measure but measure what we value. Which it is harder to do that, but hey, you got billions of dollars to work with, it seems like the resources are there.
JohnFen 4y ago
The software industry has made me very aware that being metric-driven can be a terrible thing as easily as a good thing. Software has been getting substantially worse over time (in my opinion) in large part because of the use of metrics.
- FalconSensei 4y ago
  Goodhart's law basically also. I've seen ppl on Twitter posting 'unpopular opinions' that they don't even agree, just to get replies. And then admit on the conversation that they don't agree with that.

slackerIII 4y ago

Early on in a community, there often aren't enough data points to cover other possible metrics. With that in mind, starting somewhere is better than not starting at all — measuring engagement is a useful way to begin understanding what's resonating in your community. As you continue building your community, there are many different angles that you should use to evaluate its health. Measuring engagement then becomes one piece of data that's valuable, but shouldn't be the only piece. When engagement data is combined with qualitative community surveys, enthusiasm from members to contribute to a community, clear and timely responsiveness to community needs, membership growth over time and geographies, depth of member interactions (whether across community channels or within specific channels), what's topically important to members and why, and overall sentiment and change in sentiment over time - that's when community builders can begin to better understand the health of their communities and their impact on their business. Engagement is an important piece, but just one of the pieces, that helps paint the full picture. Disclaimer: I’m a co-founder of Common Room [1] and a we’ve invested a lot of energy in solving for this exact problem. You’re welcome to check out the product (it’s free to sign up) and would love to hear of ideas or feedback. 1: https://www.commonroom.io/ https://www.commonroom.io/

nmilo 4y ago

I don't get it. What is a "community manager"? Is it someone that runs a company's social media page? And do they think that people care about their brand so much that they would join a community of people whose only common trait is liking the same brand? And in the first place, how do you even build a "community" manually and artificially? It's almost a bastardization of the word, as if a community is not a set of people with similar goals but instead yet another way to convert people's time and desire for social interaction into profit. The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of "meaningful relationships" created? But building an artificial community in the first place can never be altruistic because the end goal of it all is to guide people to your product or conference or whatever. If you were an actual altruistic community builder, you would be telling people to get off the Internet and go make "meaningful relationships" with people in real life.

deleted 4y ago
[deleted]
mhall119 4y ago
> I don't get it. What is a "community manager"? Is it someone that runs a company's social media page? No, that's a social media manager, completely different role. > And do they think that people care about their brand so much that they would join a community of people whose only common trait is liking the same brand? Yes! It happens all the time. Most of the biggest brands have a community of people who share that common trait. > And in the first place, how do you even build a "community" manually and artificially? Again yes, just like you build a garden. The gardener doesn't make the plants grow, but they do make sure they have the right environment and resources to grow in. > The article implies that community building should be some kind of altruistic purpose, where your only goal is to maximize the amount of "meaningful relationships" created? That's not altruism. Those meaningful relationships help the company/project the community was built around. They provide feedback, help improve processes, make connections outside the company/project, provide support to other users, there are too many things that communities do to list them all here.
- 4y ago

wsb_mod2 4y ago

The profit motive is being blamed here for optimizing for engagement. But how do you assess progress when you remove all desire to monetize? I run r/WallStreetBets and "quality" is an extremely nebulous term. I look for things like "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research". But these are human assessed metrics. Some more easily measurable metrics might include, "length of submissions", or "number of outbound links excluding blacklisted domains". Or even "number of tickers or quality-correlated keywords mentioned". All these metrics have very clear downsides, and if generally well-known, become useless. Interestingly, a score too high can also result in something being unlikely to be authentic. Another challenge is your relationship with users. Surprisingly, moderators are not innately adversarial to users, they can also promote content through other channels (discord, twitter) or sticky threads for a viewership boost. So, even without a profit motive, what do you do?

hammock 4y ago
Hard to scale, but farming out content samples to a panel of your target market for them to rate on the attributes of "novelty", "thought-provoking / well reasoned commentary", "original content", "authenticity", "self-awareness" or "primary research" might be an approach
empathy_m 4y ago
Historically, the immortals on MUDs used to optimize for thing like "fun" and "clout" - how much do you enjoy interacting with the players and being seen as the person in charge of their game? How much do you enjoy telling people you volunteer your time to work on it? If WSB members were doing bad stuff like, making mean jokes about the leadership of companies Ryan Cohen divests from in; or coordinating real-life harassment of family members of employees of the Depository Trust Company or something; you'd probably want to put a stop to that and focus them somewhere else. Because it wouldn't be fun. If Fox News interviewed you and you sounded silly and all the community members got mad at you, you'd probably also quit the volunteer job. But as long as you make choices that make the community a fun and engaging place to hang out and feel like you're part of a big secret treasure hunt, that seems awesome, and you would probably want to make choices that maximize that. TBH most MUDs were not data-driven and kinda govern by feeling -- they'd sort of add steering they thought was interesting and keep it in place unless there's a backlash. The tedious administrative stuff (bans, moderation, player requests for item reimbursement) would always have a backlog and you'd recruit junior imms to help out and they would feel like they were part of the fun too.

jamesknelson 4y ago

This is an issue with more than just the tech industry. It’s an issue with modern society itself. Take the GDP, the number that when it’s first derivative goes negative, we shout “recession” and use it as an excuse for all manner of harmful (and sometimes beneficial) behaviors. What is GDP? It’s not a measure of wealth; it’s a measure of production. Take the boots paradox, for example. A society making $50 boots that need replacing after a few months, will have a higher GDP than a society making $100 boots that last for years. And the shitty boots society, despite spending more on boots - to paraphrase Terry Pratchett - will still have soggy feat. It’s a hard problem, and I can’t see any obvious way to solve it. Both in social networks, and in economics. For whatever reason, consumption seems to be much easier to measure than wealth.

mikkergp 4y ago

I think people enjoy this argument because it's counter intuitive, but I also think it's counter intuitive because it's wrong. If you build a playground and everyone avoids the teeter-totter, there's probably something wrong with the teeter totter. Oddly the monsters inc example seems counter to his point since it seems like they were measuring something to specific and the proper solution would be to something more general (like engagement) instead. It seems to me like trying to start with something other than engagement would be premature optimization. Of course engagement is insufficient and of course if you want to know something else, measure that other thing, and of course you should interrogate your metrics, and maybe it's just this thing that bugs me about modern writing where you have to take this ridiculous extreme stance to get eyeballs but yeah.

JAA1337 4y ago

Couldn't agree more. I believe this falls into the 'Results Oriented' mindset of delivering value. This means figuring out your desired outcome ahead of time, then measure it over time. In addition, overtime, you may determine that your desired outcome has changed and you need to measure differently. Yes, iterating is not just for software. IMHO pure quantitative metrics are of little value for customer experience. I believe metrics should be a little squishy and subjective. The value in the subjectivity is that it spurns conversation and actual thought instead of simple counting.

felipelalli 4y ago

But... How? It's missing how to do it (measure value) in the article.

mhall119 4y ago
I've started writing those answers out as separate blog posts: Measuring Connections: https://www.savannahhq.com/2022/09/06/measuring-your-community-connections/ https://www.savannahhq.com/2022/09/06/measuring-your-communi... Measuring Support: https://www.savannahhq.com/2022/09/08/measuring-your-community-support/ https://www.savannahhq.com/2022/09/08/measuring-your-communi...

robust-cactus 4y ago

Lately - it seems we're all caught between 2 worlds: 1. If you over focus on impact the quality of your product gets torn to shreds. You've built great direction but do people really use/like your product? 2. You build for engagement, people use the thing, but what impact did it drive? This article is great, but it's unbalanced potentially. I think directionally, some average in the middle is likely the way - so why not both? And while we're add it, add in other indicators. Success metrics that balance all of that, plus just a vibe metric for good measure.

photochemsyn 4y ago

'Stop making decisions based on engagment measurement' might be a better approach, if what you care about is maintaining the quality of your online social group. Some social media sites have tons of high-quality original content submitted by people with expertise in their domains, and some are 90% fluff. If you want to attract people who'll make high-quality contributions, getting rid of the fluff is the most important goal.

swyx 4y ago

when community engagement is measured badly, its because community is being treated as a product like any app where MAU and time on site is being prioritized, yet companies dont dedicate even 1/100 of the resources to community “product” that they do their software product (perhaps because it is not viewed as a real job, as perceived in other comments here) that said, you do need a way to justify to others why you are a better community manager than the next, and whether you are doinng a good job. that part went unanswered here and the article would be better if it had a constructive replacement instead of just arguing to be unaccountable to the rest of the org.

ChrisMarshallNY 4y ago

That's a great post, and I plan to share it with others. Sadly, it's pretty difficult to measure for some of the "values" that my communities run on, but "engagement" isn't even on the map. I suspect that "engagement" does, indeed, provide real monetary value for advertising-based sites, as clicks == money. Since that isn't a factor in my communities, we don't worry about that. In fact, we try to minimize "engagement."

tamsaraas 4y ago

> Originally they were there to connect people and let them share in each other’s joy. No, they looking for a ways how to easily and faster to fuck. > They couldn’t measure joy, but they could measure engagement, and so they did. > That measurement became their goal, and they focused on maximizing it. Because this is social networks profit. Ads, and more people attached to some topic where can be shown an ads to these people. That's why and only that's why it is works like that. Nobody cares about principles. Only money. Power in money, a money is a power. > What they didn’t understand was that negative, divisive, hate-inspiring posts get more engagement than positive, supportive, kind posts do. For what do your country has intelligence and different defense structures? Because it is very easy manipulate by people and do wars. Extremely easy to organize that and involve a lot of people to mass destruction events. Extremely easy. And i do not know (the same as you too) who is behind some kind of negative behind. > . The algorithm didn’t care whether the posts brought joy or anger, it only cared about whether they brought engagement. Because this is about ads and money. More people will watch ads more money social network will get. Easy. > So in order to maximize engagement, the algorithm actively encourages and elevates posts that cause unhappiness among the platform’s users. You said that? Is a bots with fake votes, SEO thing is not about manipulating data, opinions? Its easy to make fake accounts and promote hate as a wide interested topic, while it's not. Only social network know that. > They join looking for support, or connections, to give back, or find meaningful relationships. No, they looking for themselves reflected in the community. > Measuring engagement doesn’t tell you if your community is happy, or healthy, or even accomplishing any of the things you were hoping it would accomplish. Depends on a case. Some measuring engagement can prevent negative business processes. > Knowing how much engagement you have doesn’t tell you how much value you’re getting For that you have another metrics. Depends on a case. Registration, profit, grow, etc. Final countable results. > Let’s bring this back to community building. We still want our members to be happy, right? Who the fuck am i to let people happy? Happiness is subjective thing. I have no clue, no ability to make someone happy. The same as you, the same as anybody else. This is very agile thing depends on many factors and the most important wish to be happy and objective situation. > So what should we value? Does your community member even read or know about "values" at all? Is it some kind of cult, or what? ---- I have a well known secret. To get maximum engagement need to say that the white color is black color. Water - is not a liquide, and planet earth is flat. And then your "engagement" will go above the sky. Just always need to split people, and say that one thing is not the thing what you used to think, it's another thing. totally vise versa. Oh... Darling, exactly this you have done by saying: "stop measuring engagement". And by this title invited to engagement a lot of people who forced to read all of your nonsense to one more time confirm that white is white, black is black.

pdntspa 4y ago

Hot take: stop building communities around everything. People are highly tribalist and banding together like that produces a lot of ugly outcomes. Communities form and likely won't go away, particularly the more people internalize them emotionally and integrate them into their identity. It is exhausting to see a community for literally everything, even the smallest products. Perhaps we should let products simply exist.

BrianOnHN 4y ago
The other day I was think, it's actually the platforms where most of these communities that benefit directly from their existence. And we know those platforms are tools for manipulation. How much of the desire for these communities might be manufactured in the first place?
- pdntspa 4y ago
  That's the problem... so much of both the communities and desire itself are manufactured. When they grow organically... cool, for the most part. But the minute money becomes a focus things go to hell.
neilk 4y ago
You're raising an interesting point, but you're coming close to saying that communities are bad, and in my experience they don't have to be. Even ones based around a product. Near where I used to live in the Valley, there was a real-world club for enthusiasts of a particular weird antique car. Not toxic, just eccentric. They just liked getting together and drinking in a nearby pub and showing off how they'd restored this or that aspect of the car. I think our goals should be to replicate that kind of success. I think the other posters in this thread have it right, that the reason why we usually can't is because our metrics are all about selling audiences to advertisers.

yieldcrv 4y ago

It’s because investors aren’t discerning either They chose to pay higher prices for shares, divorced from actual revenue The companies, in-turn, found non revenue metrics to show quarter over quarter and thousands of startups copied the model Just don’t rely on the A/B test, be aware of it, but don’t rely on it. Put the human control back into the process. Users aren't staying on your platform 5 seconds longer after you hid the escape hatch because they love it, they are frustrated and lost! The A/B test doesn't tell you why the engagement is longer, only that it is. It takes an empathic human to say “okay, B got us that result but not for the reason we want”

tikkun 4y ago

tough 4y ago
damn yeah that is good, should build a bot we can summon by mentioning or something for the lazy: like @tldrbot
torbTurret 4y ago
You preface all of your posts with, “people seemed to like…” No. HN does not need the spam. Save that for Reddit. Ironically given the topic, it’s posters like you that ruin the quality of online communities and make engagement hard to measure.

dfabulich 4y ago

This article completely undermines itself by arguing in the conclusion ("Measure what you value") that we can and should measure "meaningful relationships" instead. > They’re not as easy to measure as engagement, sure, but they can be measured. I'm afraid that a citation is sorely needed here. There are no real measurements for "meaningful relationships." In fact, the only proxy metric we have for "meaningful relationships" is engagement! And, yes, engagement is not a very good proxy metric for meaningful relationships, but since there's no alternative, it's all we've got, so this article is pointless.

scanr 4y ago

I wonder if measuring engagement and optimising for engagement should be separated. I’m glad that there is lots of engagement on HN but I’m also glad that HN chooses not to implement a number of engagement optimisation features as well as downranking topics that result in high engagement flame wars.

mikejulietbravo 4y ago

engagement takes many forms. If you value people connecting with each other, measure how many messages they're sending or connections they're making. Saying stop measuring community engagement is a good clickbait headline, but it's absurd advice for community managers that even your own article doesn't agree with.

RicoElectrico 4y ago

So please tell me, what should I optimize for when moderating a local OpenStreetMap community. I personally feel it's crucial people internalize that's a project with a bottom-up organization. A "civil society", if you will. Alas, despite 200-250 daily active mappers in Poland [1] only a fraction ever posts on the forum, even if just to ask something. Meeting IRL is also futile except 2-3 largest cities. We have a forum, Facebook group and Discord. Whatever I'd come up with, I'd like it to be a self-healing organism that would continue to thrive even if some important members become inactive. And to feel it's not a Sisyphean task. [1] https://osmstats.neis-one.org/?item=countries&country=Poland https://osmstats.neis-one.org/?item=countries&country=Poland

scoofy 4y ago
I'm confused. I see OSM as a huge success. Firms like Mapbox and Strava have an incentive to update that wiki with you. The maps are amazing. Any local sub-community will be challenging to organize, but maybe the metric should be making it fun for you, and if you're having a good time others will join. I am also starting a wiki, a golf course wiki: https://golfcourse.wiki https://golfcourse.wiki It's still very early for me, but I see myself in this post. It's going to be extremely slow going, as the main consumers of the content created have no desire to contribute, but since golf courses are mostly static, as long as i plug away, adding a little bit all the time, the site will be a success. It costs almost nothing to operate. Generating traffic is challenging, and i've been trying to find more people willing to contribute, but again, I see my product as best placed if i do exactly the opposite of what most of the golf course info sites are doing. Don't base my success on engagement, no aggregation, no overly-prodding to contribute. Just one day at a time, with the assumption that the task is endless anyway, having people share their passion on their own time. If i can make that profitable at all, and it shouldn't be too difficult as the site grows, i'll see it as a total win.

MoscMob 4y ago

Sophistifunk 4y ago

Twitter isn't a community, it's a platform. There's nothing at all wrong with community engangement. Platform engagement and the advertising model that requires it are the problems.

swayvil 4y ago

Maximizing community engagement, trolling and advertising are related. They're different heads of the same multiheaded demon. All 3 are concerned with attention. Are hungry for it. If social media could be optimized to deliver engaging conversation to everybody then the demon would be slain. Because the hunger would be satisfied. Does that sound right?

actionfromafar 4y ago
Engaging conversation is not as intense as raging. The hunger can never be satisfied.
sfink 4y ago
Seems reasonable to me, if you reduce it to putting the demon to sleep. The demon is always there. A forum filled with engaging, productive conversation can and will slide back to toxic crap if there isn't something keeping it in check. It may be "optimized" for engaging conversation, but what does that mean in practice? If it's fully or mostly automated, then it's relying on metrics, and humans will always always always find a way to maximize the score :: effort ratio of any metric. Low effort is incompatible with engaging productive conversation. If it's done fully or mostly manually, then that means actively keeping that demon snoozing.

civilized 4y ago

Here's the little discussion I have in my head when this issue comes up. "Engagement isn't a good metric. Optimize for having a good community." "But engagement is easy to measure AND it means people are using the product. If people are choosing to use the product, doesn't that mean the product is giving them value?" "Not necessarily. Drugs like tobacco and heroin have fantastic 'engagement' but we consider them bad for us." "But how do you tell the difference between something that engages because it's good and something that engages because it's some kind of drug?" "I guess one important signal would be when people know something is bad for them but they're doing it anyway?" "Is that the case for social media? Do people feel addicted - feel that it's bad for them but they can't stop?" I think we've heard a lot of anecdata on this last question, but I can't recall seeing any big survey on it. EDIT: according to this, people will pay to have their social media restricted: https://www.washingtonpost.com/outlook/2021/07/19/social-media-addiction-social-science/ https://www.washingtonpost.com/outlook/2021/07/19/social-med...

bawolff 4y ago

Is engagement really value neutral? In the extreme, if you spend all your time "engaging", something toxic is happening

swayvil 4y ago

Advertising is a kind of trolling. I think that's an important connection to ponder. It could lead to something.

nitwit005 4y ago

If you can find a way to easily quantify the quality of human social interactions, you're probably on the short list for a Nobel prize. It'd have applications in every field related to humans.

joelthelion 4y ago
I don't think that's completely unachievable with the current state of NLP tools. You could (roughly) measure the diversity of discussion topics, the frequency of positive and negative emotions, etc. It'd be work to do it well, but I think it could work decently.
- nitwit005 4y ago
  You're suggesting that you can find things you assume correlate with what people want out of a community, but several people here commenting they can't define what they want. There's low odds of it somehow matching what they want, but struggle to express.

Sohcahtoa82 4y ago

As an end user, fighting back against using anger to drive engagement is easy. Stop engaging with content that angers you. Don't follow politicians and news outlets you despise. If someone you follow retweets the latest garbage being spewed by Fox News, don't interact with it. No retweets, no replies, no likes, no matter how well they refute the bullshit or how well they "own" the original messenger. The algorithms feed you content they think you'll engage with. Train the algorithm to feed you better content.