7 ms·

The initial call to deprecate this page was quite obviously driven by usage telemetry (i.e. "engagement" metrics), rather than actually speaking to users about

The initial call to deprecate this page was quite obviously driven by usage telemetry (i.e. "engagement" metrics), rather than actually speaking to users about what they like and don't like. I suspect that companies sometimes believe they can automate their way into quality designs, by relying on analytics, rather than empathizing deeply with their customers.

ryanisnan 4y ago
I mean, how exactly do you argue with something like "0.5% of users have visited this page in the past 12 months, and 0.3% of users have only visited once"? I don't know what their telemetry said, but generally it's something like this. This is empirical enough to say decisively "most people do not care about this feature."
- gryn 4y ago
  if we follow that logic big red emergency stop buttons should be removed from machinery because of their low usage.
  - ryanisnan 4y ago
    Sure. But good-sense would, in the case of emergency stop buttons, tell you that despite a lack of thought or usage, they are required for safe operation. You can't make that conclusion with a webpage for content discovery.
    - gryn 4y ago
      You would need to model what kind of users you have on your website, And how they differ from one another. Do they even that that feature even exist? I'd say a lot of users ignore most of the (social) features of GitHub beside "a public remote git repo I can download"
- BiteCode_dev 4y ago
  I love the github trending page, but I need only to visit it 10 times a year for it to fullfill its role. Yet I go to github every day. So visits are not the proper metric to depict how useful it is. Just because I consult a doctor once a year doesn't make this appointment less useful than the pizza truck I go to every week.
  - ryanisnan 4y ago
    This is bad logic. Almost everyone goes to the doctor in a year. And again, what is the importance of a doctor’s office. I once used a security tool that looked at the world in a very narrow sense. Despite being a general purpose tool, it would often flag really bad false-positives like “this security group allows all ingress!”, while applied to zero resources, while completely missing other, real, consequential issues. My point is, you have to pull your head up, look at the real world and the context of your decision, and use your judgement. These aren’t philosophical mind teasers.
    - BiteCode_dev 4y ago
      It's not bad logic, it's an imperfect analogy which you are taking at face value instead of understanding the point that something can be useful even if infrequently used. Wether or not it applies at this case is not determined, but it does underline the fact using only visit metrics for this particular page does not give the whole story. Which is congruent with the reaction the announcement got.
      - ryanisnan 4y ago
        To be clear, I don’t know what telemetry they are using, I only used percentage of users who visited as an example. It does sound like, of the small group of folks who use it, they do enjoy it. Sounds like GitHub has to find a way to either expand that pool, or decrease the maintenance cost of the feature, or it’s possibly not worth it.
    - OJFord 4y ago
      > This is bad logic. Almost everyone goes to the doctor in a year. They do? I'm not even making a point about developing countries and access to healthcare, I assume 'everyone' was implicitly 'everyone here' or in developed countries or whatever. But I would be incredibly surprised if that's true here in the UK. Is it an artefact of US health insurance that you can typically claim a 'check-up' per year (so why not even if not needed, you paid for it) or even that it's insisted on for the validity of the policy (ensure you don't have something expensive developing) or something?
- baq 4y ago
  0.5% of 100M is still a lot, right?
- a_c 4y ago
  Understand your audience, then back it up with metrics. Otherwise we will easily exclude audience e.g. those who need accessibility support
naikrovek 4y ago
> The initial call to deprecate this page was quite obviously driven by usage telemetry (i.e. "engagement" metrics) yeah, they said that. > rather than actually speaking to users about what they like and don't like how do you suggest they talk to their users to do this? less than 1% of people invited to take a survey actually take the survey. however, when you say you want to remove something, those who use it make the effort to respond; at a far higher rate than would participate in a survey. by saying they were going to remove the feature, they solicited better feedback than they would have gotten in any other way. if that was not intentional on some level, I would be surprised. > I suspect that companies sometimes believe they can automate their way into quality designs, by relying on analytics, rather than empathizing deeply with their customers. I suspect that some people will look for any excuse at all to share their opinion about the horrible new world with its horrible new ways as if only they and no one else knows what they are doing. I am not referring to you specifically, but your comment reminded me of just how often I see this "new ways to do things are always bad" sentiment and it is so wrong in so many ways.
- DelightOne 4y ago
  Maybe add a page-improvement survey banner, where one option is to take the survey and another is to say "it does all that I need" and another is "I don't need it". Note on the banner that this is about the page not seeing much use.
  - naikrovek 4y ago
    very few people will even see that prompt. <1% of those that do will respond. i encourage anyone who doesn’t believe me to spend time with UX researchers while they hold focus groups. it is absolutely terrifying what must be done to have a thin hope that 50% of your users will find something that they want to find. i have always had very little difficulty noticing differences in pages i visit even semi-regularly, so it was very easy for me to dismiss those who don’t as “idiot users” and to write them off an unimportant. it turns out that people who don’t notice things that you put in a large font, bold, in the center of the screen are a very large portion of your users. it’s frightening what people simply do not see.
A4ET8a8uTh0 4y ago
I was recently listening to GGG founder Chris Wilson and he was discussing how analytics can easily misfire if you do not apply them in a more conscious way. In his story, when reviewing logs someone saw that there was a big drop in engagement from quest A to quest B ( people stopped playing for a while between those two events ). What Wilson explained is that it is not exactly a surprise. Quest A effectively marks the end of a chapter and a lot of people stop at that point. Chris's insight from that was and I am paraphrasing so hopefully I will get it right: 'It is ok to dive into data with a question you want answered in mind; we should be careful about stumbling onto data and assume the question as a result.' Needless to say, it is a little different from, say, Amazon's creed so it is somewhat controversial for analysts.
- hn_user2 4y ago
  That’s a good perspective from Chris. Did not expect to see this mentioned though with the current mutiny happening by their players.
impulser_ 4y ago
But even if it not used by a lot of people, why remove it? It's literally just webpage with a list of repos that have been starred the most. It's not like it a complicated feature that needs a bunch of people to maintain lol.
- w-ll 4y ago
  When you have a score (stars) people tend to start gaming it. Doubly so now
- sofixa 4y ago
  From TFA: > "The data pipeline that powers the page has historically proven problematic to maintain and so given the low usage, we decided to switch off the Trending page." https://github.com/orgs/community/discussions/31644#discussioncomment-3539690 https://github.com/orgs/community/discussions/31644#discussi...

ryanisnan 4y ago

I mean, how exactly do you argue with something like "0.5% of users have visited this page in the past 12 months, and 0.3% of users have only visited once"? I don't know what their telemetry said, but generally it's something like this. This is empirical enough to say decisively "most people do not care about this feature."

gryn 4y ago
if we follow that logic big red emergency stop buttons should be removed from machinery because of their low usage.
- ryanisnan 4y ago
  Sure. But good-sense would, in the case of emergency stop buttons, tell you that despite a lack of thought or usage, they are required for safe operation. You can't make that conclusion with a webpage for content discovery.
  - gryn 4y ago
    You would need to model what kind of users you have on your website, And how they differ from one another. Do they even that that feature even exist? I'd say a lot of users ignore most of the (social) features of GitHub beside "a public remote git repo I can download"
BiteCode_dev 4y ago
I love the github trending page, but I need only to visit it 10 times a year for it to fullfill its role. Yet I go to github every day. So visits are not the proper metric to depict how useful it is. Just because I consult a doctor once a year doesn't make this appointment less useful than the pizza truck I go to every week.
- ryanisnan 4y ago
  This is bad logic. Almost everyone goes to the doctor in a year. And again, what is the importance of a doctor’s office. I once used a security tool that looked at the world in a very narrow sense. Despite being a general purpose tool, it would often flag really bad false-positives like “this security group allows all ingress!”, while applied to zero resources, while completely missing other, real, consequential issues. My point is, you have to pull your head up, look at the real world and the context of your decision, and use your judgement. These aren’t philosophical mind teasers.
  - BiteCode_dev 4y ago
    It's not bad logic, it's an imperfect analogy which you are taking at face value instead of understanding the point that something can be useful even if infrequently used. Wether or not it applies at this case is not determined, but it does underline the fact using only visit metrics for this particular page does not give the whole story. Which is congruent with the reaction the announcement got.
    - ryanisnan 4y ago
      To be clear, I don’t know what telemetry they are using, I only used percentage of users who visited as an example. It does sound like, of the small group of folks who use it, they do enjoy it. Sounds like GitHub has to find a way to either expand that pool, or decrease the maintenance cost of the feature, or it’s possibly not worth it.
  - OJFord 4y ago
    > This is bad logic. Almost everyone goes to the doctor in a year. They do? I'm not even making a point about developing countries and access to healthcare, I assume 'everyone' was implicitly 'everyone here' or in developed countries or whatever. But I would be incredibly surprised if that's true here in the UK. Is it an artefact of US health insurance that you can typically claim a 'check-up' per year (so why not even if not needed, you paid for it) or even that it's insisted on for the validity of the policy (ensure you don't have something expensive developing) or something?
baq 4y ago
0.5% of 100M is still a lot, right?
a_c 4y ago
Understand your audience, then back it up with metrics. Otherwise we will easily exclude audience e.g. those who need accessibility support

naikrovek 4y ago

> The initial call to deprecate this page was quite obviously driven by usage telemetry (i.e. "engagement" metrics) yeah, they said that. > rather than actually speaking to users about what they like and don't like how do you suggest they talk to their users to do this? less than 1% of people invited to take a survey actually take the survey. however, when you say you want to remove something, those who use it make the effort to respond; at a far higher rate than would participate in a survey. by saying they were going to remove the feature, they solicited better feedback than they would have gotten in any other way. if that was not intentional on some level, I would be surprised. > I suspect that companies sometimes believe they can automate their way into quality designs, by relying on analytics, rather than empathizing deeply with their customers. I suspect that some people will look for any excuse at all to share their opinion about the horrible new world with its horrible new ways as if only they and no one else knows what they are doing. I am not referring to you specifically, but your comment reminded me of just how often I see this "new ways to do things are always bad" sentiment and it is so wrong in so many ways.

DelightOne 4y ago
Maybe add a page-improvement survey banner, where one option is to take the survey and another is to say "it does all that I need" and another is "I don't need it". Note on the banner that this is about the page not seeing much use.
- naikrovek 4y ago
  very few people will even see that prompt. <1% of those that do will respond. i encourage anyone who doesn’t believe me to spend time with UX researchers while they hold focus groups. it is absolutely terrifying what must be done to have a thin hope that 50% of your users will find something that they want to find. i have always had very little difficulty noticing differences in pages i visit even semi-regularly, so it was very easy for me to dismiss those who don’t as “idiot users” and to write them off an unimportant. it turns out that people who don’t notice things that you put in a large font, bold, in the center of the screen are a very large portion of your users. it’s frightening what people simply do not see.

A4ET8a8uTh0 4y ago

I was recently listening to GGG founder Chris Wilson and he was discussing how analytics can easily misfire if you do not apply them in a more conscious way. In his story, when reviewing logs someone saw that there was a big drop in engagement from quest A to quest B ( people stopped playing for a while between those two events ). What Wilson explained is that it is not exactly a surprise. Quest A effectively marks the end of a chapter and a lot of people stop at that point. Chris's insight from that was and I am paraphrasing so hopefully I will get it right: 'It is ok to dive into data with a question you want answered in mind; we should be careful about stumbling onto data and assume the question as a result.' Needless to say, it is a little different from, say, Amazon's creed so it is somewhat controversial for analysts.

hn_user2 4y ago
That’s a good perspective from Chris. Did not expect to see this mentioned though with the current mutiny happening by their players.

impulser_ 4y ago

But even if it not used by a lot of people, why remove it? It's literally just webpage with a list of repos that have been starred the most. It's not like it a complicated feature that needs a bunch of people to maintain lol.

w-ll 4y ago
When you have a score (stars) people tend to start gaming it. Doubly so now
sofixa 4y ago
From TFA: > "The data pipeline that powers the page has historically proven problematic to maintain and so given the low usage, we decided to switch off the Trending page." https://github.com/orgs/community/discussions/31644#discussioncomment-3539690 https://github.com/orgs/community/discussions/31644#discussi...