35 ms·

Why we collect telemetry ...our team needs visibility into how features are being used in practice. We use this data to prioritize our work and evaluate wh

by a2128 6mo ago

Why we collect telemetry ...our team needs visibility into how features are being used in practice. We use this data to prioritize our work and evaluate whether features are meeting real user needs. I'm curious why corporate development teams always feel the need to spy on their users? Is it not sufficient to employ good engineering and design practices? Git has served us well for 20+ years without detailed analytics over who exactly is using which features and commands. Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction?

embedding-shape 6mo ago
> Is it not sufficient to employ good engineering and design practices? It's not that it's insufficient, new developers, product people and designers literally don't know how to make tasteful and useful decisions without first "asking users" by experimenting on them. Used to be you built up an intuition for your user base, but considering everyone is changing jobs every year, I guess people don't have time for that anymore, so literally every decision is "data driven" and no user is super happy or not anymore, everyone is just "OK, that's fine".
j_maffe 6mo ago
> Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction? I'm not sure if you're implying it's obvious but it's not obvious to me that it would be unhelpful.
- a2128 6mo ago
  Just anecdotally, I get the feeling telemetry often does more harm than good, because it's too easy to misinterpret or lie with statistics. There needs to be proper statistical methodology and biases need to be considered, but this doesn't always happen. Maybe a contrived example, but someone wants to show high impact on their next performance review? Implement the new feature in such a way that everyone easily misclicks it, then show the extremely high engagement as demonstration that their work is a huge success. For Git, I'm not sure it would be widely adopted today if the development process was mainly telemetry-driven rather than Torvalds developing it based solely on his expertise and intuition.
  - wongarsu 6mo ago
    Not to mention it's really hard to statistically tell the difference between people spending a lot of time with a feature because it's really useful or because it's really difficult to get to do what you want Telemetry is a really poor substitute for actually observing a couple of your users. But it's cheap and feels scientific and inclusive/fair (after all you are looking at everyone)
    - Sytten 6mo ago
      That is just poor analytics IMO, if you have a good harness you can definitely tell if a feature is not well designed. You have to optimize for things like number of clicks to perform an operation not time spent in app.
- sammorrowdrums 6mo ago
  I think the seeing the underutilized commands and flags (with real data not just a hunch) would have helped identify where users were not understanding why they should use it, and could have helped refine the interface and docs to make it gradually more usable. I mean no solution is perfect, and some underused things are just only sometimes extremely useful, but data used smartly is not a waste of time.
Forgeties79 6mo ago
This is where (surprise surprise) I respect Valve. The hardware survey is opt in and transparent. They get useful info out of it and it’s just..not scummy. There are all sorts of best practices for getting info without vacuuming up everyone’s data in opaque ways.
- salomonk_mur 6mo ago
  They are analyzing absolutely every click you make, I can guarantee it.
  - Forgeties79 6mo ago
    And if you provide evidence of this (and yes I think it is possible) then I will say it’s bad. The hardware survey is not that.
- embedding-shape 6mo ago
  To be fair, you can be pretty sure they're heavily leveraging all their store data, in loads of ways. They probably sit on the biggest dataset of video game preferences for people in general, and I'm betting they make use of it heavily.
  - Forgeties79 6mo ago
    If you have details on what they’re collecting and how they’re using it/if they’re selling it to advertisers/etc, I’m happy to make a judgment. I’m not saying they don’t engage in any of those practices, I am specifically talking about the hardware survey.
    - embedding-shape 6mo ago
      > If you have details on what they’re collecting Well, you can start with everything a typical HTTP request and TCP connection comes with, surely they're already storing those things for "anti-fraud practices", wouldn't be far to imagine this data warehouse is used for analytics and product decisions as well.
gardaani 6mo ago
It isn't only corporate development teams — open source development teams want to spy on their users, too. For instance, Homebrew: "Anonymous analytics allow us to prioritise fixes and features based on how, where and when people use Homebrew." [1] [1] https://docs.brew.sh/Analytics https://docs.brew.sh/Analytics
- leftnode 6mo ago
  Is it spying if: 1. It's anonymous 2. They're telling you they're doing it 3. You can opt out of it
  - brontosaurusrex 6mo ago
    "I'am watching you" is neat way to communicate with people?
  - chillfox 6mo ago
    Yes!
giancarlostoro 6mo ago
> I'm curious why corporate development teams always feel the need to spy on their users? I've repeatedly talked about this on HN; I call it Marketing Driven Development. It's when some Marketing manager goes to your IT manager and starts asking for things that no customer wants or needs, so they can track if their initiatives justify their job, aka are they bringing in more people to x feature? Honestly, with something as sensitive as software developer tools, I think any sort of telemetry should ALWAYS be off by default.
figmert 6mo ago
While I agree, I personally always opt out if I'm aware, and hate it when a tool suddenly gets telemetry, I don't think Git is comparable, same with Linux. Linux and Git are fully open source, and have big companies contribute to it. If a company like Google, Microsoft etc need a feature, they can usually afford to hire someone and develop _and_ maintain this feature. Something like gh is the opposite. It's maintained by a singular organisation, the team maintaining this has a finite resources. I don't think it's much to ask for understand what features are being used, what errors might come up, etc.
- LtWorf 6mo ago
  Good news! gh is actually a client of a web API so they can just read their logs to know what's being used!
hansmayer 6mo ago
> I'm curious why corporate development teams always feel the need to spy on their users Unfortunately this is due to a large part of "decision makers" being non-technical folks, not being able to understand how the tools is actually used, as they don't use such tools themselves. So some product manager "responsible" for development tooling needs this sort of stuff to be able to perform in their job, just as some clueless product manager in the e-commerce absolutely has to overload your frontend with scripts tracking your behaviour, also to be able to perform in their job. Of course the question remains, why do those jobs exist in the first place, as the engineers were perfectly capable of designing interaction with their users before the VCs imposed the unfortunate paradigm of a deeply non-technical person somehow leading the design and development of highly technical products...So here we are, sharing our data with them, because how else will Joe collect their PM paycheck, in between prompting the AI for his slides and various "very important" meetings...
- ryanmcbride 6mo ago
  Man if I had a nickle for every time a PM asked me to violate user privacy for the purposes of making a slide that will be shown to their boss for 2.5 seconds I'd probably make enough to actually retire someday.
  - phillipcarter 6mo ago
    There's obviously misuse and abuse in the world, but telemetry from production systems out in the wild is incredibly useful for all kinds of decision-making. It's silly to dismiss it outright.
    - hansmayer 6mo ago
      Some of it, yes. But not the kind that's put in place so the clueless Joe who used to do copy-writing 3 years ago can keep their high salary because they decided to join the proverbial "IT".
    - ryanmcbride 5mo ago
      Sure but I'm talking about actual privacy violations that have been pitched to me over my career I'm not calling all telemetry a privacy violation.
charcircuit 6mo ago
Git notoriously has had performance issues and did not scale and has had a horrible user interface. Both of these problems can be measured using telemetry and improvements can be measured once telemetry is in place.
- LtWorf 6mo ago
  How was it notorious if git has no telemetry? According to you without telemetry nothing can be known, and nothing can become notorious.
  - charcircuit 6mo ago
    Because you already have a general idea by just using it. "Oh wow this is slow." Telemetry gives you hard data.
  - tuwtuwtuwtuw 6mo ago
    He did not write that.
gordonhart 6mo ago
The impact of a few more network calls and decreased privacy is basically never felt by users beyond this abstract "they're spying on me" realization. The impact of this telemetry for a product development team is material. Not saying that telemetry more valuable than privacy, just that it's a straightforward decision for a company to make when real benefits are only counterbalanced by abstract privacy concerns. This is why it's so universally applied across apps and tools developed commercially.
- TheDong 6mo ago
  For most CLIs, I definitely feel extra network calls because they translate to real latency for commands that _should_ be quick. If I run "gh alias set foo bar", and that takes even a marginally perceptible amount of time, I'll feel like the tool I'm using is poorly built since a local alias obviously doesn't need network calls. I do see that `gh` is spawning a child to do sending in the background (https://github.com/cli/cli/blob/3ad29588b8bf9f2390be652f46ee4ef75b875b03/internal/telemetry/telemetry.go#L350 https://github.com/cli/cli/blob/3ad29588b8bf9f2390be652f46ee...), which also is something I'd be annoyed at since having background processes lingering in a shell's session is bad manners for a command that doesn't have a very good reason to do so.
  - SchemaLoad 6mo ago
    If it's done in a background process then it won't impact the speed of the tool at all. When the choice is between getting data to help improve the tool at the cost of "bad manners" whatever that means, the choice is pretty easy.
dale_glass 6mo ago
> I'm curious why corporate development teams always feel the need to spy on their users? Is it not sufficient to employ good engineering and design practices? No, because users have different needs and thoughts from the developers. And because sometimes it's hard to get good feedback from people. Maybe everyone loves the concept of feature X, but then never uses it in practice for some reason. Or a given feature has a vocal fan base that won't actually translate to sales/real usage. > Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction? I think yes, because git famously has a terrible UI, and any amount of telemetry would quickly tell you people fumble around a lot at first. I imagine that in an alternate world, a git with telemetry would have come out with a less confusing UI because somebody would have looked at the stats and for instance have added "git restore" right from the very start, because "git checkout -- foo.txt" is an absolutely unintuitive command.
- wongarsu 6mo ago
  A more intuitive git UI would reduce engagement. Do you really want to cut a 30 minute git session down to five minutes by introducing things like 'git restore' or 'git undo'? /s
- throwaway27448 6mo ago
  > because git famously has a terrible UI Thankfully, github has zero control over git. If they did have control they would have sank the whole operation on year one > because somebody would have looked at the stats and for instance have added "git restore" right from the very start, because "git checkout -- foo.txt" is an absolutely unintuitive command. How is git restore any better? Restoring what from when? At least git checkout is clear in what it does.
  - dale_glass 6mo ago
    > How is git restore any better? Restoring what from when? At least git checkout is clear in what it does. And this is exactly where disconnects happen, and where you need telemetry or something like it to tell you how your users actually use the system, rather than imagining how they should. A technical user deep into the guts of Git thinks "you need to check out again this specific file". A novice thinks "I want to restore this file to the state it had before I touched it". Now we can argue about whether "restore" is the ideal word here, but all the same, end users tend to think it terms of "I want to undo what I did", and not in terms of git internals. So a hypothetical git with telemetry would probably show people repeatedly trying "git restore", "git undo", "git revert", etc, trying to find an undo command.
redsocksfan45 6mo ago
[dead]
reaperducer 6mo ago
I'm curious why corporate development teams always feel the need to spy on their users? Because they're too shy, lazy, or socially awkward to actually ask their users questions. They cover up this anxiety and laziness by saying that it costs too much, or it doesn't "scale." Both of these are false. My company requires me to actually speak to the people who use the web sites I build; usually about every ten to twelve months. The company pays for my time, travel, and other expenses. The company does this because it cares about the product. It has to, because it is beholden to the customers for its financial position, not to anonymous stock market trading bots a continent away.
- Sytten 6mo ago
  Respectfully I think your argument defeats itself. If you can only speak to your users once every 10-12 months it means your process doesn't scale by definition. Good analytics (not useless vanity metrics) should allow you to spot a problem days after it was launched not wait 3 quarters for a user to air their grievances.
  - Citizen_Lame 6mo ago
    Ah yes, all the spyware on Windows 11 really helped Microsoft scale up development and make it the best Windows version ever. Now, let's replicate this with GitHub. What can go wrong?
  - reaperducer 6mo ago
    You're describing a different problem. Bug fixing absolutely gets taken care of immediately, and our customers are very active in telling us about them through these strange new feedback mechanisms known as "e-mail" and "a telephone." But we don't spy on people to fix bugs. Nothing that the big tech "telemetry" is doing is about bug fixes. In the article we're all talking about the spying that Microsoft proposes isn't to fix bugs. Re-read what it wrote. It's all for things that may not appear for weeks, months, or years. And to think that a trillion-dollar company like Microsoft can't figure out how, or doesn't have the money available to scale real customer feedback is just sticking your head in the sand and making excuses. Microsoft doesn't need people to apologize for its failure.
Sytten 6mo ago
I used to believe that it was not necessary until I started building my own startup. If you dont have analytics you are flying blind. You don't know what your users actually care about and how to optimize a successful user journey. The difference between what people tell you when asked directly and how they actually use your software is actually shocking.
- EGG_CREAM 6mo ago
  [dead]
- sidkshatriya 6mo ago
  > If you dont have analytics you are flying blind If you have too much emphasis on (invasive) analytics you might end up flying empty i.e. without customers.
- throwaway27448 6mo ago
  I'm pretty ok with the github cli tool team flying blind. The tool isn't exactly a necessary part of any workflow. You don't need telemetry to glean that
  - merlindru 6mo ago
    that's akin to saying "i do not need their product therefore i don't care"... so what's your point? someone may have made it part of their workflow!
    - deleted 6mo ago
      [deleted]
    - throwaway27448 6mo ago
      True. Some people shouldn't use git if their workflow doesn't beg it.
- embedding-shape 6mo ago
  You're only flying blind if you make decisions not looking and thinking. Analytics isn't the only way to figure out "what your users actually care about", you can also try the old school way, commonly referred to as "Talking with people", then after taking notes, you think about it, maybe discuss with others. Don't take what people say at face value, but think about it together with your knowledge and experience, and you'll make even better product decisions than the people who are only making "data driven decisions" all the time.
ForHackernews 6mo ago
Arguably yes. git has a terrible developer experience and we've only gotten to this point where everyone embraces it through Stockholm syndrome. If someone had been looking at analytics from git, they'd have seen millions of confused people trying to find the right incantation in a forest of confusing poorly named flags. Sincerely, a Mercurial user from way back.
- ButlerianJihad 6mo ago
  https://xkcd.com/1597/ https://xkcd.com/1597/
dualvariable 6mo ago
> Is it not sufficient to employ good engineering and design practices? Git... Git has horrible design and ergonomics. It is an excellent example of engineers designing interfaces for engineers without a good feedback loop. Ironically, you just proved your point that engineers need to better understand how users are actually using their product, because their mental visualizations of how their product gets used is usually poor.
- consp 6mo ago
  Apparently I use git wrong since I do not feel this design and ergonomics issue.
  - deleted 6mo ago
    [deleted]
  - halapro 6mo ago
    How many years of experience with git do you have? How much of git do you use? I bet you use 5 commands and 10 flags at most. Take a look at git's docs
- skydhash 6mo ago
  > Git has horrible design and ergonomics. People say this and never has written about the supposed failure of design. Git has a very good conceptual model, and then provides operations (aptly named when you know about the model) to manipulate it. Most people who complains about git only think of it as code storage (folder {v1,v2,...}) instead of version control.
  - halapro 6mo ago
    > never has written about If you don't want to look at what people write you can't say that they haven't written about it. > the supposed failure of design I don’t think people complain about the internals of git itself as much as the complexity of all the operations. If you want to read about complaints, you really don't have to look further than the myriad of git GUIs, TUIs and otherwise alternative/simplified interfaces.
dietr1ch 6mo ago
It's not the devs themselves, but the team/project/product management show that needs to pretend they are data driven, but then resort to the silliest metrics that are easy to measure.
rafram 6mo ago
> Would Git have been significantly better if it had collected telemetry Yes, probably. Git is seriously hard to use beyond basic tasks. It has a byzantine array of commands, and the "porcelain" feels a lot closer to "plumbing" than it should. You and I are used to it, but that doesn't make it good. I mean, it took 14 years before it gained a `switch` command! `checkout` and `reset` can do like six different things depending on how your arguments resolve, from nondestructive to very, very destructive; safe(r) operations like --force-with-lease are made harder to find than their more dangerous counterparts; it's a mess. Analytics alone wouldn't solve the problem - you also need a team of developers who are willing to listen to their users, pore through usage data, and prioritize UX - but it would be something.
1vuio0pswjnm7 6mo ago
Perhaps the more interesting question is why these companies feel the need to "explain" why they are collecting telemetry or "disclose" how the data is used The software user has no means to verify the explanation or disclosure is accurate or complete. Once the data is transferred to the company then the user has no control over where it goes, who sees it or how it is used When the company states "We use the data for X" it is not promising to use the data for X in the future, nor does it prevent the company, or one of its "business partners", from using the data additionally for something else besides X Why "explain" the reason for collecting telemetry Why "disclose" how the data is used What does this accomplish
lo1tuma 6mo ago
I’m curious as well. Github is one of the rare products out there that get actual valuable user feedback. So why not just ask the users for specific feedback instead of tracking all of them.
mbreese 6mo ago
> I'm curious why corporate development teams always feel the need to spy on their users? This isn’t that surprising to me. Having usage data is important for many purposes. Even Debian has an opt-in usage tracker (popcon) to see wha packages they should keep supporting. What I’m curious about is why this is included in the CLI. Why aren’t they measuring this at the API level where they wouldn’t need to disclose it to anyone? What is done locally with the GH CLI tool that doesn’t interact with the GitHub servers?
_heimdall 6mo ago
Anonymous telemetry isn't necessarily spying, though "pseudoanonymous" sounds about as well protected as distinguishing between free speech and "absolutism." Github also wouldn't be tracking git use here, but the `gh` CLI that you don't need to install. All that said, having been in plenty of corporate environments I would be surprised if the data is anonymized and wouldn't be surprised if the primary motivator boils down to something like internal OKRs and politics.
chrishill89 6mo ago
Git relatively recently got an `--i-still-use-this` option for two deprecated commands that you have to run if you want to use them. The error you get tells you about it and that you should "please email us here" if you really am unable to figure out an alternative. I guess that's the price of regular and non-invasive software.
naikrovek 6mo ago
I'm curious why people think this is in the same ballpark as that something like a private investigator can do. This isn't spying at all. "oh no, they're aware of someone at the computer 19416146-F56B-49E4-BF16-C0D8B337BF7F running `gh api` a lot! that's spying!"
poulpy123 6mo ago
The current IA boom is entirely based on data . The more data you have the more you can train and the more money you make
rienbdj 6mo ago
When allocating engineering spend you need to predict impact. If you know how features of GitHub CLI are used and how you can do this more easily.
KronisLV 6mo ago
> I'm curious why corporate development teams always feel the need to spy on their users? Cause the alternative is viewing all of your app as one opaque blob - you don't know exactly how it's being used, which features actually need your attention, especially if you're spread thin. If you're in consulting or something like that and the clients haven't let you configure and/or access analytics (and the same goes for APM and log shipping), it's like flying blind. Couple that with vague bug reports instead of automated session recording and if you need to maintain that, you'll have gray hairs appearing by the age of 30. Take that disregard of measurement and spread it all across the development culture and you'll get errors in the logs that nobody is seeing and no insights into application performance - with the system working okay at a load X, but falling over at X+1 and you having to spend late evenings trying to refactor it, knowing that it needs to be shipped in less than a week because of client deadlines. Unless the data is something that's heavily regulated and more trouble than it's worth, more data will be better than less data, if you do something meaningful with it. > Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction? Knowing the most common fuck ups and foot guns might inform better CLI design. Otherwise people saying that it's good have about as much right to do so as saying that it's bad (at least in regards to UX), without knowing the ground level truth about what 90% of the users experience.
- skydhash 6mo ago
  > you don't know exactly how it's being used, which features actually need your attention, especially if you're spread thin. Why not conduct a survey? > vague bug reports instead of automated session recording and if you need to maintain that, you'll have gray hairs appearing by the age of 30. If it's a customer, why not reach directly to him? > with the system working okay at a load X, but falling over at X+1 and you having to spend late evenings trying to refactor it, No one is talking about telemetry on your servers. We're talking about telemetry on client's computers.
  - KronisLV 6mo ago
    > Why not conduct a survey? Large amounts of time before getting feedback, low percentage of people responding, not an accurate sample of all users (you will get signal from the loudest ones) and inaccurate data (biases in perception) instead of measurable reality. Not useless, but not a full replacement for telemetry. > If it's a customer, why not reach directly to him? Layers of indirection (and also the slowness of getting through them). You might not control the processes and approvals needed to do that in a non-startup environment. You will probably control enough of the app to add various technical solutions to aid you in collecting information. > No one is talking about telemetry on your servers. I am. Culture of not collecting client side data also often comes together with a culture of not collecting server side data properly either. Competent teams will evaluate both. My argument is that all of this data can meaningfully help in development and that other approaches don't replace it well enough.
Lammy 6mo ago
The people who write any individual feature want to be able to prove usage in order to get good performance reviews and promotions. It's so awful that it's become normalized. Back in The Day we had the term “spyware” to refer to any piece of software that phoned home to report user behavior, but now that's just All Software.
rprend 6mo ago
Product work can be counterintuitive. An engineer / PM might think that a design or feature “makes sense”, but you don’t actually know that unless you measure usage.
bastardoperator 6mo ago
You have three features, A, B, and C. They are core features. Two of the features break. How do you prioritize which feature gets fixed first? With telemetry its obvious, without it, you're guessing. Also, gh cli is not about git, its about the github api. In theory the app has its own user agent and of course their LB is tracking all http requests, so not anonymous ever.
stronglikedan 6mo ago
> always feel the need to spy on their users? If it's truly pseudoanonymous then it's hardly spying, just sayin'... Others have answered your actual question better than I could have.
teeray 6mo ago
Because dashboard need to show number go up
high_na_euv 6mo ago
git is terrible from ux perspective >Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction? Definitely
sidkshatriya 6mo ago
> ...our team needs visibility into how features are being used in practice. We use this data to prioritize our work and evaluate whether features are meeting real user needs. You should be able to see what features are being used by seeing what server endpoints are being hit and how often. Don't need intrusive telemetry. Yes, it's not perfect. Many features could use the same endpoint. You could totally anonymise this and you could still get a great understanding what features users are using by looking at endpoint stats. Companies need to decide whether they want customer goodwill or very detailed insight into what they are doing. By having invasive telemetry you may have less customers (people leaving for github competitors). Is it worth it ?
ravenstine 6mo ago
"Real" numbers make it easier for them to lie to leadership who then use or ignore those lies to justify decisions they were already going to make in spite of users.
paulddraper 6mo ago
To state the obvious, "good engineering/design practices" will not tell you what features are used or not. > Git has served us well for 20+ years Funny. I think that, but the usual HN narrative is that Git is UX hostile.
caymanjim 6mo ago
The language used in statements like this always annoys me. "We want visibility", ok fine. "We need"...the hell you don't.
crubier 6mo ago
Come on, do you actually think "corporate engineers" care about what you are doing individually? Do you think they look specifically for you and make fun at your individual usage pattern? Do you think it gives them interesting information about your private life? No one cares, there are millions of users, no one is going to look at your data, and even less be able to actually know which person a given user is. We legit just want to know aggregated and objective information about how people use our products so we can make it better for you. "Why do all corporate try to spy on our usage patterns?" Because the ones who don't have a crap product and all died long ago.
raxxorraxor 6mo ago
Especially true because it has also been proved that these telemetry tools often picture a distorted reality. My "telemetry" check is Windows. I does the same, forced it on users and it certainly didn't get better from it. The correlation with quality decline might be independent, but I would wager some shitty manager put up a shitty metric, everything got optimized towards that and leadership has lost the bigger picture because these numbers cannot transport general understanding.
hulitu 6mo ago
> Why we collect telemetry ...our team needs visibility into how features are being used in practice. This was Microsoft's mantra, while bugs were not fixed for months.

by a2128 6mo ago

embedding-shape 6mo ago

> Is it not sufficient to employ good engineering and design practices? It's not that it's insufficient, new developers, product people and designers literally don't know how to make tasteful and useful decisions without first "asking users" by experimenting on them. Used to be you built up an intuition for your user base, but considering everyone is changing jobs every year, I guess people don't have time for that anymore, so literally every decision is "data driven" and no user is super happy or not anymore, everyone is just "OK, that's fine".

j_maffe 6mo ago

> Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction? I'm not sure if you're implying it's obvious but it's not obvious to me that it would be unhelpful.

a2128 6mo ago
Just anecdotally, I get the feeling telemetry often does more harm than good, because it's too easy to misinterpret or lie with statistics. There needs to be proper statistical methodology and biases need to be considered, but this doesn't always happen. Maybe a contrived example, but someone wants to show high impact on their next performance review? Implement the new feature in such a way that everyone easily misclicks it, then show the extremely high engagement as demonstration that their work is a huge success. For Git, I'm not sure it would be widely adopted today if the development process was mainly telemetry-driven rather than Torvalds developing it based solely on his expertise and intuition.
- wongarsu 6mo ago
  Not to mention it's really hard to statistically tell the difference between people spending a lot of time with a feature because it's really useful or because it's really difficult to get to do what you want Telemetry is a really poor substitute for actually observing a couple of your users. But it's cheap and feels scientific and inclusive/fair (after all you are looking at everyone)
  - Sytten 6mo ago
    That is just poor analytics IMO, if you have a good harness you can definitely tell if a feature is not well designed. You have to optimize for things like number of clicks to perform an operation not time spent in app.
sammorrowdrums 6mo ago
I think the seeing the underutilized commands and flags (with real data not just a hunch) would have helped identify where users were not understanding why they should use it, and could have helped refine the interface and docs to make it gradually more usable. I mean no solution is perfect, and some underused things are just only sometimes extremely useful, but data used smartly is not a waste of time.

Forgeties79 6mo ago

This is where (surprise surprise) I respect Valve. The hardware survey is opt in and transparent. They get useful info out of it and it’s just..not scummy. There are all sorts of best practices for getting info without vacuuming up everyone’s data in opaque ways.

salomonk_mur 6mo ago
They are analyzing absolutely every click you make, I can guarantee it.
- Forgeties79 6mo ago
  And if you provide evidence of this (and yes I think it is possible) then I will say it’s bad. The hardware survey is not that.
embedding-shape 6mo ago
To be fair, you can be pretty sure they're heavily leveraging all their store data, in loads of ways. They probably sit on the biggest dataset of video game preferences for people in general, and I'm betting they make use of it heavily.
- Forgeties79 6mo ago
  If you have details on what they’re collecting and how they’re using it/if they’re selling it to advertisers/etc, I’m happy to make a judgment. I’m not saying they don’t engage in any of those practices, I am specifically talking about the hardware survey.
  - embedding-shape 6mo ago
    > If you have details on what they’re collecting Well, you can start with everything a typical HTTP request and TCP connection comes with, surely they're already storing those things for "anti-fraud practices", wouldn't be far to imagine this data warehouse is used for analytics and product decisions as well.

gardaani 6mo ago

It isn't only corporate development teams — open source development teams want to spy on their users, too. For instance, Homebrew: "Anonymous analytics allow us to prioritise fixes and features based on how, where and when people use Homebrew." [1] [1] https://docs.brew.sh/Analytics https://docs.brew.sh/Analytics

leftnode 6mo ago
Is it spying if: 1. It's anonymous 2. They're telling you they're doing it 3. You can opt out of it
- brontosaurusrex 6mo ago
  "I'am watching you" is neat way to communicate with people?
- chillfox 6mo ago
  Yes!

giancarlostoro 6mo ago

> I'm curious why corporate development teams always feel the need to spy on their users? I've repeatedly talked about this on HN; I call it Marketing Driven Development. It's when some Marketing manager goes to your IT manager and starts asking for things that no customer wants or needs, so they can track if their initiatives justify their job, aka are they bringing in more people to x feature? Honestly, with something as sensitive as software developer tools, I think any sort of telemetry should ALWAYS be off by default.

figmert 6mo ago

While I agree, I personally always opt out if I'm aware, and hate it when a tool suddenly gets telemetry, I don't think Git is comparable, same with Linux. Linux and Git are fully open source, and have big companies contribute to it. If a company like Google, Microsoft etc need a feature, they can usually afford to hire someone and develop _and_ maintain this feature. Something like gh is the opposite. It's maintained by a singular organisation, the team maintaining this has a finite resources. I don't think it's much to ask for understand what features are being used, what errors might come up, etc.

LtWorf 6mo ago
Good news! gh is actually a client of a web API so they can just read their logs to know what's being used!

hansmayer 6mo ago

> I'm curious why corporate development teams always feel the need to spy on their users Unfortunately this is due to a large part of "decision makers" being non-technical folks, not being able to understand how the tools is actually used, as they don't use such tools themselves. So some product manager "responsible" for development tooling needs this sort of stuff to be able to perform in their job, just as some clueless product manager in the e-commerce absolutely has to overload your frontend with scripts tracking your behaviour, also to be able to perform in their job. Of course the question remains, why do those jobs exist in the first place, as the engineers were perfectly capable of designing interaction with their users before the VCs imposed the unfortunate paradigm of a deeply non-technical person somehow leading the design and development of highly technical products...So here we are, sharing our data with them, because how else will Joe collect their PM paycheck, in between prompting the AI for his slides and various "very important" meetings...

ryanmcbride 6mo ago
Man if I had a nickle for every time a PM asked me to violate user privacy for the purposes of making a slide that will be shown to their boss for 2.5 seconds I'd probably make enough to actually retire someday.
- phillipcarter 6mo ago
  There's obviously misuse and abuse in the world, but telemetry from production systems out in the wild is incredibly useful for all kinds of decision-making. It's silly to dismiss it outright.
  - hansmayer 6mo ago
    Some of it, yes. But not the kind that's put in place so the clueless Joe who used to do copy-writing 3 years ago can keep their high salary because they decided to join the proverbial "IT".
  - ryanmcbride 5mo ago
    Sure but I'm talking about actual privacy violations that have been pitched to me over my career I'm not calling all telemetry a privacy violation.

charcircuit 6mo ago

Git notoriously has had performance issues and did not scale and has had a horrible user interface. Both of these problems can be measured using telemetry and improvements can be measured once telemetry is in place.

LtWorf 6mo ago
How was it notorious if git has no telemetry? According to you without telemetry nothing can be known, and nothing can become notorious.
- charcircuit 6mo ago
  Because you already have a general idea by just using it. "Oh wow this is slow." Telemetry gives you hard data.
- tuwtuwtuwtuw 6mo ago
  He did not write that.

gordonhart 6mo ago

The impact of a few more network calls and decreased privacy is basically never felt by users beyond this abstract "they're spying on me" realization. The impact of this telemetry for a product development team is material. Not saying that telemetry more valuable than privacy, just that it's a straightforward decision for a company to make when real benefits are only counterbalanced by abstract privacy concerns. This is why it's so universally applied across apps and tools developed commercially.

TheDong 6mo ago
For most CLIs, I definitely feel extra network calls because they translate to real latency for commands that _should_ be quick. If I run "gh alias set foo bar", and that takes even a marginally perceptible amount of time, I'll feel like the tool I'm using is poorly built since a local alias obviously doesn't need network calls. I do see that `gh` is spawning a child to do sending in the background (https://github.com/cli/cli/blob/3ad29588b8bf9f2390be652f46ee4ef75b875b03/internal/telemetry/telemetry.go#L350 https://github.com/cli/cli/blob/3ad29588b8bf9f2390be652f46ee...), which also is something I'd be annoyed at since having background processes lingering in a shell's session is bad manners for a command that doesn't have a very good reason to do so.
- SchemaLoad 6mo ago
  If it's done in a background process then it won't impact the speed of the tool at all. When the choice is between getting data to help improve the tool at the cost of "bad manners" whatever that means, the choice is pretty easy.

dale_glass 6mo ago

> I'm curious why corporate development teams always feel the need to spy on their users? Is it not sufficient to employ good engineering and design practices? No, because users have different needs and thoughts from the developers. And because sometimes it's hard to get good feedback from people. Maybe everyone loves the concept of feature X, but then never uses it in practice for some reason. Or a given feature has a vocal fan base that won't actually translate to sales/real usage. > Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction? I think yes, because git famously has a terrible UI, and any amount of telemetry would quickly tell you people fumble around a lot at first. I imagine that in an alternate world, a git with telemetry would have come out with a less confusing UI because somebody would have looked at the stats and for instance have added "git restore" right from the very start, because "git checkout -- foo.txt" is an absolutely unintuitive command.

wongarsu 6mo ago
A more intuitive git UI would reduce engagement. Do you really want to cut a 30 minute git session down to five minutes by introducing things like 'git restore' or 'git undo'? /s
throwaway27448 6mo ago
> because git famously has a terrible UI Thankfully, github has zero control over git. If they did have control they would have sank the whole operation on year one > because somebody would have looked at the stats and for instance have added "git restore" right from the very start, because "git checkout -- foo.txt" is an absolutely unintuitive command. How is git restore any better? Restoring what from when? At least git checkout is clear in what it does.
- dale_glass 6mo ago
  > How is git restore any better? Restoring what from when? At least git checkout is clear in what it does. And this is exactly where disconnects happen, and where you need telemetry or something like it to tell you how your users actually use the system, rather than imagining how they should. A technical user deep into the guts of Git thinks "you need to check out again this specific file". A novice thinks "I want to restore this file to the state it had before I touched it". Now we can argue about whether "restore" is the ideal word here, but all the same, end users tend to think it terms of "I want to undo what I did", and not in terms of git internals. So a hypothetical git with telemetry would probably show people repeatedly trying "git restore", "git undo", "git revert", etc, trying to find an undo command.

redsocksfan45 6mo ago

[dead]

reaperducer 6mo ago

I'm curious why corporate development teams always feel the need to spy on their users? Because they're too shy, lazy, or socially awkward to actually ask their users questions. They cover up this anxiety and laziness by saying that it costs too much, or it doesn't "scale." Both of these are false. My company requires me to actually speak to the people who use the web sites I build; usually about every ten to twelve months. The company pays for my time, travel, and other expenses. The company does this because it cares about the product. It has to, because it is beholden to the customers for its financial position, not to anonymous stock market trading bots a continent away.

Sytten 6mo ago
Respectfully I think your argument defeats itself. If you can only speak to your users once every 10-12 months it means your process doesn't scale by definition. Good analytics (not useless vanity metrics) should allow you to spot a problem days after it was launched not wait 3 quarters for a user to air their grievances.
- Citizen_Lame 6mo ago
  Ah yes, all the spyware on Windows 11 really helped Microsoft scale up development and make it the best Windows version ever. Now, let's replicate this with GitHub. What can go wrong?
- reaperducer 6mo ago
  You're describing a different problem. Bug fixing absolutely gets taken care of immediately, and our customers are very active in telling us about them through these strange new feedback mechanisms known as "e-mail" and "a telephone." But we don't spy on people to fix bugs. Nothing that the big tech "telemetry" is doing is about bug fixes. In the article we're all talking about the spying that Microsoft proposes isn't to fix bugs. Re-read what it wrote. It's all for things that may not appear for weeks, months, or years. And to think that a trillion-dollar company like Microsoft can't figure out how, or doesn't have the money available to scale real customer feedback is just sticking your head in the sand and making excuses. Microsoft doesn't need people to apologize for its failure.

Sytten 6mo ago

I used to believe that it was not necessary until I started building my own startup. If you dont have analytics you are flying blind. You don't know what your users actually care about and how to optimize a successful user journey. The difference between what people tell you when asked directly and how they actually use your software is actually shocking.

EGG_CREAM 6mo ago
[dead]
sidkshatriya 6mo ago
> If you dont have analytics you are flying blind If you have too much emphasis on (invasive) analytics you might end up flying empty i.e. without customers.
throwaway27448 6mo ago
I'm pretty ok with the github cli tool team flying blind. The tool isn't exactly a necessary part of any workflow. You don't need telemetry to glean that
- merlindru 6mo ago
  that's akin to saying "i do not need their product therefore i don't care"... so what's your point? someone may have made it part of their workflow!
  - deleted 6mo ago
    [deleted]
  - throwaway27448 6mo ago
    True. Some people shouldn't use git if their workflow doesn't beg it.
embedding-shape 6mo ago
You're only flying blind if you make decisions not looking and thinking. Analytics isn't the only way to figure out "what your users actually care about", you can also try the old school way, commonly referred to as "Talking with people", then after taking notes, you think about it, maybe discuss with others. Don't take what people say at face value, but think about it together with your knowledge and experience, and you'll make even better product decisions than the people who are only making "data driven decisions" all the time.

ForHackernews 6mo ago

Arguably yes. git has a terrible developer experience and we've only gotten to this point where everyone embraces it through Stockholm syndrome. If someone had been looking at analytics from git, they'd have seen millions of confused people trying to find the right incantation in a forest of confusing poorly named flags. Sincerely, a Mercurial user from way back.

ButlerianJihad 6mo ago
https://xkcd.com/1597/ https://xkcd.com/1597/

dualvariable 6mo ago

> Is it not sufficient to employ good engineering and design practices? Git... Git has horrible design and ergonomics. It is an excellent example of engineers designing interfaces for engineers without a good feedback loop. Ironically, you just proved your point that engineers need to better understand how users are actually using their product, because their mental visualizations of how their product gets used is usually poor.

consp 6mo ago
Apparently I use git wrong since I do not feel this design and ergonomics issue.
- deleted 6mo ago
  [deleted]
- halapro 6mo ago
  How many years of experience with git do you have? How much of git do you use? I bet you use 5 commands and 10 flags at most. Take a look at git's docs
skydhash 6mo ago
> Git has horrible design and ergonomics. People say this and never has written about the supposed failure of design. Git has a very good conceptual model, and then provides operations (aptly named when you know about the model) to manipulate it. Most people who complains about git only think of it as code storage (folder {v1,v2,...}) instead of version control.
- halapro 6mo ago
  > never has written about If you don't want to look at what people write you can't say that they haven't written about it. > the supposed failure of design I don’t think people complain about the internals of git itself as much as the complexity of all the operations. If you want to read about complaints, you really don't have to look further than the myriad of git GUIs, TUIs and otherwise alternative/simplified interfaces.

dietr1ch 6mo ago

It's not the devs themselves, but the team/project/product management show that needs to pretend they are data driven, but then resort to the silliest metrics that are easy to measure.

rafram 6mo ago

> Would Git have been significantly better if it had collected telemetry Yes, probably. Git is seriously hard to use beyond basic tasks. It has a byzantine array of commands, and the "porcelain" feels a lot closer to "plumbing" than it should. You and I are used to it, but that doesn't make it good. I mean, it took 14 years before it gained a `switch` command! `checkout` and `reset` can do like six different things depending on how your arguments resolve, from nondestructive to very, very destructive; safe(r) operations like --force-with-lease are made harder to find than their more dangerous counterparts; it's a mess. Analytics alone wouldn't solve the problem - you also need a team of developers who are willing to listen to their users, pore through usage data, and prioritize UX - but it would be something.

1vuio0pswjnm7 6mo ago

Perhaps the more interesting question is why these companies feel the need to "explain" why they are collecting telemetry or "disclose" how the data is used The software user has no means to verify the explanation or disclosure is accurate or complete. Once the data is transferred to the company then the user has no control over where it goes, who sees it or how it is used When the company states "We use the data for X" it is not promising to use the data for X in the future, nor does it prevent the company, or one of its "business partners", from using the data additionally for something else besides X Why "explain" the reason for collecting telemetry Why "disclose" how the data is used What does this accomplish

lo1tuma 6mo ago

I’m curious as well. Github is one of the rare products out there that get actual valuable user feedback. So why not just ask the users for specific feedback instead of tracking all of them.

mbreese 6mo ago

> I'm curious why corporate development teams always feel the need to spy on their users? This isn’t that surprising to me. Having usage data is important for many purposes. Even Debian has an opt-in usage tracker (popcon) to see wha packages they should keep supporting. What I’m curious about is why this is included in the CLI. Why aren’t they measuring this at the API level where they wouldn’t need to disclose it to anyone? What is done locally with the GH CLI tool that doesn’t interact with the GitHub servers?

_heimdall 6mo ago

Anonymous telemetry isn't necessarily spying, though "pseudoanonymous" sounds about as well protected as distinguishing between free speech and "absolutism." Github also wouldn't be tracking git use here, but the `gh` CLI that you don't need to install. All that said, having been in plenty of corporate environments I would be surprised if the data is anonymized and wouldn't be surprised if the primary motivator boils down to something like internal OKRs and politics.

chrishill89 6mo ago

Git relatively recently got an `--i-still-use-this` option for two deprecated commands that you have to run if you want to use them. The error you get tells you about it and that you should "please email us here" if you really am unable to figure out an alternative. I guess that's the price of regular and non-invasive software.

naikrovek 6mo ago

I'm curious why people think this is in the same ballpark as that something like a private investigator can do. This isn't spying at all. "oh no, they're aware of someone at the computer 19416146-F56B-49E4-BF16-C0D8B337BF7F running `gh api` a lot! that's spying!"

poulpy123 6mo ago

The current IA boom is entirely based on data . The more data you have the more you can train and the more money you make

rienbdj 6mo ago

When allocating engineering spend you need to predict impact. If you know how features of GitHub CLI are used and how you can do this more easily.

KronisLV 6mo ago

> I'm curious why corporate development teams always feel the need to spy on their users? Cause the alternative is viewing all of your app as one opaque blob - you don't know exactly how it's being used, which features actually need your attention, especially if you're spread thin. If you're in consulting or something like that and the clients haven't let you configure and/or access analytics (and the same goes for APM and log shipping), it's like flying blind. Couple that with vague bug reports instead of automated session recording and if you need to maintain that, you'll have gray hairs appearing by the age of 30. Take that disregard of measurement and spread it all across the development culture and you'll get errors in the logs that nobody is seeing and no insights into application performance - with the system working okay at a load X, but falling over at X+1 and you having to spend late evenings trying to refactor it, knowing that it needs to be shipped in less than a week because of client deadlines. Unless the data is something that's heavily regulated and more trouble than it's worth, more data will be better than less data, if you do something meaningful with it. > Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction? Knowing the most common fuck ups and foot guns might inform better CLI design. Otherwise people saying that it's good have about as much right to do so as saying that it's bad (at least in regards to UX), without knowing the ground level truth about what 90% of the users experience.

skydhash 6mo ago
> you don't know exactly how it's being used, which features actually need your attention, especially if you're spread thin. Why not conduct a survey? > vague bug reports instead of automated session recording and if you need to maintain that, you'll have gray hairs appearing by the age of 30. If it's a customer, why not reach directly to him? > with the system working okay at a load X, but falling over at X+1 and you having to spend late evenings trying to refactor it, No one is talking about telemetry on your servers. We're talking about telemetry on client's computers.
- KronisLV 6mo ago
  > Why not conduct a survey? Large amounts of time before getting feedback, low percentage of people responding, not an accurate sample of all users (you will get signal from the loudest ones) and inaccurate data (biases in perception) instead of measurable reality. Not useless, but not a full replacement for telemetry. > If it's a customer, why not reach directly to him? Layers of indirection (and also the slowness of getting through them). You might not control the processes and approvals needed to do that in a non-startup environment. You will probably control enough of the app to add various technical solutions to aid you in collecting information. > No one is talking about telemetry on your servers. I am. Culture of not collecting client side data also often comes together with a culture of not collecting server side data properly either. Competent teams will evaluate both. My argument is that all of this data can meaningfully help in development and that other approaches don't replace it well enough.

Lammy 6mo ago

The people who write any individual feature want to be able to prove usage in order to get good performance reviews and promotions. It's so awful that it's become normalized. Back in The Day we had the term “spyware” to refer to any piece of software that phoned home to report user behavior, but now that's just All Software.

rprend 6mo ago

Product work can be counterintuitive. An engineer / PM might think that a design or feature “makes sense”, but you don’t actually know that unless you measure usage.

bastardoperator 6mo ago

You have three features, A, B, and C. They are core features. Two of the features break. How do you prioritize which feature gets fixed first? With telemetry its obvious, without it, you're guessing. Also, gh cli is not about git, its about the github api. In theory the app has its own user agent and of course their LB is tracking all http requests, so not anonymous ever.

stronglikedan 6mo ago

> always feel the need to spy on their users? If it's truly pseudoanonymous then it's hardly spying, just sayin'... Others have answered your actual question better than I could have.

teeray 6mo ago

Because dashboard need to show number go up

high_na_euv 6mo ago

git is terrible from ux perspective >Would Git have been significantly better if it had collected telemetry, or would the data not have just been a distraction? Definitely

sidkshatriya 6mo ago

> ...our team needs visibility into how features are being used in practice. We use this data to prioritize our work and evaluate whether features are meeting real user needs. You should be able to see what features are being used by seeing what server endpoints are being hit and how often. Don't need intrusive telemetry. Yes, it's not perfect. Many features could use the same endpoint. You could totally anonymise this and you could still get a great understanding what features users are using by looking at endpoint stats. Companies need to decide whether they want customer goodwill or very detailed insight into what they are doing. By having invasive telemetry you may have less customers (people leaving for github competitors). Is it worth it ?

ravenstine 6mo ago

"Real" numbers make it easier for them to lie to leadership who then use or ignore those lies to justify decisions they were already going to make in spite of users.

paulddraper 6mo ago

To state the obvious, "good engineering/design practices" will not tell you what features are used or not. > Git has served us well for 20+ years Funny. I think that, but the usual HN narrative is that Git is UX hostile.

caymanjim 6mo ago

The language used in statements like this always annoys me. "We want visibility", ok fine. "We need"...the hell you don't.

crubier 6mo ago

Come on, do you actually think "corporate engineers" care about what you are doing individually? Do you think they look specifically for you and make fun at your individual usage pattern? Do you think it gives them interesting information about your private life? No one cares, there are millions of users, no one is going to look at your data, and even less be able to actually know which person a given user is. We legit just want to know aggregated and objective information about how people use our products so we can make it better for you. "Why do all corporate try to spy on our usage patterns?" Because the ones who don't have a crap product and all died long ago.

raxxorraxor 6mo ago

Especially true because it has also been proved that these telemetry tools often picture a distorted reality. My "telemetry" check is Windows. I does the same, forced it on users and it certainly didn't get better from it. The correlation with quality decline might be independent, but I would wager some shitty manager put up a shitty metric, everything got optimized towards that and leadership has lost the bigger picture because these numbers cannot transport general understanding.

hulitu 6mo ago

> Why we collect telemetry ...our team needs visibility into how features are being used in practice. This was Microsoft's mantra, while bugs were not fixed for months.