4 ms·

User telemetry replaces what were once focus groups and market research. Without telemetry, Microsoft would be unable to determine what products were used, what

User telemetry replaces what were once focus groups and market research. Without telemetry, Microsoft would be unable to determine what products were used, what features were used, and unable to determine prioritization for backwards compatibility and bugfixes, because it chose to rely on telemetry for basic business operations.

reaperducer 5y ago
User telemetry replaces what were once focus groups and market research. These things still exist, and many companies still use focus groups and market research. The company I work for, for example, does this extensively because I'm in the healthcare space and the legal department doesn't want us collecting information about our users without them really really really knowing its happening. Another example is MailChimp. It not only does video interviews with its users to determine how they use the product, it pays you for your time. Microsoft has the money and staff to do this basic research. "Telemetry" is just a synonym for lazy and cheap.
- skohan 5y ago
  Yes and even besides traditional methods like user research, there are lots of indirect ways to measure the success of a product without resorting to invasive data collection practices. For instance, you can track downloads, page views in documentation, the number of times a certain endpoint is pinged on the server etc. while still respecting the privacy of the user. The practice to track a user's behavior across a wide array of products imo crosses a line, and is not needed to inform good product development. A company could probably sell better to you if they hired a PI to follow your movements, and tapped your phone line, but that doesn't give them the right to do so.
- hakfoo 5y ago
  I suspect there's also a fear about the accuracy of self-reported data. If you ask a user if he does XYZ, he may say 'no', because of a myriad of reasons (misunderstood the question, embarrassed, not allowed to disclose, etc.), but if you instrument it up and see he clicks it 200 times a day, that's probably the truth. The problem is that analytics in a vacuum are just as misleading as testimonial data, if not more so. "0.6% of users touvh XYZ and only use it for an average of 17 seconds per year" could mean that it's an unloved feature that can be sunset, or that it solves a single niche task very completely and perfectly, and ripping it out will cost those 0.6% of users five hours each to replace.
  - skohan 5y ago
    I think there's also the risk of talking yourself into bad decisions with data as a justification. For instance, if you have a KPI to increase X, and making your UI harder to understand confuses users into doing X more often, from a data-driven perspective you've done the right thing, but you've made the product worse.
skohan 5y ago
That might be the case, but it seems like mostly what they've been able to do with it is build products like Teams which nobody asked for and nobody wants, but people have to use because MS has some leverage to get it used in certain organizations. There are exceptions like Office and VSCode, but it seems like for the most part MS products are things which people curse under their breath and don't like to use. Part of that is because of things like how their aggressive approach to telemetry bleeds into the UX, and it just seems a bit off if the tool you have for optimizing your product is actively making it worse.
- elygre 5y ago
  You may not want Teams, but do not live in the illusion that nobody wants Teams. As proof: I want Teams.
  - skohan 5y ago
    What do you prefer about teams over alternatives? It may be anecdotal, but I have only ever heard people complain about teams.
    - fennecfoxen 5y ago
      Teams video-chat is great, easily better than things like Cisco's offering (and zero-security offerings like Zoom). On the other hand, it's also trying to be a weird combination of Slack and Facebook at the same time (only less searchable than either) and that's pretty terrible.
      - GekkePrutser 5y ago
        Really think so? I think the free alternative jitsi is so much better than teams. I use teams for work and its video is pretty slow. Jitsi (I use this for my local makerspace) achieves a much better frame rate. Which makes it more realistic and improves communication. It's also so much lighter on the computer than teams.
    - 5y ago

reaperducer 5y ago

User telemetry replaces what were once focus groups and market research. These things still exist, and many companies still use focus groups and market research. The company I work for, for example, does this extensively because I'm in the healthcare space and the legal department doesn't want us collecting information about our users without them really really really knowing its happening. Another example is MailChimp. It not only does video interviews with its users to determine how they use the product, it pays you for your time. Microsoft has the money and staff to do this basic research. "Telemetry" is just a synonym for lazy and cheap.

skohan 5y ago
Yes and even besides traditional methods like user research, there are lots of indirect ways to measure the success of a product without resorting to invasive data collection practices. For instance, you can track downloads, page views in documentation, the number of times a certain endpoint is pinged on the server etc. while still respecting the privacy of the user. The practice to track a user's behavior across a wide array of products imo crosses a line, and is not needed to inform good product development. A company could probably sell better to you if they hired a PI to follow your movements, and tapped your phone line, but that doesn't give them the right to do so.
hakfoo 5y ago
I suspect there's also a fear about the accuracy of self-reported data. If you ask a user if he does XYZ, he may say 'no', because of a myriad of reasons (misunderstood the question, embarrassed, not allowed to disclose, etc.), but if you instrument it up and see he clicks it 200 times a day, that's probably the truth. The problem is that analytics in a vacuum are just as misleading as testimonial data, if not more so. "0.6% of users touvh XYZ and only use it for an average of 17 seconds per year" could mean that it's an unloved feature that can be sunset, or that it solves a single niche task very completely and perfectly, and ripping it out will cost those 0.6% of users five hours each to replace.
- skohan 5y ago
  I think there's also the risk of talking yourself into bad decisions with data as a justification. For instance, if you have a KPI to increase X, and making your UI harder to understand confuses users into doing X more often, from a data-driven perspective you've done the right thing, but you've made the product worse.

skohan 5y ago

That might be the case, but it seems like mostly what they've been able to do with it is build products like Teams which nobody asked for and nobody wants, but people have to use because MS has some leverage to get it used in certain organizations. There are exceptions like Office and VSCode, but it seems like for the most part MS products are things which people curse under their breath and don't like to use. Part of that is because of things like how their aggressive approach to telemetry bleeds into the UX, and it just seems a bit off if the tool you have for optimizing your product is actively making it worse.

elygre 5y ago
You may not want Teams, but do not live in the illusion that nobody wants Teams. As proof: I want Teams.
- skohan 5y ago
  What do you prefer about teams over alternatives? It may be anecdotal, but I have only ever heard people complain about teams.
  - fennecfoxen 5y ago
    Teams video-chat is great, easily better than things like Cisco's offering (and zero-security offerings like Zoom). On the other hand, it's also trying to be a weird combination of Slack and Facebook at the same time (only less searchable than either) and that's pretty terrible.
    - GekkePrutser 5y ago
      Really think so? I think the free alternative jitsi is so much better than teams. I use teams for work and its video is pretty slow. Jitsi (I use this for my local makerspace) achieves a much better frame rate. Which makes it more realistic and improves communication. It's also so much lighter on the computer than teams.
  - 5y ago