7 ms·

I sometimes wonder how profitable it actually is for Microsoft to be so pushy on data. What I mean is: I understand that MS is quite zealous internally about t

by skohan 5y ago

I sometimes wonder how profitable it actually is for Microsoft to be so pushy on data. What I mean is: I understand that MS is quite zealous internally about telemetry, but how effective is this data in terms of actually improving their profitability? From the user side, I am quite sure it is hurting the experience. One example is logins. MS is pretty bad about requiring logins for products which don't really require it in order to complete the user journey - for instance I bought the Halo collection on Steam, and in order to play an offline, single-player game I have to log into an MS account. As a result I haven't played the game in months, because I have to use this MS account that I don't even want to have, and I can't be bothered to reset the password for when there are other games I can play which just start up with no arbitrary hoop to jump through. And what does MS actually get out of it? I guess they can optimize their dark UI patterns to be better at tricking people into setting Edge as their default browser? Being data driven is good, but it seems to me it's not working if the end result is compromising the core user experience.

freeone3000 5y ago
User telemetry replaces what were once focus groups and market research. Without telemetry, Microsoft would be unable to determine what products were used, what features were used, and unable to determine prioritization for backwards compatibility and bugfixes, because it chose to rely on telemetry for basic business operations.
- reaperducer 5y ago
  User telemetry replaces what were once focus groups and market research. These things still exist, and many companies still use focus groups and market research. The company I work for, for example, does this extensively because I'm in the healthcare space and the legal department doesn't want us collecting information about our users without them really really really knowing its happening. Another example is MailChimp. It not only does video interviews with its users to determine how they use the product, it pays you for your time. Microsoft has the money and staff to do this basic research. "Telemetry" is just a synonym for lazy and cheap.
  - skohan 5y ago
    Yes and even besides traditional methods like user research, there are lots of indirect ways to measure the success of a product without resorting to invasive data collection practices. For instance, you can track downloads, page views in documentation, the number of times a certain endpoint is pinged on the server etc. while still respecting the privacy of the user. The practice to track a user's behavior across a wide array of products imo crosses a line, and is not needed to inform good product development. A company could probably sell better to you if they hired a PI to follow your movements, and tapped your phone line, but that doesn't give them the right to do so.
  - hakfoo 5y ago
    I suspect there's also a fear about the accuracy of self-reported data. If you ask a user if he does XYZ, he may say 'no', because of a myriad of reasons (misunderstood the question, embarrassed, not allowed to disclose, etc.), but if you instrument it up and see he clicks it 200 times a day, that's probably the truth. The problem is that analytics in a vacuum are just as misleading as testimonial data, if not more so. "0.6% of users touvh XYZ and only use it for an average of 17 seconds per year" could mean that it's an unloved feature that can be sunset, or that it solves a single niche task very completely and perfectly, and ripping it out will cost those 0.6% of users five hours each to replace.
    - skohan 5y ago
      I think there's also the risk of talking yourself into bad decisions with data as a justification. For instance, if you have a KPI to increase X, and making your UI harder to understand confuses users into doing X more often, from a data-driven perspective you've done the right thing, but you've made the product worse.
- skohan 5y ago
  That might be the case, but it seems like mostly what they've been able to do with it is build products like Teams which nobody asked for and nobody wants, but people have to use because MS has some leverage to get it used in certain organizations. There are exceptions like Office and VSCode, but it seems like for the most part MS products are things which people curse under their breath and don't like to use. Part of that is because of things like how their aggressive approach to telemetry bleeds into the UX, and it just seems a bit off if the tool you have for optimizing your product is actively making it worse.
  - elygre 5y ago
    You may not want Teams, but do not live in the illusion that nobody wants Teams. As proof: I want Teams.
    - skohan 5y ago
      What do you prefer about teams over alternatives? It may be anecdotal, but I have only ever heard people complain about teams.
      - fennecfoxen 5y ago
        Teams video-chat is great, easily better than things like Cisco's offering (and zero-security offerings like Zoom). On the other hand, it's also trying to be a weird combination of Slack and Facebook at the same time (only less searchable than either) and that's pretty terrible.
        GekkePrutser 5y ago
        Really think so? I think the free alternative jitsi is so much better than teams. I use teams for work and its video is pretty slow. Jitsi (I use this for my local makerspace) achieves a much better frame rate. Which makes it more realistic and improves communication. It's also so much lighter on the computer than teams.
      - 5y ago
throwaway_8625 5y ago
> how effective is this data in terms of actually improving their profitability? I work closely with telemetry data coming back from Windows devices. There's surely a line from collecting this telemetry to an answer to your question, but I've never heard anyone in our group talk about the profitability of this data. In fact, it costs a lot of money to process and store such huge data, which is absolutely a concern -- so there's strong belief (which I share) that this cost of business is worth it because it lets us make what I hope are good decisions about where we should focus our efforts. The telemetry that comes back encompasses many dimensions of the user experience, and it helps us do things like figure out when an insider feature is ready to GA; what drivers or devices do poorly and how we can improve the experience; how accurately we can predict battery life; etc. > they can optimize their dark UI patterns to be better at tricking people into setting Edge as their default browser? I don't think whoever is in charge of 'tricking' people into using Edge actually have the kind of telemetry that most people think about with respect to Windows data collection. A lot of people think there's some sinister plot here -- apparently as evidenced by your phrasing. But if you look at the UI for default programs, it's really just a slightly annoying "are you sure?"-kind of message. (I don't use Edge by default, but I just reset my prefs to Edge then back, and I wasn't harassed again, so this isn't a persistent nagging.)
- formerly_proven 5y ago
  > how accurately we can predict battery life You do remember that Windows nowadays only shows "Battery xy %" instead of giving an estimated runtime, right? (Which is, btw., a "holy fuck what braindead idiot thought removing this is a good idea" moment, at first I thought Windows wasn't giving an estimate only in the first few minutes, but after a while I started googling around thinking that the lack of estimated runtime was a bug, but no, someone actually went in and removed it. A very clear and obvious example of actively, explicitly and intentionally user-hostile UX for what sure seems like no other reason than "we could, so we did".)
  - dunnevens 5y ago
    Apple did the same thing. Big Sur will only show you the percentage. Not the estimated runtime. You can find the estimated runtime by looking at the energy tab in Activity Monitor. There are also 3rd party tools that will show it. But by default, it's not readily visible. Apple claims they did it because the runtime estimate wasn't accurate enough. Maybe Microsoft's reason was the same. Though it does seem user hostile to remove it, unless the estimate was substantially off and there's no way to fix it.
mastazi 5y ago
Their telemetry pushed me away from buying Windows licences. I used to pay hundred of dollars for Windows Pro licences for all my machines at home (desktop, self-built HTPC, self-built NAS etc). - After MS went overboard with telemetry, I switched to Linux on everything except my desktop. Recently I set up two new machines (printer server and gaming server), in the past I would have bought two Windows licenses for those, but I didn't, and installed Linux instead. That's more than AU $650 in missed sales for MS.[1] I am aware that I am just one single data point with no statistical significance, but I can imagine I'm not the only person in the world who is bothered by this. [1] One copy of Win 10 pro is currently sold for AU $339. https://www.microsoft.com/p/windows-10-pro/df77x4d43rkt https://www.microsoft.com/p/windows-10-pro/df77x4d43rkt

by skohan 5y ago

freeone3000 5y ago

User telemetry replaces what were once focus groups and market research. Without telemetry, Microsoft would be unable to determine what products were used, what features were used, and unable to determine prioritization for backwards compatibility and bugfixes, because it chose to rely on telemetry for basic business operations.

reaperducer 5y ago
User telemetry replaces what were once focus groups and market research. These things still exist, and many companies still use focus groups and market research. The company I work for, for example, does this extensively because I'm in the healthcare space and the legal department doesn't want us collecting information about our users without them really really really knowing its happening. Another example is MailChimp. It not only does video interviews with its users to determine how they use the product, it pays you for your time. Microsoft has the money and staff to do this basic research. "Telemetry" is just a synonym for lazy and cheap.
- skohan 5y ago
  Yes and even besides traditional methods like user research, there are lots of indirect ways to measure the success of a product without resorting to invasive data collection practices. For instance, you can track downloads, page views in documentation, the number of times a certain endpoint is pinged on the server etc. while still respecting the privacy of the user. The practice to track a user's behavior across a wide array of products imo crosses a line, and is not needed to inform good product development. A company could probably sell better to you if they hired a PI to follow your movements, and tapped your phone line, but that doesn't give them the right to do so.
- hakfoo 5y ago
  I suspect there's also a fear about the accuracy of self-reported data. If you ask a user if he does XYZ, he may say 'no', because of a myriad of reasons (misunderstood the question, embarrassed, not allowed to disclose, etc.), but if you instrument it up and see he clicks it 200 times a day, that's probably the truth. The problem is that analytics in a vacuum are just as misleading as testimonial data, if not more so. "0.6% of users touvh XYZ and only use it for an average of 17 seconds per year" could mean that it's an unloved feature that can be sunset, or that it solves a single niche task very completely and perfectly, and ripping it out will cost those 0.6% of users five hours each to replace.
  - skohan 5y ago
    I think there's also the risk of talking yourself into bad decisions with data as a justification. For instance, if you have a KPI to increase X, and making your UI harder to understand confuses users into doing X more often, from a data-driven perspective you've done the right thing, but you've made the product worse.
skohan 5y ago
That might be the case, but it seems like mostly what they've been able to do with it is build products like Teams which nobody asked for and nobody wants, but people have to use because MS has some leverage to get it used in certain organizations. There are exceptions like Office and VSCode, but it seems like for the most part MS products are things which people curse under their breath and don't like to use. Part of that is because of things like how their aggressive approach to telemetry bleeds into the UX, and it just seems a bit off if the tool you have for optimizing your product is actively making it worse.
- elygre 5y ago
  You may not want Teams, but do not live in the illusion that nobody wants Teams. As proof: I want Teams.
  - skohan 5y ago
    What do you prefer about teams over alternatives? It may be anecdotal, but I have only ever heard people complain about teams.
    - fennecfoxen 5y ago
      Teams video-chat is great, easily better than things like Cisco's offering (and zero-security offerings like Zoom). On the other hand, it's also trying to be a weird combination of Slack and Facebook at the same time (only less searchable than either) and that's pretty terrible.
      - GekkePrutser 5y ago
        Really think so? I think the free alternative jitsi is so much better than teams. I use teams for work and its video is pretty slow. Jitsi (I use this for my local makerspace) achieves a much better frame rate. Which makes it more realistic and improves communication. It's also so much lighter on the computer than teams.
    - 5y ago

throwaway_8625 5y ago

> how effective is this data in terms of actually improving their profitability? I work closely with telemetry data coming back from Windows devices. There's surely a line from collecting this telemetry to an answer to your question, but I've never heard anyone in our group talk about the profitability of this data. In fact, it costs a lot of money to process and store such huge data, which is absolutely a concern -- so there's strong belief (which I share) that this cost of business is worth it because it lets us make what I hope are good decisions about where we should focus our efforts. The telemetry that comes back encompasses many dimensions of the user experience, and it helps us do things like figure out when an insider feature is ready to GA; what drivers or devices do poorly and how we can improve the experience; how accurately we can predict battery life; etc. > they can optimize their dark UI patterns to be better at tricking people into setting Edge as their default browser? I don't think whoever is in charge of 'tricking' people into using Edge actually have the kind of telemetry that most people think about with respect to Windows data collection. A lot of people think there's some sinister plot here -- apparently as evidenced by your phrasing. But if you look at the UI for default programs, it's really just a slightly annoying "are you sure?"-kind of message. (I don't use Edge by default, but I just reset my prefs to Edge then back, and I wasn't harassed again, so this isn't a persistent nagging.)

formerly_proven 5y ago
> how accurately we can predict battery life You do remember that Windows nowadays only shows "Battery xy %" instead of giving an estimated runtime, right? (Which is, btw., a "holy fuck what braindead idiot thought removing this is a good idea" moment, at first I thought Windows wasn't giving an estimate only in the first few minutes, but after a while I started googling around thinking that the lack of estimated runtime was a bug, but no, someone actually went in and removed it. A very clear and obvious example of actively, explicitly and intentionally user-hostile UX for what sure seems like no other reason than "we could, so we did".)
- dunnevens 5y ago
  Apple did the same thing. Big Sur will only show you the percentage. Not the estimated runtime. You can find the estimated runtime by looking at the energy tab in Activity Monitor. There are also 3rd party tools that will show it. But by default, it's not readily visible. Apple claims they did it because the runtime estimate wasn't accurate enough. Maybe Microsoft's reason was the same. Though it does seem user hostile to remove it, unless the estimate was substantially off and there's no way to fix it.

mastazi 5y ago

Their telemetry pushed me away from buying Windows licences. I used to pay hundred of dollars for Windows Pro licences for all my machines at home (desktop, self-built HTPC, self-built NAS etc). - After MS went overboard with telemetry, I switched to Linux on everything except my desktop. Recently I set up two new machines (printer server and gaming server), in the past I would have bought two Windows licenses for those, but I didn't, and installed Linux instead. That's more than AU $650 in missed sales for MS.[1] I am aware that I am just one single data point with no statistical significance, but I can imagine I'm not the only person in the world who is bothered by this. [1] One copy of Win 10 pro is currently sold for AU $339. https://www.microsoft.com/p/windows-10-pro/df77x4d43rkt https://www.microsoft.com/p/windows-10-pro/df77x4d43rkt