7 ms·
The hysteria about this is absurd. To make product decisions you need to measure actual usage patterns. To do that you need telemetry.
by efficax 5y ago
The hysteria about this is absurd. To make product decisions you need to measure actual usage patterns. To do that you need telemetry.
- readflaggedcomm 5y agoNo, you don't. You could talk to people, instead.
- tester756 5y agothen you'll have opinion of vocal minority
- readflaggedcomm 5y agoYes. Hopefully developers and power users, if they're chosen correctly and given the proper incentives (like having their ideas implemented).
- deleted 5y ago[deleted]
- efficax 5y agoSurveys and user interviews are also useful, but nothing beats having the data. And usually we use the telemetry data for the app to craft the surveys and interviews. Did you stick a cool feature on a dialog but nobody ever uses it? Telemetry tells us that's something to look into. The thing is that this is very easy to do with web apps, since you get a ton of telemetry for free through API requests. A desktop app is s black box without telemetry. Just release and pray
- atatatat 5y agoA 10-30 min walkthrough+interview of how 5 users use the software will elicit by who and why the dialog isn't being used.
- realusername 5y agoWhen was the last time you reached to talk to some project maintainers of your favorite open-source software? You have the answer to your question then.
- mixmastamyk 5y agoI give opinions in tickets all the time. This is floss were talking about, so budgets are a factor.
- TingPing 5y agoThat also biases data to a very narrow technical user.
- robmsmt 5y agoI think we as a community need to come up with officially recognized names for such activities to lessen the cognitive burden on installing such software...
- simias 5y agoTelemetry can be useful but it's hardly the only way, and it doesn't have to be opt-out. People made "product decisions" before the internet.
- masklinn 5y ago> People made "product decisions" before the internet. People still pull numbers and assertions out their asses all the time. That doesn't mean it is, or was, good.
- simias 5y agoTelemetry doesn't guarantee that people won't pull numbers out of their asses. There are lies, damned lies, and statistics. Designing good telemetry is hard, interpreting the data well is harder (and it's tempting to cherry-pick post facto to justify your decisions). Are users not using functionality X because it's useless or because there's some other issue preventing its use? Or maybe users just don't know that it exists? Crash reports are more of an unmitigated good IMO however, since they can be used to pinpoint the source of crashes and identify regressions.
- sharklazer 5y ago> Designing good telemetry is hard, interpreting the data well is harder The biggest problem facing any use of data is quality of the collected data. For example, if you wanted to train a model but all you had were examples of a particular class... well, you excluded yourself from being able to predict any other class from the git-go. Data collection itself require a deep understanding of statistics and probability. Otherwise you’re just shooting yourself in the foot. But let’s be honest, machines don’t “learn” things, people do. Machines have no teleology they determine for themselves. People are both the problem and solution. Anyone who thinks big data and ML will be able to generate the answers are deluded and have already turned off or lost completely their critical thinking skills. Data collection in the form of telemetry is only a mechanism for finding solutions and an incredibly INVASIVE one at that.
- zamadatix 5y ago
- pfortuny 5y agoSoftware does not need to move at the speed of light, much less something like audacity which has a huge user base and hopes of staying useful for decades.
- masklinn 5y agoTelemetry has nothing to do with "moving at the speed of light", it has to do with discovering actual usage patterns. There's no requirement to move fast on telemetry. In fact you probably want to do the exact opposite, because you want large representative aggregates.
- majewsky 5y agoIf you want to discover actual usage patterns, there is a much more efficient way: Sit down with 10 users and observe them while doing their everyday work or while performing a predetermined set of tasks. This is even easier nowadays thanks to screen sharing, so you can even put a focus on getting a geographically and demographically diverse sample of users. From personal experience, I've always had more insights into actual usage patterns from sitting next to a random user for 30 minutes than I could have had from heaps and heaps of telemetry.
- atoav 5y agoThis is a strawman. The heat about this is not about whether a project needs telemetry or not. It is about how Audacity chose their service partners (without thinking about it) and it is about how they communicated it (a PR request without prior discussion). That being said – even if they had communicated transparently and discussed openly – if their goal was to get good telemetry maybe self hosting allows more users to trust you, and therefore you receive more telemetry? Because the overlap within the venn diagram of users who trust Audacity and users who trust Google/Yandex will always be considerably smaller than the overlap of people who trust Audacity and people who trust Audacity.
- jraph 5y agoWhat you call hysteria was caused in big part by the fact they were going to use Google and Yandex for this.
- jjulius 5y agoMy pessimistic take is that Audacity has been one of the most popular and well-loved FOSS projects for 20+ years, without telemetry. That's 22 years of amazing software that didn't have to track users in order to be so good. It's hard not to see the recent sale of Audacity and the introduction of telemetry as the beginning of the end of an otherwise stellar piece of software.
- TingPing 5y agoAudacity is very rough and buggy. Its popularity came from being free and featureful, not polish or focus.
- swiley 5y agoIMO: the UI is pretty good the way it is, I was even thinking that the last time I used it (last fall, I don't use it all the time which shows just how intuitive it is to people who don't use it often.)
- indymike 5y agoNot really. There's lots of other ways to learn about how your product is used, and without them telemetry will lead to some bad decisions. I've personally had to clean up the mess when telemetry driven decisions go sideways so many times: * No one seems to use American express, so let's remove it from payment options. Turns out, the site had a bug that made sure 100% of AMEX transactions would fail. By fixing, we picked up 6% in sales, and lost a UX associate who was (and probably still is) arguing to drop AMEX because analytics. * Since very few people actually click the apply button on the job board, let's remove the apply feature. The idea was we could send candidate profiles to employers based on views instead of apply clicks. Had to revert this one. * Same company: let's take job posts out of our mobile job search app because most interactions end on a job post (and a click on the apply button). Best ticket the next day, "I can't find any jobs in this job search app?" * No one uses the UI customizer (dashboard colors, fonts, and preferences). 100% of the tickets the next day were from people who were PISSED BEYOND BELIEF that we removed their favorite feater. Turns out the customizer basically was invisible to our app analytics. Had to revert this and re-do two sprints of work because the other new features wouldn't work with the customizer. In 100% of the cases, talking to actual users would have been beyond wise before using analytics data to make decisions.
- jonpurdy 5y agoTotally agreed. Just because a feature is used by 2% of users doesn't mean it should be killed. That feature (plus others) could be critical to the workflow of those users. It seems the telemetry-driven product development in the past decade has made significantly worse apps than in previous decades. There are definitely a higher quantity of apps on a wider variety of platforms, but it seems harder to find apps that are powerful enough to allow a wide variety of users to do all the things they need.
- mixmastamyk 5y agoThe long tail of features is where a system achieves its status as "polished." Is this why software never gets polished any more? (And no, all features don't need to be visible at once.)
- 5y ago
- HelloNurse 5y agoTelemetry in the sense of automating the collection of useful data might be worth the effort, but there's no need to share that data with Google or Yandex. Considering sharing user data with some cloud service is reckless enough to cast a serious shadow on a company's intent and morality, even without an actual GDPR breach.
- sneak 5y ago1) It's not hysteria and calling it such is disrespectful and dismissive. 2) No you don't.
- nimih 5y agoDid no one make product decisions when building software prior to the mid 00s?