4 ms·
> Telemetry is godsend for developers, since it lets them track down actual product usage This is is actually a terrible trap, as product usage tells you next
by marginalia_nu 4y ago
> Telemetry is godsend for developers, since it lets them track down actual product usage
This is is actually a terrible trap, as product usage tells you next to nothing about which features are useful. I suspect it' a big part of why a lot of software has turned to shit last decade or so.
Look at it this way: Your average fire extinguisher sits mounted on a wall its entire life and is never actually used. You still rather have it and not need it than need it and not have it.
- falcolas 4y ago> This is is actually a terrible trap Agreed. > You still rather have it and not need it than need it and not have it. And there's the trap. Data is a liability, but until a company gets burned for losing it, they won't do anything about it. Well, being real, even if they do get burned it's just the "cost of doing business".
- lcnPylGDnU4H9OF 4y agoI get what you're saying and I agree but I also think you misinterpreted the parent comment. > You still rather have it and not need it than need it and not have it. Given the context of the fire extinguisher, "it" in this case is a useful (possibly critical), not-commonly-needed feature, rather than a collection of data.
- ajross 4y ago> This is is actually a terrible trap, as product usage tells you next to nothing about which features are useful. I suspect it' a big part of why a lot of software has turned to shit last decade or so. This seems exactly backwards to me. No one who has ever managed a deployed product with good failure telemetry would say that it tells you "next to nothing". Telemetry detects failures that mere testing simply never can and never will. And no one who lived through earlier ages of consumer software would ever say that it's "turned to shit". Software is getting better. It's getting better steadily and inexorably, and it's been doing that for half a century. It may not be doing what you want, but it's doing it with higher quality than people in the 90's or 00's would have ever dreamed. When was the last time[1] you saw a major consumer application (the TikTok or Instagram client, or your web browser, or VSCode) crash hard and fail on you? This was a daily (or worse) event for most of software history! And certainly robust telemetry practice is one of the big drivers of this quality revolution we've seen. [1] Heh, cue the peanut gallery of everyone wanting to post crash reports. That's on me for poor framing, I guess. But seriously, everyone: Both anecdotally and via actual studies, consumer-visible software is getting much better.
- marginalia_nu 4y ago> When was the last time you saw a major consumer application (the TikTok or Instagram client, or your web browser, or VSCode) crash hard and fail on you? This was a daily (or worse) event for most of software history! Huh. I've used software since the mid 1990s, and I can't recall using software that crashed daily, let alone frequently. Which is honestly a bit of a miracle, since a lot of software back then was shipped without the ability to patch it at all.
- pseudalopex 4y ago> Telemetry detects failures that mere testing simply never can and never will. Product usage and failure reports are different. It wasn't called telemetry when apps just asked to send failure reports. > When was the last time you saw a major consumer application (the TikTok or Instagram client, or your web browser, or VSCode) crash hard and fail on you? An hour ago or so.
- falcolas 4y ago> When was the last time you saw a major consumer application (the TikTok or Instagram client, or your web browser, or VSCode) crash hard and fail on you? Yesterday. The kindle app. It has a habit of soft locking on a book's cover. TikTok's failure mode tends to be more of a "refuses to play a video" kind. Especially if you're scrolling through videos and one does not exist. It doesn't like that. Instagram soft reboots pretty often (you can tell because you're back to the home feed, as opposed to where you left off), especially after being suspended. I have to reboot my Mac weekly, both for updates and to keep it from misbehaving oddly. It's one thing our development culture actually fosters these days - let it fail and restart in a known state; as opposed to doing what you can to keep the software alive through non-fatal errors.
- Macha 4y agoMacOS, an hour ago. Before that, Fastmail on Android, this morning. Slack on iOS gets stuck a few times a week.
- dogleash 4y agoWhen was the last time[1] you saw a major consumer application [] crash hard and fail on you? This was a daily (or worse) event for most of software history! The apps I use are now riddled with hang bugs rather than crash bugs. Huzza.
- fragmede 4y ago"trap". If feature A has 100% of users using it and feature B has 0%, what should I, as the app developer spend my time working on?
- marginalia_nu 4y agoIf your application keeps opening popups all the time, I will use the close popup button very often. Doesn't mean I enjoy closing your popups or want to have to close them. The only reason I'm using the feature is to get around an annoyance in your hypothetical application. Frequent use of a feature is just as much a signal that the feature is an obstacle as it is a signal that the feature is beneficial.
- robalni 4y agoExactly. If I use feature X very often, that can be because: * Feature X is good and I like to use it. * Or feature X is broken so I have to try multiple times. * Or feature Y is broken so I have to try using another feature instead. Like, most of the times I open a settings window it's not because I like settings but because the program is not good enough as it is. Or if I follow a lot of links on your web page, it might not be because I like your web page but because I'm unable to find what I'm looking for.
- kahrl 4y agoI do not understand this. Because this mythical moronic figure everyone in this thread is talking about doesn't account for confounding variables, telemetry is just bad generally?
- fragmede 4y agoObviously. My point is that some information as to how users are using your application (so it can be made better for them) is better than no signal.
- paulclinger 4y agoRestore backups may be a rarely used feature (where applicable), but it's not a good reason not to have it (working properly).
- majormajor 4y agoWhy are we just focused on the "stupid" developer here - the sort who thinks your later "wow the 'close popup' button is so popular" example is a good reason to add more popups? Do we think the people making calls like that would suddenly have great ideas in the absence of data? Meanwhile, there are plenty of folks out there who can reason about data in less blindly idiotic ways.
- marginalia_nu 4y agoLet's take a different example. Let's say you have a search box. Option A generates 10 times more searches than option B. This may mean the search box is 10 times more useful than in the B case, or the users require 10 times as many searches to find what they want. Looking at the data, you really can't tell. Even without the data, you can dogfood your application and fairly quickly tell whether the feature is good or bad. I'm not saying anyone is stupid. Naive, perhaps, but not stupid. The problem is that interpreting data is difficult. Incredibly difficult. Scientists, who construct experiments and interpret experimental data for a living, with a decade of education behind their back, get this wrong all the time.
- cardanome 4y ago> Do we think the people making calls like that would suddenly have great ideas in the absence of data? Personally, I would be so bold to suggest that yes the availability of data for people ill equipped to interpret it can lead to WORSE decisions than the absence of data. That is because statistics and the general skill to correctly interpret data is actively counterintuitive. Untrained people will be more likely to generate wrong conclusions than right conclusions. So, yes data in the wrong hands can and will be harmful.
- jrockway 4y agoI'm not sure software is getting worse. It does more, and there is more. A lot of complaints on HN are about businesses they don't like, or technology they thought should never have been invented. It's unfortunate that tracking you across the web is hugely profitable, but the reason that business works isn't due to software malfunction. The software is working great. Certainly, there is a lot of software out there with poor polish. My pet peeve is that devices like multimeters that used to have segmented VFDs, so had a certain design aesthetic by default. Clean, straightforward, readable. Now those have moved to low resolution LCDs, so you notice that the font they picked is terrible, and the font rendering engine they picked is even worse. It just looks bad, in exchange for more flexibility and features. But, none of this is indicative that the software itself is getting worse. In the 80s, trivial software bugs were literally killing people: https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25 In terms of radiation therapy machines, the underlying business hasn't changed much; people with tumors want them to be killed, and the difference between now and 40 years ago is that the machine doesn't kill them because of integer overflows. (One interesting note about the Therac-25 is that it reused software for previous models that had hardware interlocks that masked the software defects. They didn't have any telemetry on when the hardware interlocks triggered, so they didn't know that the hardware interlocks were masking bugs in the software when it came time to remove them. If they had the data, they might have kept the hardware interlocks, or fixed the bugs before 6 people died. So maybe lack of telemetry is a greater ethics issue than including it!)
- marginalia_nu 4y agoSoftware is still killing people despite telemetry. Look at Tesla's autopilot.
- jrockway 4y agoI don't know if that's an interesting data point. Humans are terrible at driving cars. AIs are terrible at driving cars. I'm not sure that Tesla is obviously doing a sub-par job here, as it's a challenging problem space. The main effect of the telemetry in self-driving cars is that we can all Monday morning quarterback the exact defect that caused a particular crash. With human drivers, we don't have as much data to get to the root cause. If someone falls asleep at the wheel and drives into a truck, was it their medication? Their sleep schedule? An emotional conversation they just had? The music that the radio station chose to play? A combination of all of those factors? What actions do we take to prevent it from happening again?