5 ms·

You're only flying blind if you make decisions not looking and thinking. Analytics isn't the only way to figure out "what your users actually care about", you c

You're only flying blind if you make decisions not looking and thinking. Analytics isn't the only way to figure out "what your users actually care about", you can also try the old school way, commonly referred to as "Talking with people", then after taking notes, you think about it, maybe discuss with others. Don't take what people say at face value, but think about it together with your knowledge and experience, and you'll make even better product decisions than the people who are only making "data driven decisions" all the time.

Sytten 7mo ago
We do both and they yield different learnings. They are complementary. We also have an issue tracking board with upvotes. I would say to your point that you can't improve what you don't measure.
- bfivyvysj 7mo ago
  I would say to your point that you can't not spy on me while also spying on me. Maybe just don't?
  - lukevp 7mo ago
    If I was running a physical business and I wrote down each person’s name and credit card number and the exact time and order they placed, that would be pretty invasive and “spying”. If I write down how many units I sold of each item per day, and the volume of transactions by credit card vs cash, it’s anonymized and I don’t think this would generally be considered “spying”, just normal business metrics. How’s the latter much different than anonymized product analytics?
    - lamasery 7mo ago
      Watching me use my computer in my house or office is spying. Aggregating request statistics server-side unless you're only generating those requests to spy on what I'm doing on my computer is more like the not-spying you're talking about.
      - vlovich123 7mo ago
        The logical conclusion is you’re asking for no local products and everything to run server side. It’s kind of a ridiculous position that doesn’t change the spying being done other than it’s on the other side of a browser.
        lamasery 7mo ago
        I accounted for this in my post. Obviously if you’re making requests just so you can spy, that’s spying.
        vlovich123 7mo ago
        No you didn’t. If I build you a web video editor, is that because I want to spy on you or because I want to make deployment easier and reduce install friction? You’re making a distinction that puts you in the privileged judge position of evaluating if a service is making requests just so you can spy vs what the app author’s might believe is a critical design feature in how they want the product to operate.
      - embedding-shape 7mo ago
        > Watching me use my computer in my house or office is spying. I agree, but once you cross the borders out to the internet, I'd say you need to stop seeing that as "Me sitting at my computer at home", because you're actually "on someone else's property" at that point essentially. And I say this as someone who care greatly about preserving personal privacy.
        lamasery 7mo ago
        I deeply hate that this attitude took over even among “hackers”. Watching people move their mouse and click stuff on “your webpage” is fucking spying. It’s in my browser. On my machine. Not running on your hardware. Tracking what I do on my own computer doesn’t stop being spying because the program I’m doing stuff in can make network requests. WTF.
    - account42 6mo ago
      You can write down your business transactions, sure. Telemetry would be more akin to looking through your customers windows each day and noting down how often they use the tools you sold them. Not cool.
johnfn 7mo ago
Sure, you can spend the weeks to months of expensive and time consuming work it takes to get a fuzzy, half accurate and biased picture of what your users workflows look like through user interviews and surveys. Or you can look at the analytics, which tell you everything you need to know immediately, always up to date, with perfect precision. Sometimes HN drives me crazy. From this thread you’d think telemetry is screen recording your every move and facial expression and sending it to the government. I’ve worked at places that had telemetry and it’s more along the granularity of “how many people clicked the secondary button on the third tab?” This is a far cry from “spying on users”.
- embedding-shape 7mo ago
  > Sure, you can spend the weeks to months of expensive and time consuming work it takes to get a fuzzy, half accurate and biased picture of what your users workflows look like through user interviews and surveys. Or you can look at the analytics, which tell you everything you need to know immediately, always up to date, with perfect precision. Yes, admittedly, the first time you do these things, they're difficult, hard and you have lots to learn. But as you do this more often, build up a knowledge base and learn about your users, you'll gain knowledge and experience you can reuse, and it'll no longer take you weeks or months of investigations to answer "Where should this button go?", you'll base it on what you already know.
  - acedTrex 7mo ago
    So if you don't want to spend the time doing that, or as is more accurate in corporate settings, the general turnover of the team is high enough that no one is around long enough to build that deep foundational product knowledge, and to be frank most people do not care enough. This is why telemetry happens, its faster, easier and more resilient to organizational turmoil.
    - embedding-shape 7mo ago
      > This is why telemetry happens, its faster, easier and more resilient to organizational turmoil. I don't disagree with that, I was mainly talking about trying to deliver an experience that makes sense, is intuitive and as helpful and useful as possible, even in exchange for it taking longer time. Of course this isn't applicable in every case, sometimes you need different tradeoffs, that's OK too. But that some favor quality over shorter implementation time shouldn't drive people crazy, it's just making different tradeoffs.
Arch485 7mo ago
Exactly - purely "data driven" decisions are how we end up with ads really close to (or overlapping with) some button you want to press, because the data says that increase click-through rate! But it's actually a user-hostile feature that everyone hates.
- PhoenixFlame101 7mo ago
  But collecting data and looking for insights doesn't mean you mechanically optimize features, especially user-hostile ones? This is just as, if not more, likely to happen when basing your decisions on what people say they want over what they actually do.
  - defmacr0 7mo ago
    If we were perfectly rational, then yeah, more data should never lead to worse decisions. However, it's easy to fall into the trap where being data-driven makes you only work on those things that you know how to measure.
    - account42 6mo ago
      Or worse, it makes you go look for data backing up the assumptions you already made.
  - account42 6mo ago
    It's easier to not care about users if they are just numbers and you can tell yourself to ignore what they actually say.
- mgfist 7mo ago
  The reason that feature gets implemented is not because the devs think users will like it ... they know users don't want it, but it drives revenue and pays salaries.
staticassertion 7mo ago
It's sort of hilarious to compare "talking to people" with analytics. I'm not defending Github here, but you can't possibly think that "talking to 1M customers" is viable.
- almostjazz 7mo ago
  You could survey a representative sample
  - staticassertion 7mo ago
    Not really. (a) People hate responding to surveys and hate emails, you're more likely to lose users than to get data (b) there's no way you're surveying people's in a way that gets you information like "time spent on a page" or "time between commits" or whatever. This is just nonsense tbh. Surveys and customer outreach solve completely different problems from analytics.
    - almostjazz 7mo ago
      I agree you can't practically get the same information as you could with telemetry. Survey data is still real data that can be used for "analytics". Some people also hate telemetry. It feels invasive. I have a guess about what direction the percentage of consumers who hate telemetry is moving toward.
    - codedokode 7mo ago
      You can hire people to test your product and provide analytics. But not try to siphon the data for free.
      - staticassertion 7mo ago
        I'm not taking a side on whether a product should add telemetry. I'm rejecting the absurd notion that these suggestions are at all giving the same information.
dpark 7mo ago
> Don't take what people say at face value, but think about it together with your knowledge and experience While you’re comparing different information sources, you might even want to consider telemetry, too.
SchemaLoad 7mo ago
What people say, and what people do are different things. Especially when the people who agree to talk to you aren't representative of the whole user base.
- account42 6mo ago
  And what people do and the reasons they do it are even more different.

Sytten 7mo ago

We do both and they yield different learnings. They are complementary. We also have an issue tracking board with upvotes. I would say to your point that you can't improve what you don't measure.

bfivyvysj 7mo ago
I would say to your point that you can't not spy on me while also spying on me. Maybe just don't?
- lukevp 7mo ago
  If I was running a physical business and I wrote down each person’s name and credit card number and the exact time and order they placed, that would be pretty invasive and “spying”. If I write down how many units I sold of each item per day, and the volume of transactions by credit card vs cash, it’s anonymized and I don’t think this would generally be considered “spying”, just normal business metrics. How’s the latter much different than anonymized product analytics?
  - lamasery 7mo ago
    Watching me use my computer in my house or office is spying. Aggregating request statistics server-side unless you're only generating those requests to spy on what I'm doing on my computer is more like the not-spying you're talking about.
    - vlovich123 7mo ago
      The logical conclusion is you’re asking for no local products and everything to run server side. It’s kind of a ridiculous position that doesn’t change the spying being done other than it’s on the other side of a browser.
      - lamasery 7mo ago
        I accounted for this in my post. Obviously if you’re making requests just so you can spy, that’s spying.
        vlovich123 7mo ago
        No you didn’t. If I build you a web video editor, is that because I want to spy on you or because I want to make deployment easier and reduce install friction? You’re making a distinction that puts you in the privileged judge position of evaluating if a service is making requests just so you can spy vs what the app author’s might believe is a critical design feature in how they want the product to operate.
    - embedding-shape 7mo ago
      > Watching me use my computer in my house or office is spying. I agree, but once you cross the borders out to the internet, I'd say you need to stop seeing that as "Me sitting at my computer at home", because you're actually "on someone else's property" at that point essentially. And I say this as someone who care greatly about preserving personal privacy.
      - lamasery 7mo ago
        I deeply hate that this attitude took over even among “hackers”. Watching people move their mouse and click stuff on “your webpage” is fucking spying. It’s in my browser. On my machine. Not running on your hardware. Tracking what I do on my own computer doesn’t stop being spying because the program I’m doing stuff in can make network requests. WTF.
  - account42 6mo ago
    You can write down your business transactions, sure. Telemetry would be more akin to looking through your customers windows each day and noting down how often they use the tools you sold them. Not cool.

johnfn 7mo ago

Sure, you can spend the weeks to months of expensive and time consuming work it takes to get a fuzzy, half accurate and biased picture of what your users workflows look like through user interviews and surveys. Or you can look at the analytics, which tell you everything you need to know immediately, always up to date, with perfect precision. Sometimes HN drives me crazy. From this thread you’d think telemetry is screen recording your every move and facial expression and sending it to the government. I’ve worked at places that had telemetry and it’s more along the granularity of “how many people clicked the secondary button on the third tab?” This is a far cry from “spying on users”.

embedding-shape 7mo ago
> Sure, you can spend the weeks to months of expensive and time consuming work it takes to get a fuzzy, half accurate and biased picture of what your users workflows look like through user interviews and surveys. Or you can look at the analytics, which tell you everything you need to know immediately, always up to date, with perfect precision. Yes, admittedly, the first time you do these things, they're difficult, hard and you have lots to learn. But as you do this more often, build up a knowledge base and learn about your users, you'll gain knowledge and experience you can reuse, and it'll no longer take you weeks or months of investigations to answer "Where should this button go?", you'll base it on what you already know.
- acedTrex 7mo ago
  So if you don't want to spend the time doing that, or as is more accurate in corporate settings, the general turnover of the team is high enough that no one is around long enough to build that deep foundational product knowledge, and to be frank most people do not care enough. This is why telemetry happens, its faster, easier and more resilient to organizational turmoil.
  - embedding-shape 7mo ago
    > This is why telemetry happens, its faster, easier and more resilient to organizational turmoil. I don't disagree with that, I was mainly talking about trying to deliver an experience that makes sense, is intuitive and as helpful and useful as possible, even in exchange for it taking longer time. Of course this isn't applicable in every case, sometimes you need different tradeoffs, that's OK too. But that some favor quality over shorter implementation time shouldn't drive people crazy, it's just making different tradeoffs.

Arch485 7mo ago

Exactly - purely "data driven" decisions are how we end up with ads really close to (or overlapping with) some button you want to press, because the data says that increase click-through rate! But it's actually a user-hostile feature that everyone hates.

PhoenixFlame101 7mo ago
But collecting data and looking for insights doesn't mean you mechanically optimize features, especially user-hostile ones? This is just as, if not more, likely to happen when basing your decisions on what people say they want over what they actually do.
- defmacr0 7mo ago
  If we were perfectly rational, then yeah, more data should never lead to worse decisions. However, it's easy to fall into the trap where being data-driven makes you only work on those things that you know how to measure.
  - account42 6mo ago
    Or worse, it makes you go look for data backing up the assumptions you already made.
- account42 6mo ago
  It's easier to not care about users if they are just numbers and you can tell yourself to ignore what they actually say.
mgfist 7mo ago
The reason that feature gets implemented is not because the devs think users will like it ... they know users don't want it, but it drives revenue and pays salaries.

staticassertion 7mo ago

It's sort of hilarious to compare "talking to people" with analytics. I'm not defending Github here, but you can't possibly think that "talking to 1M customers" is viable.

almostjazz 7mo ago
You could survey a representative sample
- staticassertion 7mo ago
  Not really. (a) People hate responding to surveys and hate emails, you're more likely to lose users than to get data (b) there's no way you're surveying people's in a way that gets you information like "time spent on a page" or "time between commits" or whatever. This is just nonsense tbh. Surveys and customer outreach solve completely different problems from analytics.
  - almostjazz 7mo ago
    I agree you can't practically get the same information as you could with telemetry. Survey data is still real data that can be used for "analytics". Some people also hate telemetry. It feels invasive. I have a guess about what direction the percentage of consumers who hate telemetry is moving toward.
  - codedokode 7mo ago
    You can hire people to test your product and provide analytics. But not try to siphon the data for free.
    - staticassertion 7mo ago
      I'm not taking a side on whether a product should add telemetry. I'm rejecting the absurd notion that these suggestions are at all giving the same information.

dpark 7mo ago

> Don't take what people say at face value, but think about it together with your knowledge and experience While you’re comparing different information sources, you might even want to consider telemetry, too.

SchemaLoad 7mo ago

What people say, and what people do are different things. Especially when the people who agree to talk to you aren't representative of the whole user base.

account42 6mo ago
And what people do and the reasons they do it are even more different.