10 ms·

Most software companies ignore user behavior

139 points by mrbbk 7y ago

petepete 7y ago
Not related to the article, but on mobile I can't scroll right to see the full table. Rather annoying.
- jakub_jo 7y ago
  Here's a screenshot for you: https://imgur.com/a/vMwBhLV https://imgur.com/a/vMwBhLV
- briandoll 7y ago
  Ah, bummer, thanks for letting us know. We'll look into the magical CSS incantations necessary to fit those in well on mobile. Cheers
  - petepete 7y ago
    No problem. I think table { overflow-x: auto; } Should do it
ABeeSea 7y ago
Having worked on user data before, I think the issue is that for complex software, it’s hard to get value/insight from this data. Imagine if you are Microsoft and had every action every Excel user makes with office365. Where would you even start to be able to get value out of this data?
- ska 7y ago
  > Where would you even start to be able to get value out of this data? It's an interesting question, but I think there are a few obvious easy places to get value (UX on common operations, most common error modes/corrections, etc.) In my experience the problem is more that devs apply logging or telemetry where it is easy rather than where it is insightful. So if you didn't design the system up front to capture user behavior with some granularity you end up with a bunch of questionable proxies for it, which are hard/impossible to interpret.
  - panopticon 7y ago
    I've also observed the problem where analysts or product managers cherry pick the data that validates their assumptions. A "funny" example I ran into was product wanted to ship a notification that "test better" (i.e., had a higher interaction rate) in A/B testing. Some developers pointed out that the notification was likely seeing higher click-throughs because it was incredibly vague and users clicked through to figure out what it was about, and this would likely result in user frustration. Well, this insight was ignored and the notification change was shipped. Our notification opt-out rate spiked a few weeks later.
- draugadrotten 7y ago
  Well, mcDonalds draws maps of where their workers put their feet, and then designs the kitchens to minimize the number of steps the workers needs to take to complete their jobs. One way to use Excel telemetry could surely be to minimize the number of cognitive steps workers needs to take to complete the tasks they are trying to do. Another could be to use telemetry to see at which point a smooth flow suddenly takes a longer time - and perhaps split an action into two actions which are faster (in time) to accomplish I could go on all day.
  - james_s_tayler 7y ago
    Can we get telemetry that reports on cognitive steps for developers working on code and minimize for that? I know there is cyclomatic complexity. Mostly you can look at a method and tell "this is a clusterfuck" without needing to break out a specialized tool. I still feel like a lot of devs don't appreciate the simple philosophy of "don't make me think."
  - pbhjpbhj 7y ago
    Isn't that what Ribbons are supposed to do, raise the visibility of oft-used functions, respecting context.
  - jaclaz 7y ago
    >Well, mcDonalds draws maps of where their workers put their feet, and then designs the kitchens to minimize the number of steps the workers needs to take to complete their jobs. In a completely different field (architecture, public spaces, parks) it has become common (and I find it a particularly smart approach) to create a new garden/public space/park with no "paths" and open it to the public. And put some benches "here and there" semi-randomly. Soon the grass will be worn out where people actually walks (i.e. people that needs to cross the park because it needs to go from point A to point B), so there you make the actual paved paths and the same goes for benches, some will have a path in the grass leading to them and some will look like never used, etc. But when it comes to Excel (or similar software) there is a difference. In the case of McDonalds personnel are instructed on exactly what they have to do and how exactly do it, i.e. their footsteps are their own optimization of an exactly specified chore. In the case of the people on public parks there is no need for instructions, as people actually needs to go through the park to get from point A to point B and knows where they love to seat down. In the case of Excel (or similar software) the issue is IMHO that most often than not the user doesn't actually know what he/she is doing, in the sense that user may (and often does) take a "wrong" or exceedingly convoluted path to get to the actual wanted result (when/if he/she actually gets to the result AND the result is actually correct). I often deal with people that (most probably in good faith) bluffed their way in "familiarity with Office" on their resumè and never actually learned to work in Excel, the spreadsheet they make are "terrible", with material (and sometimes also conceptual) mistakes that are all fine if you are using Excel to - say - keep your monthly budget under control but that cause (big) issues if you are using it at work to manage a project expenses/production/forecasts, a bid offer or similar. Feedback/telemetry from these users would "dumbify" the product and probably even make it much worse. >I could go on all day. I could too, I have seen things ...
- hinkley 7y ago
  I propose you start by studying a small batch of users, documenting their behaviors, develop a set of theories, then look at whether the larger pool of data agrees with or disproves your theory. Try to come up with theories just from data and you end up with the Texas Sharpshooter scenario. We are complaining about this sort of behavior showing up in the scientific community, let's learn from that mistake.
  - munk-a 7y ago
    I agree, I think doing a deep dive interview with a small set of users is better than just collecting telemetry. Ask them questions, carry on conversations, notice when they become frustrated trying to do something. All the users could be using the right-click context menu to copy text not out of preference but instead ignorance - telemetry can tell you what's wrong but you miss out on most of the chance to tell why.
- bgdnyxbjx 7y ago
  Parent is clearly a PM in Excel looking for ideas
- tempguy9999 7y ago
  Install an accelerometer in the keyboard. When heavy impacts are detected from someone smashing their face into it repeatedly, pop up a dialog box asking them if there is something wrong and would they like assistance? Extensive feedback will be forthcoming. A bit more helpfully, MS were asked repeatedly not to fuck with the menus but they replaced it with the ribbon. Ditto that terrible new interface that was nothing like windows 7. They could have allowed the choice of new interface but they did not. MS got plenty of info and chose to discard it. MS is now in a position where it doesn't need to care.
  - airstrike 7y ago
    I, for one, love the ribbon...
    - tempguy9999 7y ago
      Good for you. But so what? (Edit: my point is not about the ribbon)
      - airstrike 7y ago
        I'm just saying it's not necessarily true that the ribbon is worse than old school menus.
        tempguy9999 7y ago
        And I'd agree with you. So what do you think my point was?
        airstrike 7y ago
        Your point seemed to be that MS chose not to care. I think they cared but made an informed decision to go down a path that many said was wrong. If anything, I'd say kudos to them for not living in an echo chamber.
- esotericn 7y ago
  The very first thing I would do is look for repetitive behaviours and crack out Clippy v2.0. You know when someone is like, writing 1 then 2 then 3 then 4 then ... or whatever? Yeah, Clippy 2.0. "Did you know this hotkey called X exists"? Multiply that by N users - bam you've saved the human race thousands of hours. I'll take my Nobel now.
  - buerkle 7y ago
    IntelliJ has a plugin for that. https://plugins.jetbrains.com/plugin/9792-key-promoter-x/ https://plugins.jetbrains.com/plugin/9792-key-promoter-x/
  - scarejunba 7y ago
    IntelliJ IDEA has a plugin that will prompt you with the keys for commands you click. Pretty neat. Not exactly what you're saying (no compound actions). Still neat.
- rzzzt 7y ago
  I read about Office 2000's dynamically hidden menu items feature (with the expand button added to the bottom of each sub-menu) somewhere, that the items that _are_ shown initially were determined by collected usage data. So it's probably aggregation that makes the most sense, not digging into each individual user's behavior.
- airstrike 7y ago
  I'd say just invite me over for a chat. I've probably spent 10,000 hours in Excel and can tell you more about its shortcomings than a PB of telemetry ever will
- gxx 7y ago
  I worked on the design of the first several versions of Excel. For Excel 3.0 we were intensely interested in user behavior. We often compared alternative prototypes of proposed features in the usability lab. We sometimes would invite develpers to watch users strugging with what they thought was great idea... Also, long before the Internet we asked users permission to send them a special version of Excel (called the Instrumented Version) that would intelligently log actions, but not data to a floppy disk. Users would send the disks back to us for analysis. We used the analysis to understand how users really did things vs what we thought the did. Sometimes we got some surprises. We used this to prioritize features where users were having difficulty that needed special attention. It also indicated frequenty performed actions that would benefit from, for example, being put on the toolbar. I think our user focus was an important reason why Excel has been so enduring. (Although Office 2003 screwed it up to some extent - I was no longer there.) Back then at Microsoft the applications group tended to take the lead in UI developement and many of the things we designed made their way into Windows. Some were even picked up by the Mac! Although Steve Jobs would never have admitted it. It grieves me to see the shoddy state of software design these days. There is no craftsmanship. Visual design take precedence over real usability. Maybe this is because with fast developemnt cycles and the ability to instantly change a web based design, it's easer to throw crap at the wall and keep iterating until it sort of works. These days user interaction designers have endless great user data on which to craft wonderful UI but it rarely happens. If they happen to hit on something good, usually another team takes over and changes it, often for the worse! (End Rant)
  - galaxyLogic 7y ago
    Interesting to hear. One thing I never understood about Excel is why the cells which have formulas don't look different than cells which contain leaf-data. That I think would make it a lot more "user-friendly". 2ndly why not allow users name their columns, why do they have to always be "A", "B", "C", .. ?
    - MaulingMonkey 7y ago
      > One thing I never understood about Excel is why the cells which have formulas don't look different than cells which contain leaf-data. There are some styles actually meant for this, but you have to apply them manually. > 2ndly why not allow users name their columns, why do they have to always be "A", "B", "C", .. ? Named columns do exist actually! Not of the excel sheet itself - but in tables you can mark within that sheet. You can use the names in equations and everything. You can also name individual cells as well. Joel Spolsky has a whole talk where he goes into this and some of the other stuff Excel has that you might not realize it has: https://www.youtube.com/watch?v=0nbkaYsR94c https://www.youtube.com/watch?v=0nbkaYsR94c
- asdff 7y ago
  You can do so much with this data, I'm surprised if it's not already collected. You can stratify your users based on what kind of commands they do, how their mouse or cursor is moved around the screen, and what functions they are getting out of excel, and see if you can improve their experiences by lowering unneeded friction and building new features.
- pbhjpbhj 7y ago
  >Where would you even start to be able to get value out of this data? I'd start with the most undo-ne operations. Which specific actions are most often immediately undone by users (that could lead to addition of functions, changing UI, adding visibility of the effects of an action, all sorts). Then probably look at most searched terms in both Excel help and online, segmented by user experience if possible (to allow improvements for noobs and advanced users, whilst aiming not to be deleterious to either). I'm not in software development though, curious what answers y'all have? FWIW as a relatively novice LibreOffice user I probably most often undo auto-fills where a reference variable hasn't altered how I hoped. (I'd maybe fix that with a tailored tooltip - probably actually implemented as an info box - fired on long-hover of the auto-fill handle).
- bonestamp2 7y ago
  > Where would you even start to be able to get value out of this data? Much like excel for certain professions, I work on a piece of software that is used all day by people in another profession. One thing we do is look at which features are used most (and least). We make it easier to find and use the most popular features and we make rarely used features less prominent (and in some rare cases we might remove them altogether). In our most recent redesign, we elevated some of the most commonly used information to the point where the user doesn't even have to click anything to gain some high level insight (previous versions needed at least 1-2 clicks to get to that same info). We have about 30 power users who we work with continuously. Some of them are nearby, and we often go look over their shoulder and pick their brains. Some of them are in other regions where we get valuable global feedback that we wouldn't otherwise get from those regions. They get early access to new features and they get to help shape the product that they rely on for their livelihood and help us decide on what we should be working on next. It's win/win and I would recommend forming this kind of relationship with a subset of your user base if you can.
  - jaclaz 7y ago
    >One thing we do is look at which features are used most (and least). We make it easier to find and use the most popular features and we make rarely used features less prominent (and in some rare cases we might remove them altogether). Well, with all due respect, this is not about making a "better" product, it is about making a product "easier to use" and possibly even "more popular" (which not necessarily are the same thing). >We have about 30 power users who we work with continuously. Some of them are nearby, and we often go look over their shoulder and pick their brains. Some of them are in other regions where we get valuable global feedback that we wouldn't otherwise get from those regions. They get early access to new features and they get to help shape the product that they rely on for their livelihood and help us decide on what we should be working on next. It's win/win and I would recommend forming this kind of relationship with a subset of your user base if you can. On the other hand this can be a much more productive approach, the program will "evolve" following the lead of these "power users", until it will become "almost perfect" (and more complex, and less popular because it has so many functions ...) In other words, both approaches have their merits (and downsides), and IMHO you have a good process in having both, but ultimately it is the "common sense" applied to the choices you make for your product that may make a difference.
- tinus_hn 7y ago
  Microsoft actually tracks this with their telemetry, they say they want to prioritize work to the features that are used most.
skybrian 7y ago
Hmm. No mention of privacy issues?
- mrbbk 7y ago
  I thought about adding something about privacy, but that's sort of a different topic for a different time. There are plenty of privacy preserving ways of learning from this data.
  - compiler-guy 7y ago
    It's not a different topic at all. If you don't design it in from the start, it is very hard to get right later.
- izacus 7y ago
  Those same companies mostly load their software and websites full of trackers against all reason. They are very very happy to collect that data. They just never use it - management personalities usually override any data findings... if anyone even ever looks at the results.
pictur 7y ago
https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ https://www.nngroup.com/articles/why-you-only-need-to-test-w...
- hinkley 7y ago
  I like some of the information in this, and unfortunately I missed the conversation when this made it to the front page a couple weeks ago, I just wasn't in the right headspace. But I disagree with one of the value judgements: > As you add more and more users, you learn less and less because you will keep seeing the same things again and again. No! The frequency of errors is your priority list for UX fixes. You may not, as they claim, find more errors with more than 15 people, but more data samples gives you information about distribution and frequency of errors (do people make this mistake once in a while or every single time?). You might not need any of that data to make a rough priority list, or it might be helpful for negotiating scope. It'll depend a lot on context.
  - marcosdumay 7y ago
    If you go through that site's (very long) history, you'll see their conclusions that quantitative UX research isn't useful for single projects. For discovering features of the population, yes, quantitative research is great. But for making your software better, you are much better iterating small qualitative tests.
  - superhuzza 7y ago
    The point of usability testing is primarily to identify sources of roadblocks and confusion. It helps you understand where people can get stuck and why, but not the frequency. Doing usability testing is both expensive and time consuming. It requires someone to analyze the videos and the testers are typically paid. So it's a very poor choice for creating frequency data. Once the usability issues have been uncovered, it might make sense to look at the relevant analytics or add better tagging to get that frequency data.
hinkley 7y ago
Most software developers ignore user behavior. It was a bit of a shock for me to figure out that much of my early success in software boiled down to a few things I was doing that nobody else was, and one of those was watching other people work and doing something - anything - more with that information than simply writing it down or complaining about it. Most of those people have been coworkers, not even customers.
- privateSFacct 7y ago
  Agreed - I credit my entire career to this. I started watching people trying to solve problems, and then would piece together something "silly" that helped them solve it. Next thing I was doing a big system implementation. Then consulting. One note - now that I'm removed from day to day and don't have time to program myself I see how easy it is NOT to pay attention to how anyone will use something. I imagine this is how windows gets bloated with candy crush saga and all the bloatware even on the "pro" versions. What employer wants this stuff on users computers? It just clutters up menues.
  - reificator 7y ago
    I play games all the time and I don't want games preinstalled. And it's not just that it's a casual game like Candy Crush. I don't care if it's the latest Call of Duty or whatever you young whippersnappers play nowadays. My work PC is for work and my gaming PC is for games I choose to download. Even if it came with Minecraft I'd go and download the Java version instead for mod support.
  - Cpoll 7y ago
    > I imagine this is how windows gets bloated with candy crush saga Who knows, it's possible the Windows team just watched a lot of people install Candy Crush, and decided it would be a good addition to Minesweeper/Solitaire/Hearts (and what employer wants those on their computers?).
PopeDotNinja 7y ago
I take some comfort in the fact that as more and more data gets collected about me, Big Sibling will be drowning in so much data that they can't possibly make sense of it all. Eventually an nearly infinite amount of analysis will conclude that I am boring actually as boring as was initially estimated. That said, I'd be pleased with less monitoring of myself.
- t0astbread 7y ago
  Reminds me of this talk[1] I've seen a while ago where an ex-NSA executive talks about how the NSA has too much data to handle it efficiently (amongst other things). [1] https://media.ccc.de/v/SHA2017-402-how_the_nsa_tracks_you https://media.ccc.de/v/SHA2017-402-how_the_nsa_tracks_you
- squirrelicus 7y ago
  Hah! It's Data Mining all over again! Do... Do you guys remember Data Mining?
gumby 7y ago
Not just software developers. I was working on a medical device and visited doctors' offices to see how and where they stored their medication (e.g. for how could we make our dispenser more prominent without getting in the way?). We also filmed our investigators holding various prototype devices to see how they picked them up, how they were balanced, how they felt in small and large hands etc. Universally we were told that nobody had ever asked them anything like that before.
- deleted 7y ago
  [deleted]
- squirrelicus 7y ago
  Well, I've never been called to participate in a political survey poll thing, but that doesn't mean it doesn't happen to people. There's a lot of doctor's offices out there. What was your sample size?
  - gumby 7y ago
    Small, a few hundred, but was in a medical speciality so it was a reasonable number. In addition, that pool included all of our clinical investigators, the “thought leaders” you’d think most likely to have been consulted.
- taneq 7y ago
  I've never understood how people can design and build anything (software, hardware, whatever) without understanding how it's used. Dogfooding is great (and so many places don't even do that!) but you don't really know what you need to build until you sit down with your users and actually watch them work.
  - kwhitefoot 7y ago
    It's really difficult to understand how a large complicated appplication should be used if you have no use for it yourself. I spent most of the last twenty years workin g on design software for transformers (the electrical kind not the newly fashionabloe machine learning stuff) but I have never designed a transformer in my life. There is no way of dogfooding such a product the developers have no use for it at all. So we do our best, observe the users, try to provide shortcuts and hints. But watching a user work is really difficult too because in a large application the users don't all use all of the application all of the time so getting a realistic picture of the whole use of the program is extremely time consuming which means expensive and the client departments just want to use it, they very often aren't interested in optimizing it.
t0astbread 7y ago
I don't doubt that this kind of invasive data collection can help developers at some point. But as a user I'm glad that most of the software I'm using doesn't track everything I do (or even register me as a distinct user at all). I'm also willing to accept poorer software quality in return (which curiously, isn't the case).
SkyPuncher 7y ago
I'm working on an app and trying to be conscientious about my user's behavior, but it's proving to be extremely time consuming for the value I get out. The problem for me is I can take a relatively accurate guess and achieve 90%+ of what the user is looking for. That last 10% takes a ton of time to figure out. For every moment I spend on the 10%, I'm only getting a fractional return compared to simply moving onto another feature. Maybe in the future, I'll come back and figure things out, but for now it's just not worth my time.
- sjlogan 7y ago
  With a few lines of code, you can integrate Segment (or a similar tool) into your software. You can start tracking user behavior and events automatically. Segment can then send this data onto tools like Mixpanel or Amplitude which has built-in user activity flow analytics. Take a look into some of these tools if you are looking for an easier way to track user behavior.
iamaelephant 7y ago
Love to add a column to one of the most critical tables in my system every time I add a feature. Galaxy brain stuff here.
JohnFen 7y ago
In my view, too many software companies have adopted the horrid practices that marketers like this guy have been inflicting on us. We don't need more of this.
yoz-y 7y ago
I would love to know which features of my app are used and by how many user. But this clashes with my requirement to not have any server side service or telemetry.
- sjlogan 7y ago
  You can use a tool like Segment to track user activity client side. With a few lines of code in your app / front end you can be sending user behavior and events to Segment via their API. No custom server side service is required. Might be worth digging into if you are interested in better user activity tracking.
  - yoz-y 7y ago
    I'd rather not include a third party either. Then I would have to do all the GDPR work to disclose what do they get, how do they store it and so on. Ideally Apple would provide some framework to do basic functionality analytics.

139 points by mrbbk 7y ago

petepete 7y ago

Not related to the article, but on mobile I can't scroll right to see the full table. Rather annoying.

jakub_jo 7y ago
Here's a screenshot for you: https://imgur.com/a/vMwBhLV https://imgur.com/a/vMwBhLV
briandoll 7y ago
Ah, bummer, thanks for letting us know. We'll look into the magical CSS incantations necessary to fit those in well on mobile. Cheers
- petepete 7y ago
  No problem. I think table { overflow-x: auto; } Should do it

ABeeSea 7y ago

Having worked on user data before, I think the issue is that for complex software, it’s hard to get value/insight from this data. Imagine if you are Microsoft and had every action every Excel user makes with office365. Where would you even start to be able to get value out of this data?

ska 7y ago
> Where would you even start to be able to get value out of this data? It's an interesting question, but I think there are a few obvious easy places to get value (UX on common operations, most common error modes/corrections, etc.) In my experience the problem is more that devs apply logging or telemetry where it is easy rather than where it is insightful. So if you didn't design the system up front to capture user behavior with some granularity you end up with a bunch of questionable proxies for it, which are hard/impossible to interpret.
- panopticon 7y ago
  I've also observed the problem where analysts or product managers cherry pick the data that validates their assumptions. A "funny" example I ran into was product wanted to ship a notification that "test better" (i.e., had a higher interaction rate) in A/B testing. Some developers pointed out that the notification was likely seeing higher click-throughs because it was incredibly vague and users clicked through to figure out what it was about, and this would likely result in user frustration. Well, this insight was ignored and the notification change was shipped. Our notification opt-out rate spiked a few weeks later.
draugadrotten 7y ago
Well, mcDonalds draws maps of where their workers put their feet, and then designs the kitchens to minimize the number of steps the workers needs to take to complete their jobs. One way to use Excel telemetry could surely be to minimize the number of cognitive steps workers needs to take to complete the tasks they are trying to do. Another could be to use telemetry to see at which point a smooth flow suddenly takes a longer time - and perhaps split an action into two actions which are faster (in time) to accomplish I could go on all day.
- james_s_tayler 7y ago
  Can we get telemetry that reports on cognitive steps for developers working on code and minimize for that? I know there is cyclomatic complexity. Mostly you can look at a method and tell "this is a clusterfuck" without needing to break out a specialized tool. I still feel like a lot of devs don't appreciate the simple philosophy of "don't make me think."
- pbhjpbhj 7y ago
  Isn't that what Ribbons are supposed to do, raise the visibility of oft-used functions, respecting context.
- jaclaz 7y ago
  >Well, mcDonalds draws maps of where their workers put their feet, and then designs the kitchens to minimize the number of steps the workers needs to take to complete their jobs. In a completely different field (architecture, public spaces, parks) it has become common (and I find it a particularly smart approach) to create a new garden/public space/park with no "paths" and open it to the public. And put some benches "here and there" semi-randomly. Soon the grass will be worn out where people actually walks (i.e. people that needs to cross the park because it needs to go from point A to point B), so there you make the actual paved paths and the same goes for benches, some will have a path in the grass leading to them and some will look like never used, etc. But when it comes to Excel (or similar software) there is a difference. In the case of McDonalds personnel are instructed on exactly what they have to do and how exactly do it, i.e. their footsteps are their own optimization of an exactly specified chore. In the case of the people on public parks there is no need for instructions, as people actually needs to go through the park to get from point A to point B and knows where they love to seat down. In the case of Excel (or similar software) the issue is IMHO that most often than not the user doesn't actually know what he/she is doing, in the sense that user may (and often does) take a "wrong" or exceedingly convoluted path to get to the actual wanted result (when/if he/she actually gets to the result AND the result is actually correct). I often deal with people that (most probably in good faith) bluffed their way in "familiarity with Office" on their resumè and never actually learned to work in Excel, the spreadsheet they make are "terrible", with material (and sometimes also conceptual) mistakes that are all fine if you are using Excel to - say - keep your monthly budget under control but that cause (big) issues if you are using it at work to manage a project expenses/production/forecasts, a bid offer or similar. Feedback/telemetry from these users would "dumbify" the product and probably even make it much worse. >I could go on all day. I could too, I have seen things ...
hinkley 7y ago
I propose you start by studying a small batch of users, documenting their behaviors, develop a set of theories, then look at whether the larger pool of data agrees with or disproves your theory. Try to come up with theories just from data and you end up with the Texas Sharpshooter scenario. We are complaining about this sort of behavior showing up in the scientific community, let's learn from that mistake.
- munk-a 7y ago
  I agree, I think doing a deep dive interview with a small set of users is better than just collecting telemetry. Ask them questions, carry on conversations, notice when they become frustrated trying to do something. All the users could be using the right-click context menu to copy text not out of preference but instead ignorance - telemetry can tell you what's wrong but you miss out on most of the chance to tell why.
bgdnyxbjx 7y ago
Parent is clearly a PM in Excel looking for ideas
tempguy9999 7y ago
Install an accelerometer in the keyboard. When heavy impacts are detected from someone smashing their face into it repeatedly, pop up a dialog box asking them if there is something wrong and would they like assistance? Extensive feedback will be forthcoming. A bit more helpfully, MS were asked repeatedly not to fuck with the menus but they replaced it with the ribbon. Ditto that terrible new interface that was nothing like windows 7. They could have allowed the choice of new interface but they did not. MS got plenty of info and chose to discard it. MS is now in a position where it doesn't need to care.
- airstrike 7y ago
  I, for one, love the ribbon...
  - tempguy9999 7y ago
    Good for you. But so what? (Edit: my point is not about the ribbon)
    - airstrike 7y ago
      I'm just saying it's not necessarily true that the ribbon is worse than old school menus.
      - tempguy9999 7y ago
        And I'd agree with you. So what do you think my point was?
        airstrike 7y ago
        Your point seemed to be that MS chose not to care. I think they cared but made an informed decision to go down a path that many said was wrong. If anything, I'd say kudos to them for not living in an echo chamber.
esotericn 7y ago
The very first thing I would do is look for repetitive behaviours and crack out Clippy v2.0. You know when someone is like, writing 1 then 2 then 3 then 4 then ... or whatever? Yeah, Clippy 2.0. "Did you know this hotkey called X exists"? Multiply that by N users - bam you've saved the human race thousands of hours. I'll take my Nobel now.
- buerkle 7y ago
  IntelliJ has a plugin for that. https://plugins.jetbrains.com/plugin/9792-key-promoter-x/ https://plugins.jetbrains.com/plugin/9792-key-promoter-x/
- scarejunba 7y ago
  IntelliJ IDEA has a plugin that will prompt you with the keys for commands you click. Pretty neat. Not exactly what you're saying (no compound actions). Still neat.
rzzzt 7y ago
I read about Office 2000's dynamically hidden menu items feature (with the expand button added to the bottom of each sub-menu) somewhere, that the items that _are_ shown initially were determined by collected usage data. So it's probably aggregation that makes the most sense, not digging into each individual user's behavior.
airstrike 7y ago
I'd say just invite me over for a chat. I've probably spent 10,000 hours in Excel and can tell you more about its shortcomings than a PB of telemetry ever will
gxx 7y ago
I worked on the design of the first several versions of Excel. For Excel 3.0 we were intensely interested in user behavior. We often compared alternative prototypes of proposed features in the usability lab. We sometimes would invite develpers to watch users strugging with what they thought was great idea... Also, long before the Internet we asked users permission to send them a special version of Excel (called the Instrumented Version) that would intelligently log actions, but not data to a floppy disk. Users would send the disks back to us for analysis. We used the analysis to understand how users really did things vs what we thought the did. Sometimes we got some surprises. We used this to prioritize features where users were having difficulty that needed special attention. It also indicated frequenty performed actions that would benefit from, for example, being put on the toolbar. I think our user focus was an important reason why Excel has been so enduring. (Although Office 2003 screwed it up to some extent - I was no longer there.) Back then at Microsoft the applications group tended to take the lead in UI developement and many of the things we designed made their way into Windows. Some were even picked up by the Mac! Although Steve Jobs would never have admitted it. It grieves me to see the shoddy state of software design these days. There is no craftsmanship. Visual design take precedence over real usability. Maybe this is because with fast developemnt cycles and the ability to instantly change a web based design, it's easer to throw crap at the wall and keep iterating until it sort of works. These days user interaction designers have endless great user data on which to craft wonderful UI but it rarely happens. If they happen to hit on something good, usually another team takes over and changes it, often for the worse! (End Rant)
- galaxyLogic 7y ago
  Interesting to hear. One thing I never understood about Excel is why the cells which have formulas don't look different than cells which contain leaf-data. That I think would make it a lot more "user-friendly". 2ndly why not allow users name their columns, why do they have to always be "A", "B", "C", .. ?
  - MaulingMonkey 7y ago
    > One thing I never understood about Excel is why the cells which have formulas don't look different than cells which contain leaf-data. There are some styles actually meant for this, but you have to apply them manually. > 2ndly why not allow users name their columns, why do they have to always be "A", "B", "C", .. ? Named columns do exist actually! Not of the excel sheet itself - but in tables you can mark within that sheet. You can use the names in equations and everything. You can also name individual cells as well. Joel Spolsky has a whole talk where he goes into this and some of the other stuff Excel has that you might not realize it has: https://www.youtube.com/watch?v=0nbkaYsR94c https://www.youtube.com/watch?v=0nbkaYsR94c
asdff 7y ago
You can do so much with this data, I'm surprised if it's not already collected. You can stratify your users based on what kind of commands they do, how their mouse or cursor is moved around the screen, and what functions they are getting out of excel, and see if you can improve their experiences by lowering unneeded friction and building new features.
pbhjpbhj 7y ago
>Where would you even start to be able to get value out of this data? I'd start with the most undo-ne operations. Which specific actions are most often immediately undone by users (that could lead to addition of functions, changing UI, adding visibility of the effects of an action, all sorts). Then probably look at most searched terms in both Excel help and online, segmented by user experience if possible (to allow improvements for noobs and advanced users, whilst aiming not to be deleterious to either). I'm not in software development though, curious what answers y'all have? FWIW as a relatively novice LibreOffice user I probably most often undo auto-fills where a reference variable hasn't altered how I hoped. (I'd maybe fix that with a tailored tooltip - probably actually implemented as an info box - fired on long-hover of the auto-fill handle).
bonestamp2 7y ago
> Where would you even start to be able to get value out of this data? Much like excel for certain professions, I work on a piece of software that is used all day by people in another profession. One thing we do is look at which features are used most (and least). We make it easier to find and use the most popular features and we make rarely used features less prominent (and in some rare cases we might remove them altogether). In our most recent redesign, we elevated some of the most commonly used information to the point where the user doesn't even have to click anything to gain some high level insight (previous versions needed at least 1-2 clicks to get to that same info). We have about 30 power users who we work with continuously. Some of them are nearby, and we often go look over their shoulder and pick their brains. Some of them are in other regions where we get valuable global feedback that we wouldn't otherwise get from those regions. They get early access to new features and they get to help shape the product that they rely on for their livelihood and help us decide on what we should be working on next. It's win/win and I would recommend forming this kind of relationship with a subset of your user base if you can.
- jaclaz 7y ago
  >One thing we do is look at which features are used most (and least). We make it easier to find and use the most popular features and we make rarely used features less prominent (and in some rare cases we might remove them altogether). Well, with all due respect, this is not about making a "better" product, it is about making a product "easier to use" and possibly even "more popular" (which not necessarily are the same thing). >We have about 30 power users who we work with continuously. Some of them are nearby, and we often go look over their shoulder and pick their brains. Some of them are in other regions where we get valuable global feedback that we wouldn't otherwise get from those regions. They get early access to new features and they get to help shape the product that they rely on for their livelihood and help us decide on what we should be working on next. It's win/win and I would recommend forming this kind of relationship with a subset of your user base if you can. On the other hand this can be a much more productive approach, the program will "evolve" following the lead of these "power users", until it will become "almost perfect" (and more complex, and less popular because it has so many functions ...) In other words, both approaches have their merits (and downsides), and IMHO you have a good process in having both, but ultimately it is the "common sense" applied to the choices you make for your product that may make a difference.
tinus_hn 7y ago
Microsoft actually tracks this with their telemetry, they say they want to prioritize work to the features that are used most.

skybrian 7y ago

Hmm. No mention of privacy issues?

mrbbk 7y ago
I thought about adding something about privacy, but that's sort of a different topic for a different time. There are plenty of privacy preserving ways of learning from this data.
- compiler-guy 7y ago
  It's not a different topic at all. If you don't design it in from the start, it is very hard to get right later.
izacus 7y ago
Those same companies mostly load their software and websites full of trackers against all reason. They are very very happy to collect that data. They just never use it - management personalities usually override any data findings... if anyone even ever looks at the results.

pictur 7y ago

https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ https://www.nngroup.com/articles/why-you-only-need-to-test-w...

hinkley 7y ago
I like some of the information in this, and unfortunately I missed the conversation when this made it to the front page a couple weeks ago, I just wasn't in the right headspace. But I disagree with one of the value judgements: > As you add more and more users, you learn less and less because you will keep seeing the same things again and again. No! The frequency of errors is your priority list for UX fixes. You may not, as they claim, find more errors with more than 15 people, but more data samples gives you information about distribution and frequency of errors (do people make this mistake once in a while or every single time?). You might not need any of that data to make a rough priority list, or it might be helpful for negotiating scope. It'll depend a lot on context.
- marcosdumay 7y ago
  If you go through that site's (very long) history, you'll see their conclusions that quantitative UX research isn't useful for single projects. For discovering features of the population, yes, quantitative research is great. But for making your software better, you are much better iterating small qualitative tests.
- superhuzza 7y ago
  The point of usability testing is primarily to identify sources of roadblocks and confusion. It helps you understand where people can get stuck and why, but not the frequency. Doing usability testing is both expensive and time consuming. It requires someone to analyze the videos and the testers are typically paid. So it's a very poor choice for creating frequency data. Once the usability issues have been uncovered, it might make sense to look at the relevant analytics or add better tagging to get that frequency data.

hinkley 7y ago

Most software developers ignore user behavior. It was a bit of a shock for me to figure out that much of my early success in software boiled down to a few things I was doing that nobody else was, and one of those was watching other people work and doing something - anything - more with that information than simply writing it down or complaining about it. Most of those people have been coworkers, not even customers.

privateSFacct 7y ago
Agreed - I credit my entire career to this. I started watching people trying to solve problems, and then would piece together something "silly" that helped them solve it. Next thing I was doing a big system implementation. Then consulting. One note - now that I'm removed from day to day and don't have time to program myself I see how easy it is NOT to pay attention to how anyone will use something. I imagine this is how windows gets bloated with candy crush saga and all the bloatware even on the "pro" versions. What employer wants this stuff on users computers? It just clutters up menues.
- reificator 7y ago
  I play games all the time and I don't want games preinstalled. And it's not just that it's a casual game like Candy Crush. I don't care if it's the latest Call of Duty or whatever you young whippersnappers play nowadays. My work PC is for work and my gaming PC is for games I choose to download. Even if it came with Minecraft I'd go and download the Java version instead for mod support.
- Cpoll 7y ago
  > I imagine this is how windows gets bloated with candy crush saga Who knows, it's possible the Windows team just watched a lot of people install Candy Crush, and decided it would be a good addition to Minesweeper/Solitaire/Hearts (and what employer wants those on their computers?).

PopeDotNinja 7y ago

I take some comfort in the fact that as more and more data gets collected about me, Big Sibling will be drowning in so much data that they can't possibly make sense of it all. Eventually an nearly infinite amount of analysis will conclude that I am boring actually as boring as was initially estimated. That said, I'd be pleased with less monitoring of myself.

t0astbread 7y ago
Reminds me of this talk[1] I've seen a while ago where an ex-NSA executive talks about how the NSA has too much data to handle it efficiently (amongst other things). [1] https://media.ccc.de/v/SHA2017-402-how_the_nsa_tracks_you https://media.ccc.de/v/SHA2017-402-how_the_nsa_tracks_you
squirrelicus 7y ago
Hah! It's Data Mining all over again! Do... Do you guys remember Data Mining?

gumby 7y ago

Not just software developers. I was working on a medical device and visited doctors' offices to see how and where they stored their medication (e.g. for how could we make our dispenser more prominent without getting in the way?). We also filmed our investigators holding various prototype devices to see how they picked them up, how they were balanced, how they felt in small and large hands etc. Universally we were told that nobody had ever asked them anything like that before.

deleted 7y ago
[deleted]
squirrelicus 7y ago
Well, I've never been called to participate in a political survey poll thing, but that doesn't mean it doesn't happen to people. There's a lot of doctor's offices out there. What was your sample size?
- gumby 7y ago
  Small, a few hundred, but was in a medical speciality so it was a reasonable number. In addition, that pool included all of our clinical investigators, the “thought leaders” you’d think most likely to have been consulted.
taneq 7y ago
I've never understood how people can design and build anything (software, hardware, whatever) without understanding how it's used. Dogfooding is great (and so many places don't even do that!) but you don't really know what you need to build until you sit down with your users and actually watch them work.
- kwhitefoot 7y ago
  It's really difficult to understand how a large complicated appplication should be used if you have no use for it yourself. I spent most of the last twenty years workin g on design software for transformers (the electrical kind not the newly fashionabloe machine learning stuff) but I have never designed a transformer in my life. There is no way of dogfooding such a product the developers have no use for it at all. So we do our best, observe the users, try to provide shortcuts and hints. But watching a user work is really difficult too because in a large application the users don't all use all of the application all of the time so getting a realistic picture of the whole use of the program is extremely time consuming which means expensive and the client departments just want to use it, they very often aren't interested in optimizing it.

t0astbread 7y ago

I don't doubt that this kind of invasive data collection can help developers at some point. But as a user I'm glad that most of the software I'm using doesn't track everything I do (or even register me as a distinct user at all). I'm also willing to accept poorer software quality in return (which curiously, isn't the case).

SkyPuncher 7y ago

I'm working on an app and trying to be conscientious about my user's behavior, but it's proving to be extremely time consuming for the value I get out. The problem for me is I can take a relatively accurate guess and achieve 90%+ of what the user is looking for. That last 10% takes a ton of time to figure out. For every moment I spend on the 10%, I'm only getting a fractional return compared to simply moving onto another feature. Maybe in the future, I'll come back and figure things out, but for now it's just not worth my time.

sjlogan 7y ago
With a few lines of code, you can integrate Segment (or a similar tool) into your software. You can start tracking user behavior and events automatically. Segment can then send this data onto tools like Mixpanel or Amplitude which has built-in user activity flow analytics. Take a look into some of these tools if you are looking for an easier way to track user behavior.

iamaelephant 7y ago

Love to add a column to one of the most critical tables in my system every time I add a feature. Galaxy brain stuff here.

JohnFen 7y ago

In my view, too many software companies have adopted the horrid practices that marketers like this guy have been inflicting on us. We don't need more of this.

yoz-y 7y ago

I would love to know which features of my app are used and by how many user. But this clashes with my requirement to not have any server side service or telemetry.

sjlogan 7y ago
You can use a tool like Segment to track user activity client side. With a few lines of code in your app / front end you can be sending user behavior and events to Segment via their API. No custom server side service is required. Might be worth digging into if you are interested in better user activity tracking.
- yoz-y 7y ago
  I'd rather not include a third party either. Then I would have to do all the GDPR work to disclose what do they get, how do they store it and so on. Ideally Apple would provide some framework to do basic functionality analytics.