7 ms·

I'll offer a contrarian opinion. A lot of the issues raised recently in our industry around privacy or algorithmic bias come from the fact that as an industry t

I'll offer a contrarian opinion. A lot of the issues raised recently in our industry around privacy or algorithmic bias come from the fact that as an industry tech has the hubris to believe that it knows better than its users what is best for them. Under the pretense of simplicity and scalability, we think that every individual behavior and tastes can be reduced into a one-size-fits-all product and business model, sprinkled with machine learning to give the illusion of a product experience tailored to different people, when in fact it is only mildly so. And I'm saying this as someone building machine learning models for a living.

solipsism 7y ago
You see it as hubris. I see it as earnestly trying to build amazing things but failing.
- deleted 7y ago
  [deleted]
- zoeysaurusrex 7y ago
  I agree with you here. The majority of the people involved are trying to do something good. I’d wager the egomaniacs are a much smaller percentage than those who want to build better things.
- benj111 7y ago
  Are they not functionally equivalent? I suppose morally there's a difference, but that's just us and our biases.
calimac 7y ago
Hubris is a wonderful adjective for the arrogance now main stream in the industry.
citrablue 7y ago
Perhaps I am thinking too narrowly, but I think the algorithm-driven approach is much less hubris than someone at the company making a decision. These models reflect what users do -- isn't that much more democratic than employees deciding? There is an orthogonal argument that this is also harmful to some people's aspect, but "give the people what they want" doesn't seem like it's reductive. Especially when it seems like people in our society can be clustered and have some clear groupings occur.
- jakelazaroff 7y ago
  Is the algorithm-driven approach really any better, though? If our UI leads users to accidentally take an action, our data will actively mislead us into thinking it's what they want! Designing clear interfaces, measuring things and coming up with models are all really hard. Data needs to be paired with human judgment and decision making.
- naravara 7y ago
  >These models reflect what users do -- isn't that much more democratic than employees deciding? What users do is different from what users would like to do. If people's aspirations lined up with their real behaviors, Planet Fitness wouldn't have a business model. But when navigating the world people often want to improve themselves into something they want to be, not fall deeper into the worst tendencies they exhibit in any given moment.
- farisjarrah 7y ago
  You aren't wrong, but that's presumptive on humans being able to create algorithms that aren't implicitly bias. 2, hopefully not overly simplistic examples, A) Automatic Paper Towel dispensers that don't work nearly as well with people with dark skin. (Unintentional bias on the hands of the designers) B) People getting google banned and then being totally helpless and useless and not being able to get themselves unbanned (an algorithm determined there was abuse, and there is no recourse to tell the machine that it was mistaken, the entire thing was automated away via algorithms.)
r00fus 7y ago
So two competing priorities in a) giving users all the control/configurability they want and b) per OWASP and most SQL security standards “never trust user input” Explain straddling that line ... oh and the third axis is cost.
- Nasrudith 7y ago
  Those two are perfectly compatible actually - although doing both is harder and thus economically more expensive. Not trusting user input is about security. Verify that everything is something which they /should/ be able to do. Think say for a game's bank account. They should never be able to directly change that value without valid transactions to and from. If they can lose money by writing a check to null that is bad. If they can /gain/ money by writing a negative check to null that is /really/ bad. Also really bad is being allowed to just take money from banks of others without a validated transaction. Configurability lets them do anything that is in their rights. If they can access their bank balance and set it as part of their signature that is allowed along with say setting their crosshair to display the target HP which they already know. If they can create a custom texture of their bank balance repeated ad nauseam that is fine. So long as they are capable of that stuff already.
  - Kalium 7y ago
    You can actually get emergent conflicts from this in somewhat unexpected ways. When users are enabled to export a bunch of private data in the name of choice and control, they can be encouraged to do so unwisely. Users rarely stop and read warning messages, no matter how simple and clear and straightforward they are.
sfink 7y ago
I don't see that as a contrarian opinion, at least not with my reading of the post. Bicking isn't saying you should decide exactly what your users want and give them that and only that. (Well, he's not saying to not do that either, if in a situation where that's doable...) He is simply saying that if your final conclusion is "users want more control", then you're copping out. You are giving up on figuring out what your users really want (whether they know what it is or not), and just throwing options at them in hopes that they'll be able to make something out of them. Or at least stop complaining. Giving your users control isn't bad. Not figuring out why your users are asking for more control is bad.
- doomglobe 7y ago
  Most users don't want control - more things to think about takes energy and time. The users that do want control, however, are likely to be the users championing your product. Take a standard transmission in automobiles - the extra control it gives isn't necessary, and a large number of consumers prefer the automatic transmission. But the racecar drivers and car lovers are much more likely to want a standard. If you have intelligent, invested users, they'll want control options, and they'll usually tell you what they should be.
  - benj111 7y ago
    On the flip side my government insists on giving me a choice of schools for my child. I don't really want choice, I want the local school to be good and for my children to go there. In this view, users still don't want control, its just a symptom of the failure to provide what users do actually want. The truth is probably between these 2 outlooks
    - rubber_duck 7y ago
      > I want the local school to be good and for my children to go there That's assuming there's a universal "good" - even if a school meets some general standards one school can have a better arts program, the other might be more biased towards sports, etc. I don't see anything wrong with having choice - having a good baseline everywhere is a separate issue.
WAHa_06x36 7y ago
This is absolutely not true. This is what they ACT like, but this is not their true motivation. These companies want all control to be theirs, and they want to be free to do what they want with users and their data, and they want to spend as little time, money and effort as possible while doing it. They know they can't state this honestly, so they dress it up in excuses, rationalisations and outright lies. They KNOW they don't know better than its users. But they don't want to let users decide, because users would make decisions that go against their own profits, so they remove the choice and lie about why.
stcredzero 7y ago
A lot of the issues raised recently in our industry around privacy or algorithmic bias come from the fact that as an industry tech has the hubris to believe that it knows better than its users what is best for them. A good way to summarize, "users want control," is to note that, "users don't want to be screwed." Users don't want to be put into a situation where they have lost so much power, they don't have practical options, or the only options they have left are punitively unpleasant.
setr 7y ago
>A lot of the issues raised recently in our industry around privacy or algorithmic bias come from the fact that as an industry tech has the hubris to believe that it knows better than its users what is best for them. I don't think this is that bad, on its own. Choice is a pain in the ass, so it's useful to have things pick for you, when you don't care to choose yourself (which is very common). The main issue is that there is never offered a means to correct the algorithm, when it chooses something the user decidedly doesn't want. That is, there's no user-inputted feedback mechanism; they go for implicit tracking like "eyeball-time" to correct the algorithm, despite the user (me) being both able and willing to give an explicit answer. I want Netflix to give me better recommendations, and I'm willing to put in a bit of effort to support that. But Netflix doesn't want me to help them help me. And thats just stupid.
- webmaven 7y ago
  Netflix used to have those explicit feedback mechanisms via user star ratings, and even ran a contest to improve their recommendations based on the data[0], but it turns out "that explicit star ratings were less relevant than other signals. Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries."[1] Changing from stars to thumbs increased the quantity of data (at the expense of granularity), but didn't change the relative advantage of tracking behavior over explicit ratings. Now, your ratings may be free of these issues, but that data isn't very useful in isolation, so blame the bad data from other users for the removal of the feedback mechanism. [0] https://www.netflixprize.com/ https://www.netflixprize.com/ [1] https://variety.com/2017/digital/news/netflix-thumbs-vs-stars-1202010492/ https://variety.com/2017/digital/news/netflix-thumbs-vs-star...

solipsism 7y ago

You see it as hubris. I see it as earnestly trying to build amazing things but failing.

deleted 7y ago
[deleted]
zoeysaurusrex 7y ago
I agree with you here. The majority of the people involved are trying to do something good. I’d wager the egomaniacs are a much smaller percentage than those who want to build better things.
benj111 7y ago
Are they not functionally equivalent? I suppose morally there's a difference, but that's just us and our biases.

calimac 7y ago

Hubris is a wonderful adjective for the arrogance now main stream in the industry.

citrablue 7y ago

Perhaps I am thinking too narrowly, but I think the algorithm-driven approach is much less hubris than someone at the company making a decision. These models reflect what users do -- isn't that much more democratic than employees deciding? There is an orthogonal argument that this is also harmful to some people's aspect, but "give the people what they want" doesn't seem like it's reductive. Especially when it seems like people in our society can be clustered and have some clear groupings occur.

jakelazaroff 7y ago
Is the algorithm-driven approach really any better, though? If our UI leads users to accidentally take an action, our data will actively mislead us into thinking it's what they want! Designing clear interfaces, measuring things and coming up with models are all really hard. Data needs to be paired with human judgment and decision making.
naravara 7y ago
>These models reflect what users do -- isn't that much more democratic than employees deciding? What users do is different from what users would like to do. If people's aspirations lined up with their real behaviors, Planet Fitness wouldn't have a business model. But when navigating the world people often want to improve themselves into something they want to be, not fall deeper into the worst tendencies they exhibit in any given moment.
farisjarrah 7y ago
You aren't wrong, but that's presumptive on humans being able to create algorithms that aren't implicitly bias. 2, hopefully not overly simplistic examples, A) Automatic Paper Towel dispensers that don't work nearly as well with people with dark skin. (Unintentional bias on the hands of the designers) B) People getting google banned and then being totally helpless and useless and not being able to get themselves unbanned (an algorithm determined there was abuse, and there is no recourse to tell the machine that it was mistaken, the entire thing was automated away via algorithms.)

r00fus 7y ago

So two competing priorities in a) giving users all the control/configurability they want and b) per OWASP and most SQL security standards “never trust user input” Explain straddling that line ... oh and the third axis is cost.

Nasrudith 7y ago
Those two are perfectly compatible actually - although doing both is harder and thus economically more expensive. Not trusting user input is about security. Verify that everything is something which they /should/ be able to do. Think say for a game's bank account. They should never be able to directly change that value without valid transactions to and from. If they can lose money by writing a check to null that is bad. If they can /gain/ money by writing a negative check to null that is /really/ bad. Also really bad is being allowed to just take money from banks of others without a validated transaction. Configurability lets them do anything that is in their rights. If they can access their bank balance and set it as part of their signature that is allowed along with say setting their crosshair to display the target HP which they already know. If they can create a custom texture of their bank balance repeated ad nauseam that is fine. So long as they are capable of that stuff already.
- Kalium 7y ago
  You can actually get emergent conflicts from this in somewhat unexpected ways. When users are enabled to export a bunch of private data in the name of choice and control, they can be encouraged to do so unwisely. Users rarely stop and read warning messages, no matter how simple and clear and straightforward they are.

sfink 7y ago

I don't see that as a contrarian opinion, at least not with my reading of the post. Bicking isn't saying you should decide exactly what your users want and give them that and only that. (Well, he's not saying to not do that either, if in a situation where that's doable...) He is simply saying that if your final conclusion is "users want more control", then you're copping out. You are giving up on figuring out what your users really want (whether they know what it is or not), and just throwing options at them in hopes that they'll be able to make something out of them. Or at least stop complaining. Giving your users control isn't bad. Not figuring out why your users are asking for more control is bad.

doomglobe 7y ago
Most users don't want control - more things to think about takes energy and time. The users that do want control, however, are likely to be the users championing your product. Take a standard transmission in automobiles - the extra control it gives isn't necessary, and a large number of consumers prefer the automatic transmission. But the racecar drivers and car lovers are much more likely to want a standard. If you have intelligent, invested users, they'll want control options, and they'll usually tell you what they should be.
- benj111 7y ago
  On the flip side my government insists on giving me a choice of schools for my child. I don't really want choice, I want the local school to be good and for my children to go there. In this view, users still don't want control, its just a symptom of the failure to provide what users do actually want. The truth is probably between these 2 outlooks
  - rubber_duck 7y ago
    > I want the local school to be good and for my children to go there That's assuming there's a universal "good" - even if a school meets some general standards one school can have a better arts program, the other might be more biased towards sports, etc. I don't see anything wrong with having choice - having a good baseline everywhere is a separate issue.

WAHa_06x36 7y ago

This is absolutely not true. This is what they ACT like, but this is not their true motivation. These companies want all control to be theirs, and they want to be free to do what they want with users and their data, and they want to spend as little time, money and effort as possible while doing it. They know they can't state this honestly, so they dress it up in excuses, rationalisations and outright lies. They KNOW they don't know better than its users. But they don't want to let users decide, because users would make decisions that go against their own profits, so they remove the choice and lie about why.

stcredzero 7y ago

A lot of the issues raised recently in our industry around privacy or algorithmic bias come from the fact that as an industry tech has the hubris to believe that it knows better than its users what is best for them. A good way to summarize, "users want control," is to note that, "users don't want to be screwed." Users don't want to be put into a situation where they have lost so much power, they don't have practical options, or the only options they have left are punitively unpleasant.

setr 7y ago

>A lot of the issues raised recently in our industry around privacy or algorithmic bias come from the fact that as an industry tech has the hubris to believe that it knows better than its users what is best for them. I don't think this is that bad, on its own. Choice is a pain in the ass, so it's useful to have things pick for you, when you don't care to choose yourself (which is very common). The main issue is that there is never offered a means to correct the algorithm, when it chooses something the user decidedly doesn't want. That is, there's no user-inputted feedback mechanism; they go for implicit tracking like "eyeball-time" to correct the algorithm, despite the user (me) being both able and willing to give an explicit answer. I want Netflix to give me better recommendations, and I'm willing to put in a bit of effort to support that. But Netflix doesn't want me to help them help me. And thats just stupid.

webmaven 7y ago
Netflix used to have those explicit feedback mechanisms via user star ratings, and even ran a contest to improve their recommendations based on the data[0], but it turns out "that explicit star ratings were less relevant than other signals. Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries."[1] Changing from stars to thumbs increased the quantity of data (at the expense of granularity), but didn't change the relative advantage of tracking behavior over explicit ratings. Now, your ratings may be free of these issues, but that data isn't very useful in isolation, so blame the bad data from other users for the removal of the feedback mechanism. [0] https://www.netflixprize.com/ https://www.netflixprize.com/ [1] https://variety.com/2017/digital/news/netflix-thumbs-vs-stars-1202010492/ https://variety.com/2017/digital/news/netflix-thumbs-vs-star...