5 ms·

Having interviewed people who worked at Facebook, I'm convinced that nobody there really understands how it works. I agree, but far from being an unique or sur

Having interviewed people who worked at Facebook, I'm convinced that nobody there really understands how it works. I agree, but far from being an unique or surprising property of Facebook and the way it (as a company) works, I'd say it's almost in the nature of any system built by a large enough group of people that "nobody there really understands how it works".

spion 9y ago
This is not about the size of the system. You really cannot predict how the system will work, since its observing input from the outside and constructing its actions based on it. Sure, there are ways to ask some ML systems why they made a decision after the fact, and they can elaborate which variables had the most effect. But before the algorithm gets the training data, you don't know what it will decide - nobody does! It picks actions that maximise a value it was told to maximise. It does this based on "well, when I encountered a situation like this other times, I did these 5 things, and this one thing maximised that value I'm told to maximise in most of those cases, so I'll do that. At the beginning it will try random things. After several iterations, it will find things that "stick" i.e. maximise the value. There are sophisticated techniques to get unstuck from local maximums and minimums, too, in order to find even bigger maximums. When you have enough data, enough users, enough actions in a system, you can very quickly try a lot of things and learn a lot of stuff. But what those stuff would be, no human can predict. The main problem is that its unethical to train a machine learning system with a value function that doesn't take into account basic ethical behaviour. There are 3 challenges here The first challenge is technical, and its the easiest one - how to come up with a value function that encodes additional measures of human ethics. Its easy as in, you take a bunch of ethicists, give them various situations and ask them to rate actions as ethical/unethical. Then once you have that, you try to put it into the value function, so that the system can learn some decency. All done. (If only it were that easy!) The second challenge is a business one, which is, how far are you willing to reduce your value maximisation to be ethical, and what to do if your competitor doesn't do that. One way to solve that is to have regulations for ethical behaviour of machine learning systems. The system could be held responsible for unethical actions. If such actions are reported by people, investigated by experts and found true in court, the company is held liable. Public pressure and exposure seems to help too. Perhaps we could make machine learning systems that detect unethical behaviour and call it the ML police. Citizens could agree to install the ML police add-on to help monitor and aggregate behaviour of online ML systems. (If these suggestions look silly, its because they are) The third challenge is philosophical. Until now, philosophers were content with "there is no right answer, but there have been many thoughts on what exactly is ethical". They better get their act together, because we'll need them to come up with a definite, quantifiable answer real soon. On the more optimistic side, lets hope that any generally agreed upon ethical system is better than no system at all.
- chii 9y ago
  > philosophers were content with "there is no right answer, but there have been many thoughts on what exactly is ethical". i propose that all legal actions are ethical, and all illegal actions unethical. There shouldn't be actions that are considered unethical, but legal (and, for completeness's sake, no ethical action is illegal either). Then, society will need to continue to codify their ethical framework as law.
  - chaosite 9y ago
    Absolutely not. Laws are based on ethics, but not everything that is unethical is also unlawful. It's usually legal to lie, break promises, exaggerate, and other things that I would consider unethical. I also submit that the should remain legal and unethical. Lying to my parents about some detail of my personal life I don't wish to share is unethical, but shouldn't be illegal.
    - pvdebbe 9y ago
      And also ethics is subjective, a matter of one's taste. It wouldn't make sense to equate law and ethics just like that.
wpietri 9y ago
I agree, but I still think Facebook is unusually bad in this respect, and perhaps uniquely so. Physical products have reasonably comprehensible user experiences. That's even true for much software. Microsoft Word is a beast, but the user experience is tethered by the external reality of what people do with it. Memo writers and novel writers and PhD dissertation writers all have different needs, but it's reasonably clear what their goals are and what successful outcomes look like. There are approximately equivalent tools that can be swapped in. Facebook, on the other hand, is creating an ongoing experience. My needs are jumbled together with that of my friends, their friends, strangers, news organizations, advertisers, companies, non-profits, celebrities, would-be celebrities, and Facebook itself. And each experience is deeply individualized, adding yet more complexity. It still probably had some intellectual coherence at a much smaller size. But they have so many people working, and so many of them beavering away at optimizing semi-conflicting metrics based on so much data that I don't believe it makes sense to anybody there. Much less sense than Word makes to its creators, for sure.

spion 9y ago

This is not about the size of the system. You really cannot predict how the system will work, since its observing input from the outside and constructing its actions based on it. Sure, there are ways to ask some ML systems why they made a decision after the fact, and they can elaborate which variables had the most effect. But before the algorithm gets the training data, you don't know what it will decide - nobody does! It picks actions that maximise a value it was told to maximise. It does this based on "well, when I encountered a situation like this other times, I did these 5 things, and this one thing maximised that value I'm told to maximise in most of those cases, so I'll do that. At the beginning it will try random things. After several iterations, it will find things that "stick" i.e. maximise the value. There are sophisticated techniques to get unstuck from local maximums and minimums, too, in order to find even bigger maximums. When you have enough data, enough users, enough actions in a system, you can very quickly try a lot of things and learn a lot of stuff. But what those stuff would be, no human can predict. The main problem is that its unethical to train a machine learning system with a value function that doesn't take into account basic ethical behaviour. There are 3 challenges here The first challenge is technical, and its the easiest one - how to come up with a value function that encodes additional measures of human ethics. Its easy as in, you take a bunch of ethicists, give them various situations and ask them to rate actions as ethical/unethical. Then once you have that, you try to put it into the value function, so that the system can learn some decency. All done. (If only it were that easy!) The second challenge is a business one, which is, how far are you willing to reduce your value maximisation to be ethical, and what to do if your competitor doesn't do that. One way to solve that is to have regulations for ethical behaviour of machine learning systems. The system could be held responsible for unethical actions. If such actions are reported by people, investigated by experts and found true in court, the company is held liable. Public pressure and exposure seems to help too. Perhaps we could make machine learning systems that detect unethical behaviour and call it the ML police. Citizens could agree to install the ML police add-on to help monitor and aggregate behaviour of online ML systems. (If these suggestions look silly, its because they are) The third challenge is philosophical. Until now, philosophers were content with "there is no right answer, but there have been many thoughts on what exactly is ethical". They better get their act together, because we'll need them to come up with a definite, quantifiable answer real soon. On the more optimistic side, lets hope that any generally agreed upon ethical system is better than no system at all.

chii 9y ago
> philosophers were content with "there is no right answer, but there have been many thoughts on what exactly is ethical". i propose that all legal actions are ethical, and all illegal actions unethical. There shouldn't be actions that are considered unethical, but legal (and, for completeness's sake, no ethical action is illegal either). Then, society will need to continue to codify their ethical framework as law.
- chaosite 9y ago
  Absolutely not. Laws are based on ethics, but not everything that is unethical is also unlawful. It's usually legal to lie, break promises, exaggerate, and other things that I would consider unethical. I also submit that the should remain legal and unethical. Lying to my parents about some detail of my personal life I don't wish to share is unethical, but shouldn't be illegal.
  - pvdebbe 9y ago
    And also ethics is subjective, a matter of one's taste. It wouldn't make sense to equate law and ethics just like that.

wpietri 9y ago

I agree, but I still think Facebook is unusually bad in this respect, and perhaps uniquely so. Physical products have reasonably comprehensible user experiences. That's even true for much software. Microsoft Word is a beast, but the user experience is tethered by the external reality of what people do with it. Memo writers and novel writers and PhD dissertation writers all have different needs, but it's reasonably clear what their goals are and what successful outcomes look like. There are approximately equivalent tools that can be swapped in. Facebook, on the other hand, is creating an ongoing experience. My needs are jumbled together with that of my friends, their friends, strangers, news organizations, advertisers, companies, non-profits, celebrities, would-be celebrities, and Facebook itself. And each experience is deeply individualized, adding yet more complexity. It still probably had some intellectual coherence at a much smaller size. But they have so many people working, and so many of them beavering away at optimizing semi-conflicting metrics based on so much data that I don't believe it makes sense to anybody there. Much less sense than Word makes to its creators, for sure.