5 ms·

It requires just one developer and a couple of reviewers to make poor choices. Which begs the question, how do you structure your organisation such that a fool

It requires just one developer and a couple of reviewers to make poor choices. Which begs the question, how do you structure your organisation such that a foolish developer that only barely understands the change that they are making can't write code that makes arbitrary queries to particular data sets in unapproved contexts?

a_imho 8y ago
But you actually say it requires negligence on part of multiple people. I find it very hard to apply Hanlon's razor in this case. Anyway, from my experience people do the stupidest/reckless things despite being told not to, just because there is no fear of backlash. Holding people liable for their actions could be a start, but who wants that? Also, don't hire foolish developers in the first place. Policing around pebkac is hard.
- hopler 8y ago
  Have you ever worked for an organization? 3 people making mistakes on a project, coupled with bystander effects, is par for the course. Being a million times wealthier isn't a million times smarter. We're still talking about how Boeing accidentally crashed a few jetplanes full of people, which required dozens of people to screw up.
AnthonyMouse 8y ago
Start by keeping the primary copy of the user's data on the user's own device so that the developers never have access to it to begin with. Then, if you ever have to hold a copy of the user's data, make sure it's encrypted by the client and your servers are never in possession of the plaintext. To access the user's data, your developers should have to intentionally crack the user's password. And if they attempt to do that they should be fired. Obviously this is not how Facebook works, but ideally it's how the thing that replaces Facebook will work.
- emerongi 8y ago
  There should be a name for this sort of software design. It's not just encrypted/privacy-oriented or whatever. It's a software design with a clear contract on who owns the data: the user. E.g. Google Drive, which claims to take privacy seriously and also encrypts your data. But the data is not encrypted with a secret unknown to the server. How should my family members differentiate between the encryption Google claims it has and client-side encryption? For them it's all the same. Maybe we need some commonly understandable name that a regular user can look at and know that this software is data-agnostic.
- sonnyblarney 8y ago
  "Start by keeping the primary copy of the user's data on the user's own device" It's an organizational policy, procedural, ethics and legal question - not a technical one. They should have feature reviews before the code reviews. The feature review panel puts bounds on what the code can do.
  - brianpgordon 8y ago
    I don't think you need a board to stop this specific case. It's pretty obvious that what they were doing is unacceptable. The problem must have been a pervasive culture of lack of respect for privacy at Facebook, not a single engineer who somehow just didn't know any better.
    - sonnyblarney 8y ago
      Situations can get complicated. There might have been some side show reason to do this or that. Without oversight, some things will fall through the cracks. A review board would a) give clear direction b) catch problems and c) put accountability where it belongs.
  - AnthonyMouse 8y ago
    > It's an organizational policy, procedural, ethics and legal question - not a technical one. Not really. You have to fall back to those things when a good technical solution isn't available, but sometimes it is. Suppose you have a car, and four children. You can enact all the laws and policies and procedures you like, you can lecture the kids not to misbehave a thousand times. But the most important thing you can do, if you really don't want them out joyriding in the street, is to not give them the keys to the car.

a_imho 8y ago

But you actually say it requires negligence on part of multiple people. I find it very hard to apply Hanlon's razor in this case. Anyway, from my experience people do the stupidest/reckless things despite being told not to, just because there is no fear of backlash. Holding people liable for their actions could be a start, but who wants that? Also, don't hire foolish developers in the first place. Policing around pebkac is hard.

hopler 8y ago
Have you ever worked for an organization? 3 people making mistakes on a project, coupled with bystander effects, is par for the course. Being a million times wealthier isn't a million times smarter. We're still talking about how Boeing accidentally crashed a few jetplanes full of people, which required dozens of people to screw up.

AnthonyMouse 8y ago

Start by keeping the primary copy of the user's data on the user's own device so that the developers never have access to it to begin with. Then, if you ever have to hold a copy of the user's data, make sure it's encrypted by the client and your servers are never in possession of the plaintext. To access the user's data, your developers should have to intentionally crack the user's password. And if they attempt to do that they should be fired. Obviously this is not how Facebook works, but ideally it's how the thing that replaces Facebook will work.

emerongi 8y ago
There should be a name for this sort of software design. It's not just encrypted/privacy-oriented or whatever. It's a software design with a clear contract on who owns the data: the user. E.g. Google Drive, which claims to take privacy seriously and also encrypts your data. But the data is not encrypted with a secret unknown to the server. How should my family members differentiate between the encryption Google claims it has and client-side encryption? For them it's all the same. Maybe we need some commonly understandable name that a regular user can look at and know that this software is data-agnostic.
sonnyblarney 8y ago
"Start by keeping the primary copy of the user's data on the user's own device" It's an organizational policy, procedural, ethics and legal question - not a technical one. They should have feature reviews before the code reviews. The feature review panel puts bounds on what the code can do.
- brianpgordon 8y ago
  I don't think you need a board to stop this specific case. It's pretty obvious that what they were doing is unacceptable. The problem must have been a pervasive culture of lack of respect for privacy at Facebook, not a single engineer who somehow just didn't know any better.
  - sonnyblarney 8y ago
    Situations can get complicated. There might have been some side show reason to do this or that. Without oversight, some things will fall through the cracks. A review board would a) give clear direction b) catch problems and c) put accountability where it belongs.
- AnthonyMouse 8y ago
  > It's an organizational policy, procedural, ethics and legal question - not a technical one. Not really. You have to fall back to those things when a good technical solution isn't available, but sometimes it is. Suppose you have a car, and four children. You can enact all the laws and policies and procedures you like, you can lecture the kids not to misbehave a thousand times. But the most important thing you can do, if you really don't want them out joyriding in the street, is to not give them the keys to the car.