6 ms·

Good System Administrators are Blind

27 points by duck 16y ago

jnoller 16y ago
jacquesm is completely correct - for example, at the company I currently work for we have very strict policies around this, and also how things are encrypted/etc. We don't want access to user data, files or say - encryption keys. Having access not only violates the trust of the user(s), it fundamentally exposes us to liability we do not want. Anyone and everyone who has access to account data - no matter how trivial that data may seem to you needs to treat that data as extremely sensitive and with the utmost respect and those that mistreat it, or abuse that power should be grossly punished.
- bbbccc 16y ago
  depending on the state and the type of service you offer, you are responsible for the data on your systems. allowing users to upload arbitrary encrypted data may be a foolish and potentially costly policy. this is one reason why many services don't allow users to upload arbitrary data
chuhnk 16y ago
I'm a system administrator at a startup now going on 3-4 years. Having complete access and control over every piece of data in the company is a great responsibility. You are given a level of trust above and beyond anyone else. Misuse of your position and power is wrong and says a lot about the type of person you are.
- tomjen3 16y ago
  Absolutely. Maybe system administrators should have a swearing in ceremony in which they take an official oath to protect their users privacy, in front of their future colleagues? In practice it does little to change anything, but I imagine it would make quite an impression - especially if they where awarded a special system administrators badge afterwards as a sign of their commitment and agreement to honor the trust the people had shown them?
  - jauer 16y ago
    Already exists, somewhat, in the SAGE Code of Ethis: http://www.sage.org/ethics/ http://www.sage.org/ethics/
    - tomjen3 16y ago
      Interesting, but ultimately too long, too wordy and too general. >I will avoid conflicts of interest and biases whenever possible. When my advice is sought, if I have a conflict of interest or bias, I will declare it if appropriate, and recuse myself if necessary. I count 6 excuses you could mentally use, just from the paragraph I quoted. In addition, you need something that is more tangible than just a piece of paper to hang on the wall. Preferably it should be something that would always be visible, and be a constant reminder of your duties and the trust that had been placed in your by the other people.
      - jauer 16y ago
        > In addition, you need something that is more tangible than just a piece of paper to hang on the wall. Preferably it should be something that would always be visible, and be a constant reminder of your duties and the trust that had been placed in your by the other people. A sysadmin clerical collar? Are there any professions besides priest (confession) where the worker has similar access to information?
  - gaius 16y ago
    Yeah, sysadmins should be like doctors or priests :-)
bbbccc 16y ago
okay jacques - you say in your post that in this case, you would have not only fired the employee, but "charged" them. well first of all, it is law enforcement that charges people secondly, charge them with what? any site that allows you to upload data is going to have users sign a user-agreement that stipulates examination by company employees (to do otherwise would be insane). in this case one could suggest that the engineer was stalking...thats one thing. but to suggest that db admins seeing what is in a row of data is somehow illegal is insane
msisk6 16y ago
Sometimes, though, life is more complicated. For example, if you're a sysadmin and end up on the receiving end of a federal subpoena (as I have), you find your self suddenly thrust into the uncomfortable role of actually having to spy on your users and sift through their personal info searching for requested bits of information. And that sucks. I bet Google and Facebook get hundreds of such requests a day. While they have automated systems to deal with some of this, at some point some poor sysadmin has to get his hands dirty and sift through logs and data to fulfill the requirements of the subpoena. During which you may be subjected to things you just don't want to see or know about. Blindness is a good trait in a sysadmin, but so is forgetfulness and keeping one's mouth shut.
- tomjen3 16y ago
  Make sure your users data is encrypted - then you don't have to worry about that.
  - 3pt14159 16y ago
    I believe this as well, but with the bare minimum product philosophy it is tough to build around.
tptacek 16y ago
Bull. I was a sysadmin. I have worked with untold numbers of sysasmins. All of them were united in being human beings with moods and occasional lapses in judgements. All of them worked a job that involved the manipulation of symbols and abstractions for the purpose of assisting other people in their jobs of manipulating symbols and abstractions. Whatever is my point? you are surely asking yourself. It is this: good system designers engineer their systems around the assumption that sysadmins might screw up, and mitigate the damages. Sometimes that means explicit confidentiality measures in logs, or in obfuscation or outright encryption of user data. More often it means that the only way to even access logs or spools is to stage through a management server (for instance via SSH forwarding) that is heavily audited. Some people have "two key" systems that require admins to cross-check each other before potentially coming into contact with hazmat. This is what we should discussing. We shouldn't be talking about how to hire sysadmins who "self-censor" themselves; it's hard enough to hire for technical competence at Google's scale.
- dedward 16y ago
  Sure, we're all human - but I noticed through my career that those who abused their privilegs didn't tend to last long, and those who had a strong ethic as to what was appropriate and not appropriate (ie: myself) tended to be trusted with quite a bit - often quite outside the scope of my job title itself. yes, there are systems that should enforce this properly - you can't build an empire on raw trust - but for the small-time systems administrator out their working throug your career - this principle should always stick with you. Respect users data. Don't ever BRAG about the access you have. People, both suits and workers, don't like to be reminded that actually you can see record and mess with their mail - everyone is happier if you don't tell them, and don't do it (and, yes, at some point set up controls to make this harder to do) I've turned away enough managers and directors, even when I agreed with them in one case on a deep personal level, and would have faced no consequnces for looking it up - I still did the right thing and insisted that if they wanted to snoop through employee's email, that I'd only do it through the OK of HR or specific others. I was firm and polite, and pointed out that I'd do the same if someone other than (let's say the owner) walked in and asked for his email. He walked away pissed. He came by an hour or so later and said, "you know, I really admire your ethics on this - that was the right thing to do, and not many people would stick to it like that - so no worries about hard feelings or anthing like that."
Vivtek 16y ago
I think you mean "recuse yourself".
- jacquesm 16y ago
  Fixed, thank you!

27 points by duck 16y ago

jacquesm is completely correct - for example, at the company I currently work for we have very strict policies around this, and also how things are encrypted/etc. We don't want access to user data, files or say - encryption keys. Having access not only violates the trust of the user(s), it fundamentally exposes us to liability we do not want. Anyone and everyone who has access to account data - no matter how trivial that data may seem to you needs to treat that data as extremely sensitive and with the utmost respect and those that mistreat it, or abuse that power should be grossly punished.

bbbccc 16y ago
depending on the state and the type of service you offer, you are responsible for the data on your systems. allowing users to upload arbitrary encrypted data may be a foolish and potentially costly policy. this is one reason why many services don't allow users to upload arbitrary data

chuhnk 16y ago

I'm a system administrator at a startup now going on 3-4 years. Having complete access and control over every piece of data in the company is a great responsibility. You are given a level of trust above and beyond anyone else. Misuse of your position and power is wrong and says a lot about the type of person you are.

tomjen3 16y ago
Absolutely. Maybe system administrators should have a swearing in ceremony in which they take an official oath to protect their users privacy, in front of their future colleagues? In practice it does little to change anything, but I imagine it would make quite an impression - especially if they where awarded a special system administrators badge afterwards as a sign of their commitment and agreement to honor the trust the people had shown them?
- jauer 16y ago
  Already exists, somewhat, in the SAGE Code of Ethis: http://www.sage.org/ethics/ http://www.sage.org/ethics/
  - tomjen3 16y ago
    Interesting, but ultimately too long, too wordy and too general. >I will avoid conflicts of interest and biases whenever possible. When my advice is sought, if I have a conflict of interest or bias, I will declare it if appropriate, and recuse myself if necessary. I count 6 excuses you could mentally use, just from the paragraph I quoted. In addition, you need something that is more tangible than just a piece of paper to hang on the wall. Preferably it should be something that would always be visible, and be a constant reminder of your duties and the trust that had been placed in your by the other people.
    - jauer 16y ago
      > In addition, you need something that is more tangible than just a piece of paper to hang on the wall. Preferably it should be something that would always be visible, and be a constant reminder of your duties and the trust that had been placed in your by the other people. A sysadmin clerical collar? Are there any professions besides priest (confession) where the worker has similar access to information?
- gaius 16y ago
  Yeah, sysadmins should be like doctors or priests :-)

bbbccc 16y ago

okay jacques - you say in your post that in this case, you would have not only fired the employee, but "charged" them. well first of all, it is law enforcement that charges people secondly, charge them with what? any site that allows you to upload data is going to have users sign a user-agreement that stipulates examination by company employees (to do otherwise would be insane). in this case one could suggest that the engineer was stalking...thats one thing. but to suggest that db admins seeing what is in a row of data is somehow illegal is insane

msisk6 16y ago

Sometimes, though, life is more complicated. For example, if you're a sysadmin and end up on the receiving end of a federal subpoena (as I have), you find your self suddenly thrust into the uncomfortable role of actually having to spy on your users and sift through their personal info searching for requested bits of information. And that sucks. I bet Google and Facebook get hundreds of such requests a day. While they have automated systems to deal with some of this, at some point some poor sysadmin has to get his hands dirty and sift through logs and data to fulfill the requirements of the subpoena. During which you may be subjected to things you just don't want to see or know about. Blindness is a good trait in a sysadmin, but so is forgetfulness and keeping one's mouth shut.

tomjen3 16y ago
Make sure your users data is encrypted - then you don't have to worry about that.
- 3pt14159 16y ago
  I believe this as well, but with the bare minimum product philosophy it is tough to build around.

tptacek 16y ago

Bull. I was a sysadmin. I have worked with untold numbers of sysasmins. All of them were united in being human beings with moods and occasional lapses in judgements. All of them worked a job that involved the manipulation of symbols and abstractions for the purpose of assisting other people in their jobs of manipulating symbols and abstractions. Whatever is my point? you are surely asking yourself. It is this: good system designers engineer their systems around the assumption that sysadmins might screw up, and mitigate the damages. Sometimes that means explicit confidentiality measures in logs, or in obfuscation or outright encryption of user data. More often it means that the only way to even access logs or spools is to stage through a management server (for instance via SSH forwarding) that is heavily audited. Some people have "two key" systems that require admins to cross-check each other before potentially coming into contact with hazmat. This is what we should discussing. We shouldn't be talking about how to hire sysadmins who "self-censor" themselves; it's hard enough to hire for technical competence at Google's scale.

dedward 16y ago
Sure, we're all human - but I noticed through my career that those who abused their privilegs didn't tend to last long, and those who had a strong ethic as to what was appropriate and not appropriate (ie: myself) tended to be trusted with quite a bit - often quite outside the scope of my job title itself. yes, there are systems that should enforce this properly - you can't build an empire on raw trust - but for the small-time systems administrator out their working throug your career - this principle should always stick with you. Respect users data. Don't ever BRAG about the access you have. People, both suits and workers, don't like to be reminded that actually you can see record and mess with their mail - everyone is happier if you don't tell them, and don't do it (and, yes, at some point set up controls to make this harder to do) I've turned away enough managers and directors, even when I agreed with them in one case on a deep personal level, and would have faced no consequnces for looking it up - I still did the right thing and insisted that if they wanted to snoop through employee's email, that I'd only do it through the OK of HR or specific others. I was firm and polite, and pointed out that I'd do the same if someone other than (let's say the owner) walked in and asked for his email. He walked away pissed. He came by an hour or so later and said, "you know, I really admire your ethics on this - that was the right thing to do, and not many people would stick to it like that - so no worries about hard feelings or anthing like that."

Vivtek 16y ago

I think you mean "recuse yourself".

jacquesm 16y ago
Fixed, thank you!