7 ms·

I won't post to say "I haven't looked at the contents of the file, but it's named 'cat.jpg'" either. I won't even post to announce that the one hundred mi

by Udo 15y ago

I won't post to say "I haven't looked at the contents of the file, but it's named 'cat.jpg'" either. I won't even post to announce that the one hundred millionth file has been stored. [...] This is because I have no way to obtain that information. The contents of files [...] is all hidden from me by Tarsnap's strong client-side encryption. I agree that cat.jpg was a privacy violation and I do believe they did more than simply look at the filename. However, I'll take the unpopular position that this is within the limits of what one can reasonably expect from a website providing a service as 37signals does. Admins will, and are completely expected to, look at the data - if only to make sure everything is working. Looking at a file called cat.jpg because it's the gazillionth file is pushing the boundary a bit, but I still think this is OK. The moment I opt to use a hosted project management software, I implicitly accept that things like this (and potentially much worse) might happen. Forgive me cperciva, but to me your post looks just like a giant plug for your own service. Client-side encryption is not warranted for everything, nor is it a reasonable goal for every app that shares data on the web. It's fine that Tarsnap does this, and frankly I would expect the same from a service like, say, DropBox - but it's not a reasonable expectation when it comes to the type of apps 37signals provides.

weavejester 15y ago
There are considerable technical hurdles in writing a web application that doesn't store unencrypted data. However, in principle, I don't see any compelling reason an admin should have access to user data like uploaded files. Ensuring an encrypted file is backed up and available for use is no different to doing the same for an unencrypted file.
- Udo 15y ago
  I don't see any compelling reason an admin should have access to user data like uploaded files. Mostly, uploaded files are stored unencrypted on a webserver though. The reason for this is that those are mostly "public" files in the sense that they can be accessed by a URL. Encrypting these does nothing really, except place additional load on the webserver when it has to decrypt them on-the-fly and admins would still be able to retrieve the key used for this from the software that is running on the server. This scenario is the most common one when it comes to user storage, and for good reason.
  - weavejester 15y ago
    Sure; that's what I meant by "considerable technical hurdles". But if these technical obstacles did not exist, if browsers had APIs to handle encrypted data for instance, would there still be a reason to give admins access to those files?
    - Udo 15y ago
      Browsers having APIs to encrypt and decrypt data in these scenarios is worth looking into - however, I do believe there might be a lot of apps that can't really profit from such an API. For example, what happens when you want to share an encrypted file with other users?
      - weavejester 15y ago
        That's what public key encryption is for, no? The server just needs to store the encrypted files and the public key of each user, and files can be shared without the server being aware of their content.
        Udo 15y ago
        To make this work, we'd need a pretty complicated system to distribute and manage those keys (and in a way they stay encrypted during transfer and storage). Furthermore, your client key pair would have to be stored on your machine, that means a service where you can't log in with just a password. Don't get me wrong, I think it's an interesting idea, I just don't see how this is not going to be a huge complexity and usability nightmare in applications that are designed for sharing data.
        weavejester 15y ago
        It's not complicated to distribute public keys. They're public keys, so put them in a public S3 bucket. Private keys are marginally more complex, but not much. If a password is sufficient security, then the private key can be stored remotely (in S3 or whatever) but encrypted (symmetrically) with a password. So say Alice wants to share a file with Bob. Alice's client downloads her encrypted private key, and prompts Alice for a password. The private key is decrypted with the password and stored in memory. Alice then downloads the file she wants to share with Bob, and decrypts it with her private key. Then she downloads Bob's public key, and re-encrypts the file with Bob's public key. She can now send the file to Bob securely without the server being aware of the content.
Dylan16807 15y ago
I will accept looking at the filename in some cases but never at the file contents.
cperciva 15y ago
I do believe they did more than simply look at the filename. We'll have to disagree there. I'd be very surprised if they did any more than looking at their log files -- most likely using tail -f -- as the 100 million mark approached. Admins will, and are completely expected to, look at the data - if only to make sure everything is working. How does looking at individual files help to confirm that things are working? Once you're operating at scale, looking at individual files doesn't tell you anything useful; if there's a big problem users will notice it before you do, and if there's a small problem the files you look at probably won't be in the affected set. Forgive me cperciva, but to me your post looks just like a giant plug for your own service. Was I plugging Tarsnap? Sure; I mention it every chance I get on my blog. But I didn't write that post because I wanted to plug Tarsnap; I wrote it because I saw the trust-is-fragile post on HN Daily and felt that revising their privacy policy wasn't the right response. (If I had noticed that post when it was first discussed here, that blog post would probably have been just a comment -- but since I was about 24 hours late to the party I figured that nobody would read a comment I made here.)
- Udo 15y ago
  We'll have to disagree there. I'd be very surprised if they did any more than looking at their log files I'm probably a little less trusting on this. An admin seeing the filename in the logfile and just calling its URL out of curiosity seems like a very likely scenario to me. They said something like "...and it was a picture of a cat", not "...and it was named cat.jpg". How does looking at individual files help to confirm that things are working? Not in this case, but having access to the file storage system per se is common and useful. But I didn't write that post because I wanted to plug Tarsnap; I understand. The combination of pointing the finger at someone for wrongdoing and then asserting your own superiority seemed inappropriate to me though. I understand where you're coming from, but I also believe to you the world is now full of places that should have client-side encryption, when in fact I don't think this is a good fit for what 37signals does at all. I wrote it because I saw the trust-is-fragile post on HN Daily and felt that revising their privacy policy wasn't the right response. I'm not a 37signals user, and I haven't read their policy. I agree that changing the policy following this incident is very bad timing, but I think this maneuver does correct an unreasonable expectation users might have.
latch 15y ago
I totally disagree that this came off sounding like a plug for Tarsnap. This is one of those perspective-changing point which can only be made by using concrete examples. That he happens to be an expert in the field of digital privacy and has a way to prove that he is such an expert shouldn't be held against him. Reading his post was like an "oh shit, he's right" moment for me and using Tarsnap as an example was key in helping me understand it.
nbashaw 15y ago
It may be normal now, but it won't be for long. Sunir said it better than I could: http://news.ycombinator.com/item?id=3471464 http://news.ycombinator.com/item?id=3471464
scott_s 15y ago
Forgive me cperciva, but to me your post looks just like a giant plug for your own service. Considering that in the discussion around cat.jpg, many people here were talking about a secure back-up service which encrypts all data at the client side with auditable source-code as if it was an unrealistic, unobtainable goal, I have zero problems with that.
mc32 15y ago
I just wonder why they didn't say it was made up -a joke with no basis in reality. Just some poetic license. Would that be worse than admitting/pretending they actually saw a file called cat.jpg? If there was such a file, it could have been a JPG for catalog of some kind, etc. I think they are responding to people's first expectations and that there was actually a file with the image of a cat. I doubt it and think it was just an attempt at being funny which backfired and they felt they had to take responsibility for the perceived breach of trust and that any other explanation, even if truthful would have been seen as a weak excuse.
- foreverbanned 15y ago
  You may be right. When you think about it, what were the odds that the 100,000,000th file would be something funny worth mentionning in a post?

by Udo 15y ago

weavejester 15y ago

There are considerable technical hurdles in writing a web application that doesn't store unencrypted data. However, in principle, I don't see any compelling reason an admin should have access to user data like uploaded files. Ensuring an encrypted file is backed up and available for use is no different to doing the same for an unencrypted file.

Udo 15y ago
I don't see any compelling reason an admin should have access to user data like uploaded files. Mostly, uploaded files are stored unencrypted on a webserver though. The reason for this is that those are mostly "public" files in the sense that they can be accessed by a URL. Encrypting these does nothing really, except place additional load on the webserver when it has to decrypt them on-the-fly and admins would still be able to retrieve the key used for this from the software that is running on the server. This scenario is the most common one when it comes to user storage, and for good reason.
- weavejester 15y ago
  Sure; that's what I meant by "considerable technical hurdles". But if these technical obstacles did not exist, if browsers had APIs to handle encrypted data for instance, would there still be a reason to give admins access to those files?
  - Udo 15y ago
    Browsers having APIs to encrypt and decrypt data in these scenarios is worth looking into - however, I do believe there might be a lot of apps that can't really profit from such an API. For example, what happens when you want to share an encrypted file with other users?
    - weavejester 15y ago
      That's what public key encryption is for, no? The server just needs to store the encrypted files and the public key of each user, and files can be shared without the server being aware of their content.
      - Udo 15y ago
        To make this work, we'd need a pretty complicated system to distribute and manage those keys (and in a way they stay encrypted during transfer and storage). Furthermore, your client key pair would have to be stored on your machine, that means a service where you can't log in with just a password. Don't get me wrong, I think it's an interesting idea, I just don't see how this is not going to be a huge complexity and usability nightmare in applications that are designed for sharing data.
        weavejester 15y ago
        It's not complicated to distribute public keys. They're public keys, so put them in a public S3 bucket. Private keys are marginally more complex, but not much. If a password is sufficient security, then the private key can be stored remotely (in S3 or whatever) but encrypted (symmetrically) with a password. So say Alice wants to share a file with Bob. Alice's client downloads her encrypted private key, and prompts Alice for a password. The private key is decrypted with the password and stored in memory. Alice then downloads the file she wants to share with Bob, and decrypts it with her private key. Then she downloads Bob's public key, and re-encrypts the file with Bob's public key. She can now send the file to Bob securely without the server being aware of the content.

Dylan16807 15y ago

I will accept looking at the filename in some cases but never at the file contents.

cperciva 15y ago

I do believe they did more than simply look at the filename. We'll have to disagree there. I'd be very surprised if they did any more than looking at their log files -- most likely using tail -f -- as the 100 million mark approached. Admins will, and are completely expected to, look at the data - if only to make sure everything is working. How does looking at individual files help to confirm that things are working? Once you're operating at scale, looking at individual files doesn't tell you anything useful; if there's a big problem users will notice it before you do, and if there's a small problem the files you look at probably won't be in the affected set. Forgive me cperciva, but to me your post looks just like a giant plug for your own service. Was I plugging Tarsnap? Sure; I mention it every chance I get on my blog. But I didn't write that post because I wanted to plug Tarsnap; I wrote it because I saw the trust-is-fragile post on HN Daily and felt that revising their privacy policy wasn't the right response. (If I had noticed that post when it was first discussed here, that blog post would probably have been just a comment -- but since I was about 24 hours late to the party I figured that nobody would read a comment I made here.)

Udo 15y ago
We'll have to disagree there. I'd be very surprised if they did any more than looking at their log files I'm probably a little less trusting on this. An admin seeing the filename in the logfile and just calling its URL out of curiosity seems like a very likely scenario to me. They said something like "...and it was a picture of a cat", not "...and it was named cat.jpg". How does looking at individual files help to confirm that things are working? Not in this case, but having access to the file storage system per se is common and useful. But I didn't write that post because I wanted to plug Tarsnap; I understand. The combination of pointing the finger at someone for wrongdoing and then asserting your own superiority seemed inappropriate to me though. I understand where you're coming from, but I also believe to you the world is now full of places that should have client-side encryption, when in fact I don't think this is a good fit for what 37signals does at all. I wrote it because I saw the trust-is-fragile post on HN Daily and felt that revising their privacy policy wasn't the right response. I'm not a 37signals user, and I haven't read their policy. I agree that changing the policy following this incident is very bad timing, but I think this maneuver does correct an unreasonable expectation users might have.

latch 15y ago

I totally disagree that this came off sounding like a plug for Tarsnap. This is one of those perspective-changing point which can only be made by using concrete examples. That he happens to be an expert in the field of digital privacy and has a way to prove that he is such an expert shouldn't be held against him. Reading his post was like an "oh shit, he's right" moment for me and using Tarsnap as an example was key in helping me understand it.

nbashaw 15y ago

It may be normal now, but it won't be for long. Sunir said it better than I could: http://news.ycombinator.com/item?id=3471464 http://news.ycombinator.com/item?id=3471464

scott_s 15y ago

Forgive me cperciva, but to me your post looks just like a giant plug for your own service. Considering that in the discussion around cat.jpg, many people here were talking about a secure back-up service which encrypts all data at the client side with auditable source-code as if it was an unrealistic, unobtainable goal, I have zero problems with that.

mc32 15y ago

I just wonder why they didn't say it was made up -a joke with no basis in reality. Just some poetic license. Would that be worse than admitting/pretending they actually saw a file called cat.jpg? If there was such a file, it could have been a JPG for catalog of some kind, etc. I think they are responding to people's first expectations and that there was actually a file with the image of a cat. I doubt it and think it was just an attempt at being funny which backfired and they felt they had to take responsibility for the perceived breach of trust and that any other explanation, even if truthful would have been seen as a weak excuse.

foreverbanned 15y ago
You may be right. When you think about it, what were the odds that the 100,000,000th file would be something funny worth mentionning in a post?