4 ms·

I just use it to upload daily encrypted backups of my mail-server (< 500MB per month)... so I wouldn't mind if they set some reasonable limit to encrypted uploa

I just use it to upload daily encrypted backups of my mail-server (< 500MB per month)... so I wouldn't mind if they set some reasonable limit to encrypted uploads (say 1-10TB). I feel like anyone who's actually uploading personal content and who isn't uploading media files that are amenable to deduplication would be comfortable with some threshold as well.

wolfgke 9y ago
> so I wouldn't mind if they set some reasonable limit to encrypted uploads (say 1-10TB). How is Amazon supposed to distinguish between encrypted and non-encrypted data that you upload?
- pinum 9y ago
  A heuristic. No common magic header and poor compressibility? Likely encrypted.
  - sillysaurus3 9y ago
    Since encrypted data is indistinguishable from random noise, I think poor compressibility is actually zero compressibility, isn't it? There are tools to search for TrueCrypt / other encrypted partitions on disks, so it's a solved problem to detect encrypted data. It would be unfortunate if services ban the ability to upload encrypted secrets, though. On the other hand, that'd be good for Tarsnap. I wonder how much it'd cost to store 10TB on it?
    - icebraining 9y ago
      Since encrypted data is indistinguishable from random noise, I think poor compressibility is actually zero compressibility, isn't it? I don't think so. Random noise can take any form - even of a string composed entirely of zeros, which would be trivially compressed. It's just very unlikely that it'll actually be compressible.
      - Dylan16807 9y ago
        Random noise can't really take any form when you apply the implicit restriction of your search fitting within finite time and space.
        icebraining 9y ago
        I don't get what you're saying. Searching? For what?
        sillysaurus3 9y ago
        It'd take longer than the universe has left to find a string of N zeroes by generating random numbers, for sufficiently large N. And N is surprisingly small.
        icebraining 9y ago
        Sure, but not zero.
        Dylan16807 9y ago
        Yes zero. As zero as zero can possibly be, measuring with the most precise instruments possible. There's an infinitely better chance of both of us being struck by lightning and imagining you found such a number randomly. It wouldn't be zero in certain math worlds. It is zero in the real universe.
        Dylan16807 9y ago
        The definition of a random number is in the process of generation. You have to actually generate random numbers if you want to have a random number that meets some criteria. And even with impossibly vast resources, you will never find a random megabyte that compresses well.
        icebraining 9y ago
    - dorgo 9y ago
      The first billion of digits of Pi might look pretty random, but there might be a short program which genrates them - which can be considered a compressed form. In general, it is impossible to decide how good a given string might be compressed. https://en.wikipedia.org/wiki/Kolmogorov_complexity https://en.wikipedia.org/wiki/Kolmogorov_complexity
      - foota 9y ago
        Might be a short program that generates them? I'm going to go ahead and file that in the understate of the century folder.

wolfgke 9y ago

> so I wouldn't mind if they set some reasonable limit to encrypted uploads (say 1-10TB). How is Amazon supposed to distinguish between encrypted and non-encrypted data that you upload?

pinum 9y ago
A heuristic. No common magic header and poor compressibility? Likely encrypted.
- sillysaurus3 9y ago
  Since encrypted data is indistinguishable from random noise, I think poor compressibility is actually zero compressibility, isn't it? There are tools to search for TrueCrypt / other encrypted partitions on disks, so it's a solved problem to detect encrypted data. It would be unfortunate if services ban the ability to upload encrypted secrets, though. On the other hand, that'd be good for Tarsnap. I wonder how much it'd cost to store 10TB on it?
  - icebraining 9y ago
    Since encrypted data is indistinguishable from random noise, I think poor compressibility is actually zero compressibility, isn't it? I don't think so. Random noise can take any form - even of a string composed entirely of zeros, which would be trivially compressed. It's just very unlikely that it'll actually be compressible.
    - Dylan16807 9y ago
      Random noise can't really take any form when you apply the implicit restriction of your search fitting within finite time and space.
      - icebraining 9y ago
        I don't get what you're saying. Searching? For what?
        sillysaurus3 9y ago
        It'd take longer than the universe has left to find a string of N zeroes by generating random numbers, for sufficiently large N. And N is surprisingly small.
        icebraining 9y ago
        Sure, but not zero.
        Dylan16807 9y ago
        Yes zero. As zero as zero can possibly be, measuring with the most precise instruments possible. There's an infinitely better chance of both of us being struck by lightning and imagining you found such a number randomly. It wouldn't be zero in certain math worlds. It is zero in the real universe.
        Dylan16807 9y ago
        The definition of a random number is in the process of generation. You have to actually generate random numbers if you want to have a random number that meets some criteria. And even with impossibly vast resources, you will never find a random megabyte that compresses well.
        icebraining 9y ago
  - dorgo 9y ago
    The first billion of digits of Pi might look pretty random, but there might be a short program which genrates them - which can be considered a compressed form. In general, it is impossible to decide how good a given string might be compressed. https://en.wikipedia.org/wiki/Kolmogorov_complexity https://en.wikipedia.org/wiki/Kolmogorov_complexity
    - foota 9y ago
      Might be a short program that generates them? I'm going to go ahead and file that in the understate of the century folder.