4 ms·

> "Furthermore, even over an encrypted connection it is not difficult to figure out which files you are downloading based on the size of the transfer" Is it re

by avar 8y ago

> "Furthermore, even over an encrypted connection it is not difficult to figure out which files you are downloading based on the size of the transfer" Is it really not difficult? I bet if you sorted all the ".deb" packages on a mirror by size a lot of them would have a similar or the same size, so you wouldn't be able to tell them apart based on the size of the dialog. Furthermore, when I update Debian I usually have to download some updates and N number of packages. I don't know if this is now done with a single keep-alive connection. If it is, then figuring out what combination of data was downloaded gets a lot harder. Finally, this out of hand dismisses a now trivial attack (just sniff URLs being downloaded with tcpdump) by pointing out that a much harder attack is theoretically possible by a really dedicated attacker. Now if you use Debian your local admin can see you're downloading Tux racer, but they're very unlikely to be dedicated enough to figure out from downloaded https sizes what package you retrieved.

ianbooker 8y ago
> "Furthermore, even over an encrypted connection it is not difficult to figure out which files you are downloading based on the size of the transfer" >> "Is it really not difficult? I bet if you sorted all the ".deb" packages on a mirror by size a lot of them would have a similar or the same size, so you wouldn't be able to tell them apart based on the size of the dialog." Human readable sizes: Sure. Byte size info: Not so much. And even if: Things would become very clear to the attacker after one update cycle for each package. If you really want to mitigate information about downloaded packages you would have to completely revamp apt to randomize package names and sizes, and also randomize read access on mirrors...
- shasheene 8y ago
  > If you really want to mitigate information about downloaded packages you would have to completely revamp apt to randomize package names and sizes, and also randomize read access on mirrors... There isn't a need to randomize package names, or randomize read access on the mirror, given fetching deb files from a remote HTTP apt repository is a series of GET requests. Randomizing order of these requests can be done completely on the client side. Package sizes are still problematic. Here's a suggestion: if each deb file was padded to nearest megabyte, and there was a handful of fixed-size files (say, 1MB, 10MB and 100MB), the apt-get client could request a suitably small number of the padding files with each download. This would improve privacy with a minimum of software changes and bandwidth wastage.
  - JdeBP 8y ago
    If each file were padded to the nearest MiB, the total download size of the packages containing the nosh toolset would increase by almost 3000% from 1.5MiB to 46MiB. No package is greater than 0.5MiB in size. I am fairly confident that this case is not an outlier. Out of the 847 packages currently in the package cache on one of my machines, 621 are less than 0.5MiB in size.
    - y4mi 8y ago
      You're arguing with a strawman. He didn't mean these file sizes specifically. it would still apply just the same with different file sizes i.e. create cutoffs every 50 or 100kbyte
      - JdeBP 8y ago
        You're abusing the notion of a straw man, which this is not. I am pointing out the consequences of Shasheene's idea as xe explicitly posited it. Xe is free to think about different sizes in turn, but needs to measure and calculate the consequences of whatever size xe then chooses. No, it would not apply the same with different sizes. Think! This is engineering, and different block sizes make different levels of trade-off. The lower the block size, for example, the fewer packages end up being the same rounded-up size and the easier it is to identify specific packages. (Hint: One hasn't thought about this properly until one has at least realized that there is a size that Debian packages are already blocked out to, by dint of their being ar archives.)
  - xorcist 8y ago
    No, don't do that. Just use tor instead. It's supported on an out of the box Debian install.
    - shasheene 8y ago
      It's still useful to be able to connect to the local mirror without tor (and enjoy the fast transfer speeds), but still mitigate privacy leaks from analysis of the transfer and timings. Transferring apt packages over tor is unlikely to ever become the default, so it's worth trying to improve the non-tor default.
j16sdiz 8y ago
If you care security, you update often. If you update often, the changeset should be small. It is easy to match in the small set.
cmsj 8y ago
> I bet if you sorted all the ".deb" packages on a mirror by size a lot of them would have a similar or the same size Why bet when you can science? :) $ rsync -r rsync://ftp.be.debian.org/debian/pool/main/ | grep "\.deb$" $ wc -l debian.txt 1246733 # total number of deb packages $ cat /tmp/debian.txt | awk '{ print $2 }' | sort | uniq -c | sort -rn | head -1 463 1044 # The most common package size has 463 occurances $ cat /tmp/debian.txt | awk '{ print $2 }' | sort | uniq -c | sort -rn | awk '{ print $1}' | grep -c "^1$" 259300 # The number of packages with a unique size $
- JdeBP 8y ago
  As I found at https://news.ycombinator.com/item?id=18960239 https://news.ycombinator.com/item?id=18960239 there can be duplication which is irrelevant for the point being discussed, as it is one version of a package duplicating another version of the same package, meaning that the size is still a unique identifier of the package. It is worth checking that.
zeroimpl 8y ago
They could also improve the download client to fix this. For example, if the download client uses the byte-range HTTP requests to download files in chunks, there is nothing stopping it from randomly requesting some additional bytes from the server. Then the attacker would have a very weak probability estimate of what was actually downloaded.

by avar 8y ago

ianbooker 8y ago

> "Furthermore, even over an encrypted connection it is not difficult to figure out which files you are downloading based on the size of the transfer" >> "Is it really not difficult? I bet if you sorted all the ".deb" packages on a mirror by size a lot of them would have a similar or the same size, so you wouldn't be able to tell them apart based on the size of the dialog." Human readable sizes: Sure. Byte size info: Not so much. And even if: Things would become very clear to the attacker after one update cycle for each package. If you really want to mitigate information about downloaded packages you would have to completely revamp apt to randomize package names and sizes, and also randomize read access on mirrors...

shasheene 8y ago
> If you really want to mitigate information about downloaded packages you would have to completely revamp apt to randomize package names and sizes, and also randomize read access on mirrors... There isn't a need to randomize package names, or randomize read access on the mirror, given fetching deb files from a remote HTTP apt repository is a series of GET requests. Randomizing order of these requests can be done completely on the client side. Package sizes are still problematic. Here's a suggestion: if each deb file was padded to nearest megabyte, and there was a handful of fixed-size files (say, 1MB, 10MB and 100MB), the apt-get client could request a suitably small number of the padding files with each download. This would improve privacy with a minimum of software changes and bandwidth wastage.
- JdeBP 8y ago
  If each file were padded to the nearest MiB, the total download size of the packages containing the nosh toolset would increase by almost 3000% from 1.5MiB to 46MiB. No package is greater than 0.5MiB in size. I am fairly confident that this case is not an outlier. Out of the 847 packages currently in the package cache on one of my machines, 621 are less than 0.5MiB in size.
  - y4mi 8y ago
    You're arguing with a strawman. He didn't mean these file sizes specifically. it would still apply just the same with different file sizes i.e. create cutoffs every 50 or 100kbyte
    - JdeBP 8y ago
      You're abusing the notion of a straw man, which this is not. I am pointing out the consequences of Shasheene's idea as xe explicitly posited it. Xe is free to think about different sizes in turn, but needs to measure and calculate the consequences of whatever size xe then chooses. No, it would not apply the same with different sizes. Think! This is engineering, and different block sizes make different levels of trade-off. The lower the block size, for example, the fewer packages end up being the same rounded-up size and the easier it is to identify specific packages. (Hint: One hasn't thought about this properly until one has at least realized that there is a size that Debian packages are already blocked out to, by dint of their being ar archives.)
- xorcist 8y ago
  No, don't do that. Just use tor instead. It's supported on an out of the box Debian install.
  - shasheene 8y ago
    It's still useful to be able to connect to the local mirror without tor (and enjoy the fast transfer speeds), but still mitigate privacy leaks from analysis of the transfer and timings. Transferring apt packages over tor is unlikely to ever become the default, so it's worth trying to improve the non-tor default.

j16sdiz 8y ago

If you care security, you update often. If you update often, the changeset should be small. It is easy to match in the small set.

cmsj 8y ago

> I bet if you sorted all the ".deb" packages on a mirror by size a lot of them would have a similar or the same size Why bet when you can science? :) $ rsync -r rsync://ftp.be.debian.org/debian/pool/main/ | grep "\.deb$" $ wc -l debian.txt 1246733 # total number of deb packages $ cat /tmp/debian.txt | awk '{ print $2 }' | sort | uniq -c | sort -rn | head -1 463 1044 # The most common package size has 463 occurances $ cat /tmp/debian.txt | awk '{ print $2 }' | sort | uniq -c | sort -rn | awk '{ print $1}' | grep -c "^1$" 259300 # The number of packages with a unique size $

JdeBP 8y ago
As I found at https://news.ycombinator.com/item?id=18960239 https://news.ycombinator.com/item?id=18960239 there can be duplication which is irrelevant for the point being discussed, as it is one version of a package duplicating another version of the same package, meaning that the size is still a unique identifier of the package. It is worth checking that.

zeroimpl 8y ago

They could also improve the download client to fix this. For example, if the download client uses the byte-range HTTP requests to download files in chunks, there is nothing stopping it from randomly requesting some additional bytes from the server. Then the attacker would have a very weak probability estimate of what was actually downloaded.