4 ms·

Anyone else notice that Dropbox globally indexes files? If you upload a large file already elsewhere on their servers (Linux distro ISO, scene-made TVRip or DV

Anyone else notice that Dropbox globally indexes files? If you upload a large file already elsewhere on their servers (Linux distro ISO, scene-made TVRip or DVDRip, etc), it "uploads" instantly, and will begin downloading immediately on your other linked Dropbox machines. Makes me wonder how hard it is to download a file by hash (via the Linux client, parts of which are open-source), without having the file, or how easy it would be for HBO or some other media company to say "here are the hashes for scene rips of The Wire--please furnish matching account emails, thanks".

andrewljohnson 17y ago
Apple does this with iTunes/MobileMe, and any cloud storage player is aiming for the same network effect. The cloud storage play is huge for Apple... first they sell you the video, then they sell you the space to store the video, but it doesn't actually take them anymore hard drive space. And as the media gets bigger and bigger (HD today, who knows tomorrow), Apple sells you more and more storage "space" and their costs only go up marginally. It's the stuff billion dollar business plans are made of.
mattdennewitz 17y ago
sounds like playdar, which was written to ease the load on streaming services. http://github.com/RJ/playdar-core http://github.com/RJ/playdar-core
jon_dahl 17y ago
The good news for video pirates is: -- Every time a video is encoded, it hashes differently. So a studio could find a torrented file and find people downloaded that exact copy, but they couldn't just ask for every copy of a show. -- If this became a problem, you could toss some random metadata on a file and it would hash differently. Or zip a file/program with a text file containing a guid. Etc.
- sailormoon 17y ago
  Every time a video is encoded, it hashes differently. Really? Even with the exact same source file and exact same codecs/methodology? Why is that, is there an "encoding time" metadata field or something?
ThinkWriteMute 17y ago
You know this makes me think: If you could find out if two files are exactly alike, could you then make a cloud storage services that doesn't require uploads of files it already has on the entire cloud? IE, John uploads True Blood Episode 4, Season 12. Paul tries to upload the same file. They match exactly (Except maybe for names). The Uploader negates the upload and makes it look like the file is now in Paul's list, when really only one copy exists.
- jefffoster 17y ago
  Some file systems have this as a built in option. As an example, there's a discussion of the ZFS implementation at http://blogs.sun.com/bonwick/entry/zfs_dedup http://blogs.sun.com/bonwick/entry/zfs_dedup.
  - ThinkWriteMute 17y ago
    Now that totally interests me. Thanks!

andrewljohnson 17y ago

Apple does this with iTunes/MobileMe, and any cloud storage player is aiming for the same network effect. The cloud storage play is huge for Apple... first they sell you the video, then they sell you the space to store the video, but it doesn't actually take them anymore hard drive space. And as the media gets bigger and bigger (HD today, who knows tomorrow), Apple sells you more and more storage "space" and their costs only go up marginally. It's the stuff billion dollar business plans are made of.

mattdennewitz 17y ago

sounds like playdar, which was written to ease the load on streaming services. http://github.com/RJ/playdar-core http://github.com/RJ/playdar-core

jon_dahl 17y ago

The good news for video pirates is: -- Every time a video is encoded, it hashes differently. So a studio could find a torrented file and find people downloaded that exact copy, but they couldn't just ask for every copy of a show. -- If this became a problem, you could toss some random metadata on a file and it would hash differently. Or zip a file/program with a text file containing a guid. Etc.

sailormoon 17y ago
Every time a video is encoded, it hashes differently. Really? Even with the exact same source file and exact same codecs/methodology? Why is that, is there an "encoding time" metadata field or something?

ThinkWriteMute 17y ago

You know this makes me think: If you could find out if two files are exactly alike, could you then make a cloud storage services that doesn't require uploads of files it already has on the entire cloud? IE, John uploads True Blood Episode 4, Season 12. Paul tries to upload the same file. They match exactly (Except maybe for names). The Uploader negates the upload and makes it look like the file is now in Paul's list, when really only one copy exists.

jefffoster 17y ago
Some file systems have this as a built in option. As an example, there's a discussion of the ZFS implementation at http://blogs.sun.com/bonwick/entry/zfs_dedup http://blogs.sun.com/bonwick/entry/zfs_dedup.
- ThinkWriteMute 17y ago
  Now that totally interests me. Thanks!