6 ms·
One idea that's been around for awhile is to identify files by their hash. That has pros and cons. The good side of that is that the file is immutable; you ca
by elihu 5y ago
One idea that's been around for awhile is to identify files by their hash. That has pros and cons. The good side of that is that the file is immutable; you can't accidentally link to something else unless someone can manufacture a hash collision somehow. The down side is that if the file is corrected in some way, you don't get the fixes.
In a lot of the peer-to-peer distributed hash table designs, all you need to retrieve a file is its hash.
https://en.wikipedia.org/wiki/Content_addressable_network https://en.wikipedia.org/wiki/Content_addressable_network
- kazinator 5y agoProblem is, you have to download the entire video to check the hash. That's not how video embedding works; the client browser is just handed some link, and it obtains pieces of the video, rendering it instantly. Basically, little segments of the video have to have a signature which is continuously validated. Or something like that.
- skipants 5y agoI don't have expertise in video codecs or file formats, but couldn't you hash the first N bytes of a stream? Stream those N bytes to the client and if it matches start the video, else stop the download and not start the video.
- jodrellblank 5y agoPresumably for SHA256 you only need to hash ~256 bits; what's anyone gonna do, try all possible combinations to find a collision?
- mkl 5y ago256 bits is only 32 bytes, and most file formats have standard stuff right at the start. Collisions would be very common.
- tomjakubowski 5y agoIf you only hash/check the first N bytes of the video stream, the remainder of the video could be anything.
- kazinator 5y agoThey will keep the first few seconds or minutes of the original video, bit-exact, and then switch to porn. The player needs to validate every section.
- giantrobot 5y agoThis has a number of problems. The most egregious is if I'm an attacker and I have the file you request I can hash the appropriate portion you'd use to verify it but fill the rest with junk or exploits. You'd receive the file, it would emit the correct hash, yet be not what you were expecting. For video especially what you receive isn't necessarily predictable by the client. With HLS or MPEG DASH streaming the video you receive could be one of a number of different encoding e.g. lower or higher bitrates to deal with changing network conditions. The actual m3u8/mpd file you might receive could change arbitrarily as the video provider adds or drops different encodings. The hash of such a file today isn't guaranteed to match the hash tomorrow for entirely banal non-malicious reasons. Fun fact: the UUHash algorithm used by the FastTrack network (Kazaa, Morpheus, etc) only hashed the first bit of a file. Hashing a large file took forever on hardware of the day. Even hashing small files was non-trivial. The RIAA through various fronts would insert spoofed files where the first portion of the file was legitimate but the content of the file would be junk or annoying sounds. The files would be named like any other MP3 someone was searching for and even have seemingly good IDv3 tags.
- dragonwriter 5y ago> Fun fact: the UUHash algorithm used by the FastTrack network (Kazaa, Morpheus, etc) only hashed the first bit of a file. The first 300KiB plus a series of 300KiB chunks at exponentially-increasing offsets, per Wikipedia. But still a small fraction of thw file.
- skipants 5y agoWell explained. Thank you!
- AgentME 5y agoMerkle trees basically accomplish this with separate hashes over every N bytes, so that the content can be verified continuously as it's downloaded.
- nextlevelwizard 5y agoAssuming there is no malicious intent behind the embeds you could have the hash in the header of the video.
- kazinator 5y agoI think that in the context of the topic of this submission "replacement by porn" is being regarded as malicious intent.
- grapehut 5y agoThat's not really a problem. You don't hash the entire video, but do something resembling a merkle tree. i.e. look at a torrent, they're identified by a hash but you can download and verify a random chunk
- kazinator 5y agoRight, merkle tree! OK, so the embedding site only stores a single hash: the root one. This hashes the the remaining hashes. The first thing we fetch from the video is those hashes and if their hash doesn't match, we flag/ignore the video and refuse to play. Multiple levels of the tree can be stored throughout the video file. The first level after the root can be for major sections, like 5 minute segments. The next levels are then at the start of each 5 minute segment, giving hashes for one second chunks. If the root hash checks out, we get the 5-min hashes. If they check out, we get the hashes for the first 5-min block, and if those hashes check out, we start to play the video, validating every second of it against a one second hash from the 5 min block. Then we get the next 5-min hash block and so on. Kind of thing.
- lelandbatey 5y agoIn actual HLS streaming videos are served in 2-10 second segments, a size small enough that you realistically can hash each segment and verify each segment. Youd have to implement it as an extension of the HLS protocol, so probably as m3u8 with additional fields, maybe as comments.
- grumbel 5y agoThat's already how it works in IPFS or Freenet. The hash is not for the data itself, but for a metadata document that contains a list of hashes to the data broken up into smaller blocks.
- ohyeshedid 5y agoSounds like something subresource integrity[1] could be expanded to include. [1]: https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity https://developer.mozilla.org/en-US/docs/Web/Security/Subres...
- deleted 5y ago[deleted]
- anfilt 5y agoThis is were a signature would come in handy, if you want the resource to be updated. Just keep track that it was signed by the same private key.