5 ms·

I bid thee The Interplanetary File System (IPFS): https://ipfs.tech/ https://ipfs.tech/

pierat 3y ago
If you do use IPFS, then be aware that it has loads of traps you wouldn't expect. 1. A unique computer key identity is created on startup. 2. It announces every network adapter you have to the dHT, including localhost, docker virtual network adapters, everything. 3. If someone is sharing a specific hash content, you can query the network for their machine key and all their network interfaces, and everything else they're sharing. 4. I have early IRC communications that the ipfs.io gateway also quietly watches for 'bad hashes' and reports them to authorities. This was initially the decision they made to hunt down child porn predators, but this can be easily pivoted to copyright violators. 5. I also caught early on (in the 0.34 version) that google was already crawling computers, and all their shared hashes'. I had a few files I created that were very unique, and a curious search showed them to me, along with my machine's key. For completely legal bulk data, IPFS is a terrific solution. But for anything that is illegal or might be illegal in the future, well, I have to question that usage for those aforementioned reasons.
- nextaccountic 3y ago
  > 1. A unique computer key identity is created on startup. > 2. It announces every network adapter you have to the dHT, including localhost, docker virtual network adapters, everything. This is totally insane, is there any client that don't enable those misfeatures? (like, generate a new key identity on startup, and don't publish any network adapter)
  - yamtaddle 3y ago
    Guessing #1 has something to do with speeding up the notoriously-slow network. Probably allows more long-term route caching or something. #2 does seem silly. On a semi-related note, it's not that hard to set up a private IPFS. You just need a publicly-reachable "lighthouse" node (and it's extremely low-resource, the cheapest hosted VM you can find can likely handle the task without trouble) and some light config edits. IPFS isn't (and isn't trying to be) anonymous, but you can avoid sharing with the entire IPFS network and share only with trusted peers instead. The ordinary IPFS network won't even know you're there, because you won't be connecting to it.
    - pierat 3y ago
      I was there doing early testing, way prior to the filecoin offering. The justification for a computer key (which is a base56 sha) was to make every machine a unique node on the network. Note that IPFS also purports to be a multi-network system that can be on top of ip4/6/tor/etc, so they wanted a backwards compat ID. Now if my computer asks the dHT for hash X, I get a response back from all the computers with that hash. And it's similar to BitTorrent in that fashion. And the content hash is like Git. There's also IPNS, which makes a connection from DNS to IPFS. You need unique machine identifiers for this. With the sharing of every network interface, prior to 0.3 the idea was that ALL content loaded with IPFS could be shared. And 'nearness' ala STP could be used to provide local data to local machines. However, due to garbage copyright law, was not turned on by default. The root problem is that basic security was never considered, as in the 'bad things someone can do with these identifiers'. Ideas like "one person loads a viral video and every device on the local network gets it from them" are amazing ideas, but preclude the bad side like 'who to DoS and doxx' and all these bad side of internet issues. It sucks, cause I really think IPFS should have absolutely taken off. I could see this as solar system worthy networking for satellite to satellite. But there's inherent problems that I think would be better in the crucible of the IETF and not a commercial entity.
      - nextaccountic 3y ago
        > However, due to garbage copyright law, was not turned on by default. I don't understand, what is not turned on by default? Do you know which setting can be used to turn it on?
        pierat 3y ago
        Right now, on the http(s) web, when you request a file, the provenance of that file is linked to the domain name and the HTTPS traffic. On IPFS, the provenance of a file is its content hash, not some server you downloaded it from. On IPFS, if I join in on hosting and sharing a hash of a file, I could be a malware spewer, but that doesn't change the hash. Where this conflicts with copyright is, say you are on a IPFS enabled Movie server from CBS. By definition, only they are approved to transmit their files. But IPFS uses Git and BitTorrent to swarm download content. And default copyright, that's a civil tort matter. The original idea for IPFS was that for the local network, that it did support BitTorrent like operations from browser cache. So if something went viral, it was 1 download from the internet connection, and then on-lical-network the rest of the way. That feature was disabled EARLY on because it strictly violates copyright. It's also why your computer published all the network interfaces to the dHT, so that you could find local nodes. But they didn't remove that; they only removed the dynamic sharing part. I think there are settings to turn that back on, but I'm not 100% sure.

If you do use IPFS, then be aware that it has loads of traps you wouldn't expect. 1. A unique computer key identity is created on startup. 2. It announces every network adapter you have to the dHT, including localhost, docker virtual network adapters, everything. 3. If someone is sharing a specific hash content, you can query the network for their machine key and all their network interfaces, and everything else they're sharing. 4. I have early IRC communications that the ipfs.io gateway also quietly watches for 'bad hashes' and reports them to authorities. This was initially the decision they made to hunt down child porn predators, but this can be easily pivoted to copyright violators. 5. I also caught early on (in the 0.34 version) that google was already crawling computers, and all their shared hashes'. I had a few files I created that were very unique, and a curious search showed them to me, along with my machine's key. For completely legal bulk data, IPFS is a terrific solution. But for anything that is illegal or might be illegal in the future, well, I have to question that usage for those aforementioned reasons.

nextaccountic 3y ago
> 1. A unique computer key identity is created on startup. > 2. It announces every network adapter you have to the dHT, including localhost, docker virtual network adapters, everything. This is totally insane, is there any client that don't enable those misfeatures? (like, generate a new key identity on startup, and don't publish any network adapter)
- yamtaddle 3y ago
  Guessing #1 has something to do with speeding up the notoriously-slow network. Probably allows more long-term route caching or something. #2 does seem silly. On a semi-related note, it's not that hard to set up a private IPFS. You just need a publicly-reachable "lighthouse" node (and it's extremely low-resource, the cheapest hosted VM you can find can likely handle the task without trouble) and some light config edits. IPFS isn't (and isn't trying to be) anonymous, but you can avoid sharing with the entire IPFS network and share only with trusted peers instead. The ordinary IPFS network won't even know you're there, because you won't be connecting to it.
  - pierat 3y ago
    I was there doing early testing, way prior to the filecoin offering. The justification for a computer key (which is a base56 sha) was to make every machine a unique node on the network. Note that IPFS also purports to be a multi-network system that can be on top of ip4/6/tor/etc, so they wanted a backwards compat ID. Now if my computer asks the dHT for hash X, I get a response back from all the computers with that hash. And it's similar to BitTorrent in that fashion. And the content hash is like Git. There's also IPNS, which makes a connection from DNS to IPFS. You need unique machine identifiers for this. With the sharing of every network interface, prior to 0.3 the idea was that ALL content loaded with IPFS could be shared. And 'nearness' ala STP could be used to provide local data to local machines. However, due to garbage copyright law, was not turned on by default. The root problem is that basic security was never considered, as in the 'bad things someone can do with these identifiers'. Ideas like "one person loads a viral video and every device on the local network gets it from them" are amazing ideas, but preclude the bad side like 'who to DoS and doxx' and all these bad side of internet issues. It sucks, cause I really think IPFS should have absolutely taken off. I could see this as solar system worthy networking for satellite to satellite. But there's inherent problems that I think would be better in the crucible of the IETF and not a commercial entity.
    - nextaccountic 3y ago
      > However, due to garbage copyright law, was not turned on by default. I don't understand, what is not turned on by default? Do you know which setting can be used to turn it on?
      - pierat 3y ago
        Right now, on the http(s) web, when you request a file, the provenance of that file is linked to the domain name and the HTTPS traffic. On IPFS, the provenance of a file is its content hash, not some server you downloaded it from. On IPFS, if I join in on hosting and sharing a hash of a file, I could be a malware spewer, but that doesn't change the hash. Where this conflicts with copyright is, say you are on a IPFS enabled Movie server from CBS. By definition, only they are approved to transmit their files. But IPFS uses Git and BitTorrent to swarm download content. And default copyright, that's a civil tort matter. The original idea for IPFS was that for the local network, that it did support BitTorrent like operations from browser cache. So if something went viral, it was 1 download from the internet connection, and then on-lical-network the rest of the way. That feature was disabled EARLY on because it strictly violates copyright. It's also why your computer published all the network interfaces to the dHT, so that you could find local nodes. But they didn't remove that; they only removed the dynamic sharing part. I think there are settings to turn that back on, but I'm not 100% sure.