3 ms·

if you are looking for something similar to ipfs but a bit more minimalistic and performance oriented, check out iroh https://github.com/n0-computer/iroh https:

if you are looking for something similar to ipfs but a bit more minimalistic and performance oriented, check out iroh https://github.com/n0-computer/iroh https://github.com/n0-computer/iroh . It is a set of open source libraries for peer to peer networking and content-addressed storage. It is written in rust, but we have bindings to many languages. One part of iroh is a work in progress implementation of the willow spec. The lower layers include a networking library similar to libp2p and a library for content-addressed storage and replication based on blake3 verified streaming. Most iroh developers have been active in the ipfs community for many years and have shared similar frustrations... See this talk from me in 2019 :-) https://youtu.be/Qzu0xtCT-R0?t=169 https://youtu.be/Qzu0xtCT-R0?t=169

nobodywillobsrv 3y ago
Is iroh suitable for use as a collaborative decentralized function cache? Had a quick skim but can't quite see how to get at the get/put etc in the python api yet. Will spend more time later.
binary132 3y ago
Interesting — does the Rust crate export a C API then?
- rklaehn 3y ago
  Not officially. We currently have bindings for rust, python, golang and swift. These were the most asked for bindings (python for ml, golang for networking and swift for ios apps). We are using uniffi https://mozilla.github.io/uniffi-rs/ https://mozilla.github.io/uniffi-rs/ Would you need C or C++ bindings?
  - binary132 3y ago
    Ah, I see. Hm. I might be interested in a C API since that could be used in C, C++, and Lua equally well. I really was just wondering what the common implementation between the bindings was since it struck me as unusual that there would be a number of bindings but not C (which is AIUI the only interface besides Rust that Rust can really export.)
    - rklaehn 3y ago
      So, I am not the one that is doing the bindings, so take this with a grain of salt. It seems uniffi does create C compatible bindings in order to make bindings for all these other languages. But these are internal bindings that are ugly and not intended to be used externally.
      - binary132 3y ago
        Perhaps if an issue were opened describing the necessary steps to provide a fluent and stable C API, staying consistent with the uniffi approach you're using for the other wrappers, then someone enterprising could pick up the ball and run with it. :)
ramrunner0xff 3y ago
https://veilid.com/ https://veilid.com/ should also be a great alternative. i haven't had time to use it yet, but it was built to address performance issues with ipfs and allow for both dht style content discovery, but also for direct websocket connections for streaming (and doing that in an anonymous fashion)
- rklaehn 3y ago
  This looks very interesting. They made very similar choices than we (iroh) did. Rust, ed keys, blake3. They seem to do their own streams, while we are adapting QUIC to a more p2p approach. Also the holepunching approach seems to be different. But I would love to get more details.
  - ramrunner0xff 3y ago
    https://yewtu.be/watch?v=Kb1lKscAMDQ https://yewtu.be/watch?v=Kb1lKscAMDQ this was the presentation at DC'31. i will also check out iroh! thanks for working in building something in this space, it is much much needed!
    - rklaehn 3y ago
      Thanks. This is awesome. I think they are doing more work themselves in terms of crypto, whereas we rely on QUIC+TLS more. Regarding holepunching, our approach is a bit less pure p2p, but has quite good success rates. We copy the DERP protocol from tailscale. I am confident that we have a better story regarding handling of large blobs. We don't just use blake3, but blake3 verified streaming to allow for range requests. Also I wrote my own rust library for blake3 verified streaming that reduces the overhead of the verification data. https://crates.io/crates/bao-tree https://crates.io/crates/bao-tree I tried to get on their discord at https://veilid.com/discord https://veilid.com/discord, but I get an invalid invite. You know a better way to get in touch?
      - ramrunner0xff 3y ago
        hmm this is strange, i tried the invite and it worked for me. If you are on fedi, @thegibson@hackers.town is part of the team. thanks for the links, i will get in touch personally when i try ir0h :)
orthecreedence 3y ago
Hi, I'm super intrigued by Willow and your work on iroh. Do you have any kind of documentation on how iroh deviates from Willow, or what parts of Willow are planned to be implemented vs omitted?
- rklaehn 3y ago
  Not yet. We have been busy with other stuff, also the willow spec has been a bit of a moving target until now. We would like to take our rust willow impl and separate it a bit more from our code base, so that iroh documents are just users of the willow crate.
  - orthecreedence 3y ago
    That makes sense. I think I might try to really jam through the Willow docs and get a good understanding. If it all looks good, I might be able to help out splitting these things out =].
snvzz 3y ago
iroh seems to have a couple of "killer tools" already, known as dumbpipe[0] and sendme[1]. Although I am concerned that while dumbpipe does mention cryptography, sendme's webpage makes no mention of it (?). 0. https://www.dumbpipe.dev/ https://www.dumbpipe.dev/ 1. https://iroh.computer/sendme https://iroh.computer/sendme
- rklaehn 3y ago
  It's using the same transport. Basically sendme is like dumbpipe, but adds blake3 verified streaming from the iroh-bytes crate for content-addressed data transport.
  - snvzz 3y ago
    I imagined as much, but the website does still not mention encryption. Which works against it. E2EE is a requirement today.
    - rklaehn 3y ago
      Thanks for letting us know. We will add a section about encryption. These tiny tools are basically one week projects to show off the tech, but they try to be useful on it's own as well.

nobodywillobsrv 3y ago

Is iroh suitable for use as a collaborative decentralized function cache? Had a quick skim but can't quite see how to get at the get/put etc in the python api yet. Will spend more time later.

binary132 3y ago

Interesting — does the Rust crate export a C API then?

rklaehn 3y ago
Not officially. We currently have bindings for rust, python, golang and swift. These were the most asked for bindings (python for ml, golang for networking and swift for ios apps). We are using uniffi https://mozilla.github.io/uniffi-rs/ https://mozilla.github.io/uniffi-rs/ Would you need C or C++ bindings?
- binary132 3y ago
  Ah, I see. Hm. I might be interested in a C API since that could be used in C, C++, and Lua equally well. I really was just wondering what the common implementation between the bindings was since it struck me as unusual that there would be a number of bindings but not C (which is AIUI the only interface besides Rust that Rust can really export.)
  - rklaehn 3y ago
    So, I am not the one that is doing the bindings, so take this with a grain of salt. It seems uniffi does create C compatible bindings in order to make bindings for all these other languages. But these are internal bindings that are ugly and not intended to be used externally.
    - binary132 3y ago
      Perhaps if an issue were opened describing the necessary steps to provide a fluent and stable C API, staying consistent with the uniffi approach you're using for the other wrappers, then someone enterprising could pick up the ball and run with it. :)

ramrunner0xff 3y ago

https://veilid.com/ https://veilid.com/ should also be a great alternative. i haven't had time to use it yet, but it was built to address performance issues with ipfs and allow for both dht style content discovery, but also for direct websocket connections for streaming (and doing that in an anonymous fashion)

rklaehn 3y ago
This looks very interesting. They made very similar choices than we (iroh) did. Rust, ed keys, blake3. They seem to do their own streams, while we are adapting QUIC to a more p2p approach. Also the holepunching approach seems to be different. But I would love to get more details.
- ramrunner0xff 3y ago
  https://yewtu.be/watch?v=Kb1lKscAMDQ https://yewtu.be/watch?v=Kb1lKscAMDQ this was the presentation at DC'31. i will also check out iroh! thanks for working in building something in this space, it is much much needed!
  - rklaehn 3y ago
    Thanks. This is awesome. I think they are doing more work themselves in terms of crypto, whereas we rely on QUIC+TLS more. Regarding holepunching, our approach is a bit less pure p2p, but has quite good success rates. We copy the DERP protocol from tailscale. I am confident that we have a better story regarding handling of large blobs. We don't just use blake3, but blake3 verified streaming to allow for range requests. Also I wrote my own rust library for blake3 verified streaming that reduces the overhead of the verification data. https://crates.io/crates/bao-tree https://crates.io/crates/bao-tree I tried to get on their discord at https://veilid.com/discord https://veilid.com/discord, but I get an invalid invite. You know a better way to get in touch?
    - ramrunner0xff 3y ago
      hmm this is strange, i tried the invite and it worked for me. If you are on fedi, @thegibson@hackers.town is part of the team. thanks for the links, i will get in touch personally when i try ir0h :)

orthecreedence 3y ago

Hi, I'm super intrigued by Willow and your work on iroh. Do you have any kind of documentation on how iroh deviates from Willow, or what parts of Willow are planned to be implemented vs omitted?

rklaehn 3y ago
Not yet. We have been busy with other stuff, also the willow spec has been a bit of a moving target until now. We would like to take our rust willow impl and separate it a bit more from our code base, so that iroh documents are just users of the willow crate.
- orthecreedence 3y ago
  That makes sense. I think I might try to really jam through the Willow docs and get a good understanding. If it all looks good, I might be able to help out splitting these things out =].

snvzz 3y ago

iroh seems to have a couple of "killer tools" already, known as dumbpipe[0] and sendme[1]. Although I am concerned that while dumbpipe does mention cryptography, sendme's webpage makes no mention of it (?). 0. https://www.dumbpipe.dev/ https://www.dumbpipe.dev/ 1. https://iroh.computer/sendme https://iroh.computer/sendme

rklaehn 3y ago
It's using the same transport. Basically sendme is like dumbpipe, but adds blake3 verified streaming from the iroh-bytes crate for content-addressed data transport.
- snvzz 3y ago
  I imagined as much, but the website does still not mention encryption. Which works against it. E2EE is a requirement today.
  - rklaehn 3y ago
    Thanks for letting us know. We will add a section about encryption. These tiny tools are basically one week projects to show off the tech, but they try to be useful on it's own as well.