4 ms·

I'm looking at integrating IPFS with sandstorm (sandstorm.io). The sandstorm (and other homelab/homeserver) community have an interest in self-hosting. IPFS al

by hosh 6y ago

I'm looking at integrating IPFS with sandstorm (sandstorm.io). The sandstorm (and other homelab/homeserver) community have an interest in self-hosting. IPFS allows people that are interested in seeing something stored to pin it themselves. But yes: bundling IPFS with the app is pretty much what would need to happen. As they are now, neither the IPFS ecosystem or sandstorm are really decentralized -- or maybe I should be more precise: not decentralized in a resilient way. Yet, I think both are great technologies that enable some interesting things. I'm not as interested in the blockchain notion of decentralized though. I'm looking for resiliency to severe netsplits. As a thought experiment, if Earth crosses a coronal mass ejection event and our power and communication grids go down, would we be able stitch things back together again when our grid fractures?

secfirstmd 6y ago
Very cool idea!
pm 6y ago
Curious about the homelab/homeserver community. Are there any sites with further information?
- hosh 6y ago
  If you reddit, r/selfhosted and r/homelab are a great place to start. I really like the sandstorm.io platform, though it is not currently supported on an rpi. It has the potential as a platform that “grandma” can use. There is a project called HomelabOS. It reminds me a bit of Homebrew.
  - pm 6y ago
    Cheers. I remember hearing SandStorm (the company) went under a couple years ago, and was curious but hadn't explored it in depth. I've been paying attention to CollapseOS as well, though it's a little more apocalyptic in nature. The eventual aim is to be able to provision a set of servers from a thumb drive without Internet (if that's even possible), but I haven't had much time to get far.
    - hosh 6y ago
      I'll check out CollapseOS! The sandstorm community is still going. The app developers are interested, but the platform itself has not received much love. For my own goals, (1) getting ARM support (which includes customizing seccomp for the ARM; patching the sandstorm app package format; and getting a good developer story), and (2) enabling IPFS. A kind of practical user-facing goal for integrating IPFS would be to decentralize the sandstorm app store. What I am finding is that the design goals for sandstorm looks like it was originally intended to be deployed in the cloud, whereas I'd like to see individuals and communities empowered to run sandstorm for their local community.
      - ocdtrekkie 6y ago
        Honestly, the biggest pain point around adding ARM support is that it'll be a pain to create ARM-compatible packages for all the apps. I definitely think the largest unanswered question in ARM support for Sandstorm.io is "how do we make it convenient for app developers to release packages for multiple architectures?" That being said, I'd be super excited at the possible expansion of the community ARM support would bring. By the way, regarding seccomp and the platform receiving love, there's actually a new experimental seccomp filter which operates as a whitelist instead of a blacklist: https://github.com/sandstorm-io/sandstorm/pull/3502 https://github.com/sandstorm-io/sandstorm/pull/3502
        hosh 6y ago
        I'm not a security engineer, so the seccomp stuff isn't something I know a lot about. Given that limited experience. whitelist sounds more sensible than blacklist, and I presume that it would be more portable on different architectures. I have a lot more confidence when it comes to tooling, packaging, testing, developer environments, CI/CD systems etc. since that is the kind of thing I do at work. Within that old Github thread: - A suggestion that there is a metadata field that can be purposed for architecture - There was a suggestion for using QEMU with the vagrant tool I'm a bit weirded out by a packaging tool that tries to automatically look for all the files related to it -- though I think there might be a good reason for it. You do lose the ability to have reproducible builds and any kind of continuous integration or delivery. I kinda wonder if just switching to something like Nix would work better. Reproducible builds would make it easier for the app ecosystem to be rebuilt on a different arch. ARM and RPI is great, but I expect RISC-V to be gaining traction over the next couple years. As far as my personal goal of helping develop resilient infrastructure, being able to quickly adopt new architectures would help a lot. I have not floated this in the sandstorm-dev mailing list yet, but I'll get some conversation started over there.
random_kris 6y ago
I was also looking at sandstorm. I even managed to install it on my homeserver. But it is sadly not really up to date. As a replacement i found homelabOS which is a bit more complicated (I think). And I was also thinking about integrating ipfs into my homeserver stack. But instead of plain ipfs I was thinking about sia coin. They offer cloud storage. So I was thinking about to rent 4tb of space. And whatever money I make with it I use it to buy backup from sia. So in a sense I provide backups to others and in return I get free backup. Sadly work is being to intensive on last month so I haven't found much inspiration in completing stuff above.
jimpick 6y ago
I've been doing some experimentation along the same lines ... I've got a small proof-of-concept working where I can sync Earthstar databases between multiple Sandstorm instances - each "grain" acts as a pub, and you can use "webkeys" as pubs. I'm using powerbox-http-proxy to request capabilities for connections, so it works even with fully sandboxed client-side networking (ALLOW_LEGACY_RELAXED_CSP=false). It's maddeningly complex to do simple networking, but it's amazing to have the security layer there. I'll put together a blog post / video soonish. https://github.com/jimpick/sandstorm-earthstar-foyer https://github.com/jimpick/sandstorm-earthstar-foyer Sandstorm is very interesting for privacy-focused peer-to-peer because it has the tools in place to sandbox networking. The networking security model is designed to prevent things from talking to each other without user approvals - so it's extra tricky to do peer-to-peer things with it. I'm excited to try to stick some of my old IPFS/IPLD/libp2p/Dat/CRDT things into Sandstorm - there's a tonne of interesting things that could be done with it.
- hosh 6y ago
  I was wondering about that. I suppose, things that the user wants to share publicly can be done so (via powerbox), but otherwise, everything else would not be. Are you using the grain model for each individual files to be shared? (I looked up your email and I would like to continue chatting about this).
  - jimpick 6y ago
    Everything is in a grain, but I think the Earthstar applications could be decomposed to just handle a single document at a time. I made a quick demo recording and updated the README with a link. Video is here: https://bafybeih6kkv4rgmfada4kfc5thmtzheiurq7ck5uz4rchqkrxw7eciozga.ipfs.dweb.link/sandstorm-earthstar-1.mp4 https://bafybeih6kkv4rgmfada4kfc5thmtzheiurq7ck5uz4rchqkrxw7...

by hosh 6y ago

secfirstmd 6y ago

Very cool idea!

pm 6y ago

Curious about the homelab/homeserver community. Are there any sites with further information?

hosh 6y ago
If you reddit, r/selfhosted and r/homelab are a great place to start. I really like the sandstorm.io platform, though it is not currently supported on an rpi. It has the potential as a platform that “grandma” can use. There is a project called HomelabOS. It reminds me a bit of Homebrew.
- pm 6y ago
  Cheers. I remember hearing SandStorm (the company) went under a couple years ago, and was curious but hadn't explored it in depth. I've been paying attention to CollapseOS as well, though it's a little more apocalyptic in nature. The eventual aim is to be able to provision a set of servers from a thumb drive without Internet (if that's even possible), but I haven't had much time to get far.
  - hosh 6y ago
    I'll check out CollapseOS! The sandstorm community is still going. The app developers are interested, but the platform itself has not received much love. For my own goals, (1) getting ARM support (which includes customizing seccomp for the ARM; patching the sandstorm app package format; and getting a good developer story), and (2) enabling IPFS. A kind of practical user-facing goal for integrating IPFS would be to decentralize the sandstorm app store. What I am finding is that the design goals for sandstorm looks like it was originally intended to be deployed in the cloud, whereas I'd like to see individuals and communities empowered to run sandstorm for their local community.
    - ocdtrekkie 6y ago
      Honestly, the biggest pain point around adding ARM support is that it'll be a pain to create ARM-compatible packages for all the apps. I definitely think the largest unanswered question in ARM support for Sandstorm.io is "how do we make it convenient for app developers to release packages for multiple architectures?" That being said, I'd be super excited at the possible expansion of the community ARM support would bring. By the way, regarding seccomp and the platform receiving love, there's actually a new experimental seccomp filter which operates as a whitelist instead of a blacklist: https://github.com/sandstorm-io/sandstorm/pull/3502 https://github.com/sandstorm-io/sandstorm/pull/3502
      - hosh 6y ago
        I'm not a security engineer, so the seccomp stuff isn't something I know a lot about. Given that limited experience. whitelist sounds more sensible than blacklist, and I presume that it would be more portable on different architectures. I have a lot more confidence when it comes to tooling, packaging, testing, developer environments, CI/CD systems etc. since that is the kind of thing I do at work. Within that old Github thread: - A suggestion that there is a metadata field that can be purposed for architecture - There was a suggestion for using QEMU with the vagrant tool I'm a bit weirded out by a packaging tool that tries to automatically look for all the files related to it -- though I think there might be a good reason for it. You do lose the ability to have reproducible builds and any kind of continuous integration or delivery. I kinda wonder if just switching to something like Nix would work better. Reproducible builds would make it easier for the app ecosystem to be rebuilt on a different arch. ARM and RPI is great, but I expect RISC-V to be gaining traction over the next couple years. As far as my personal goal of helping develop resilient infrastructure, being able to quickly adopt new architectures would help a lot. I have not floated this in the sandstorm-dev mailing list yet, but I'll get some conversation started over there.

random_kris 6y ago

I was also looking at sandstorm. I even managed to install it on my homeserver. But it is sadly not really up to date. As a replacement i found homelabOS which is a bit more complicated (I think). And I was also thinking about integrating ipfs into my homeserver stack. But instead of plain ipfs I was thinking about sia coin. They offer cloud storage. So I was thinking about to rent 4tb of space. And whatever money I make with it I use it to buy backup from sia. So in a sense I provide backups to others and in return I get free backup. Sadly work is being to intensive on last month so I haven't found much inspiration in completing stuff above.

jimpick 6y ago

I've been doing some experimentation along the same lines ... I've got a small proof-of-concept working where I can sync Earthstar databases between multiple Sandstorm instances - each "grain" acts as a pub, and you can use "webkeys" as pubs. I'm using powerbox-http-proxy to request capabilities for connections, so it works even with fully sandboxed client-side networking (ALLOW_LEGACY_RELAXED_CSP=false). It's maddeningly complex to do simple networking, but it's amazing to have the security layer there. I'll put together a blog post / video soonish. https://github.com/jimpick/sandstorm-earthstar-foyer https://github.com/jimpick/sandstorm-earthstar-foyer Sandstorm is very interesting for privacy-focused peer-to-peer because it has the tools in place to sandbox networking. The networking security model is designed to prevent things from talking to each other without user approvals - so it's extra tricky to do peer-to-peer things with it. I'm excited to try to stick some of my old IPFS/IPLD/libp2p/Dat/CRDT things into Sandstorm - there's a tonne of interesting things that could be done with it.

hosh 6y ago
I was wondering about that. I suppose, things that the user wants to share publicly can be done so (via powerbox), but otherwise, everything else would not be. Are you using the grain model for each individual files to be shared? (I looked up your email and I would like to continue chatting about this).
- jimpick 6y ago
  Everything is in a grain, but I think the Earthstar applications could be decomposed to just handle a single document at a time. I made a quick demo recording and updated the README with a link. Video is here: https://bafybeih6kkv4rgmfada4kfc5thmtzheiurq7ck5uz4rchqkrxw7eciozga.ipfs.dweb.link/sandstorm-earthstar-1.mp4 https://bafybeih6kkv4rgmfada4kfc5thmtzheiurq7ck5uz4rchqkrxw7...