6 ms·

Decentralizing Distribution

119 points by marcodiego 5y ago

redsolver 5y ago
I am working on a very similar project (decentralized app distribution on Android) called SkyDroid (https://skydroid.app/ https://skydroid.app/), with the difference that it is based on Sia Skynet (a Filecoin/IPFS competitor) and uses the DNS system for global app discovery - so for example the SkyDroid app itself is available on the skydroid.app domain inside of it. I'm curious about how they will try to solve the discovery issue, because if they just have one global decentralized pool of apps, it will be very hard to ensure that no malicious apps get in. But if they keep it a central repository of apps trusted by F-Droid by default, there's not really much decentralization going on. Most developers would still publish their apps in the main repo directly.
- deepstack 5y ago
  > it will be very hard to ensure that no malicious apps get in Well isn't the whole idea of F-Droid is that the app can be build from the source that is available and the build checksum will match the source from anyone? Sure there can be malicious code, but code for that malicious code will also be available publicly.
- eighthave 5y ago
  We're taking a hybrid approach right now, where we incorporate IPFS support into our working stack. Then we can transition more and more to IPFS as it proves mature enough to replace the more centralized methods.
awinter-py 5y ago
interesting that it's 'veracity and distribution', not just distribution. makes sense that you need to hash the file, but I wonder about various higher-level ways to decentralize CI while maintaining a cheaply checkable 'affirmation' for example, can you cheaply verify repeatable builds? (on a phone even?). can you do intense linting for privacy violations like phone-home, then publish a 'lint payload' that's cheaply verifiable against the built binary? feels like if f-droid leads here, commercial software will have to adopt their tools to maintain their position of trust (also huge fan of f-droid project, I donate, I think it gets more important every year)
- mhitza 5y ago
  Somehow related take. When I was still interested in blockchains back in 2018 I was thinking that instead of proof of work, wouldn't it be nice to have a distributed CI where you run your builds & tests on end-user machines, it computes a hash of the resulting artifact. Which is validated by multiple nodes that run the same job for validation (then split the PoW fees among themselves). It's not an easy problem to solve of course, and didn't go too deep into what TPM (Trusted Platform Module) can do, but I think you would need an isolation unit from the kernel/OS to allow running untrusted code, and prevent tampering. There's definitely a fine line between such a functionality and spyware that would need serious safeguards.
  - yjftsjthsd-h 5y ago
    Just make it fully isolated with WASM or whatever, and then port the necessary compilers to cross-compile back to whatever native platform you want to use the outputs binaries on. Simple! (Not easy, but conceptually simple and probably doable.) The big problem that I can't see an obvious solution for is proving that a given output corresponds to the work requested. I guess you can just run it on enough machines that you hope they're not collaborating?
    - mhitza 5y ago
      I think isolation itself is the harder problem, as you don't want to have the untrusted code interface with your primary OS, nor the computer owner tamper with the isolated environment. > I guess you can just run it on enough machines that you hope they're not collaborating? Right validation through numbers. Still susceptible to a 51% attack, if I recall correctly what Sybil attacks are about. This type of proof of work gives you a way to find collusion post fact. For example if your last commit on a PR (which triggers a decentralized CI), is the same commit on which you base you distribution build (assuming reproducible builds, same output), that's run on your fully trusted system, you would find out if jobs were faked.
- yjftsjthsd-h 5y ago
  I'm not sure if this is exactly what you want, but https://f-droid.org/en/docs/Reproducible_Builds/ https://f-droid.org/en/docs/Reproducible_Builds/ is a thing
yjftsjthsd-h 5y ago
> Expand outreach and assistance to software developers, media organization, tech companies and more to promote adoption of the F-Droid platform for their software, content and devices (example: Mozilla could easily run a Mozilla app store that includes all the Mozilla channels: releases, nightlies, etc. They could also include a curated collection of reproducibly built apps that Mozilla approves of. Someone who trusts Mozilla can then easily choose to only have access to the Mozilla-curated app store) That would be fantastic!
- franga2000 5y ago
  This would also finally remove the excuse the Signal devs have for not including the app in F-Droid. Third-party repos aren't very discoverable and the official repos don't let the devs use their own keys.
  - eighthave 5y ago
    Yes, but f-droid.org actually does let devs use their own keys via the reproducible builds process (https://f-droid.org/docs/Build_Metadata_Reference/#Binaries https://f-droid.org/docs/Build_Metadata_Reference/#Binaries). The only blocker for getting Signal into f-droid.org is that it includes proprietary Google libs.
throwaway984393 5y ago
FWIW, Linux distributions are decentralized without any modern tech. It is all secured via signed checksum files you can download from hundreds of geographically-distributed independent sources. There is no dependence on any single protocol or technology, because it's just copied files. Anything that can copy files can spread the distribution. This makes them future-proof, backwards-compatible, and flexible enough to work in any situation. You can cache files, proxy connections, use any storage mechanism, any transfer method. The packaging tools support multiple transport protocols. Works offline and online. You can install a Linux distribution today the same way you did 30 years ago, from the same sources, even though the underlying servers and software have been replaced.
- giantrobot 5y ago
  No. Clearly old things are bad. Only new things are good. Everything but be ridiculously complicated and extremely fragile. Reliability is bad! /s The Linux distro model is awesome. It would be more awesome with reproducible builds but it's still awesome as-is. Not only can Linux distros easily distribute their software but even mix and match packages between different mediums and have completely consistent systems with the same level of security. Even better is anyone can freely set up a distro mirror and anyone using it can trust it the same as they trust an "official" mirror.
- pabs3 5y ago
  It would still be good to have Linux distros on IPFS too though.
- boramalper 5y ago
  > It is all secured via signed checksum files you can download from hundreds of geographically-distributed independent sources. It's not permissionless though, or put differently, discoverability is the problem. You can set up your own mirror, and through cryptographic protocols the authenticity of the data you are distributing can be verified all well, but you need the distro maintainers to manually add you to their list of announced mirrors before any traffic hits your servers. The point of BitTorrent, IPFS, Dat, etc is that they create a network of participants that does not need anyone to add them (nor, consequently, their permission) to start contributing. This also reduces the administrative burden of course, and provides load-balancing as well since the content is now distributed not in file-units but block-units (~ 16 KiB).
skyfaller 5y ago
Dammit, yet another project integrating/promoting cryptocurrency (Filecoin in this case). I like and use F-Droid, and I'm really unhappy about how difficult it's becoming to avoid web3 crap turning everything into a financial instrument to fuel speculation, energy use and e-waste.
- BLKNSLVR 5y ago
  Don't let the cloud of BS surrounding web3 (and cryptocurrency as a whole) obscure your view of the potential gold that lies at its core. Your comment reminds me of Letterman ridiculing Bill Gates' description of (very) early internet potential by saying "have you ever heard of the radio?". Edit: (the following was added after a couple of upvotes were received, so the following isn't necessarily "agreed with" by anyone) Uneducated opinion: The financial instrumentalisation of some of these web3 things is required for these two sides of the same coin (pun unintended): 1. A barrier to entry against those who would seek to sabotage the system / network / <thing> 2. Incentive to participate and donate resources to the system / network / <thing> The speculation side of this financial instrumentalisation is, to a greater or lesser degree, unintended consequences (potentially inflammatory example: Bitcoin was intended as an uncensorable way to exchange value, ie. a currency, and has now turned into a store of value to 'hodl' long-term rather than regularly exchange for goods and services). I put this down to human nature rather than design.
  - sinkholed 5y ago
    I looked up the Letterman video expecting to see someone confidently dismissive about the Internet. Letterman came across as fine: https://m.youtube.com/watch?v=gipL_CEw-fk https://m.youtube.com/watch?v=gipL_CEw-fk The radio quote is taken out of context. Letterman doesn't come across as well as Bowie though: https://m.youtube.com/watch?v=8tCC9yxUIdw https://m.youtube.com/watch?v=8tCC9yxUIdw
  - giantrobot 5y ago
    > I put this down to human nature rather than design. The design of Bitcoin plays a larger part in "hodl" than human nature. The cost of mining growth with growth of the network pretty quickly eliminated Bitcoin's utility as a value exchange. It didn't take long for Bitcoin transactions to get more expensive than traditional payment methods. There's no sense in paying fees to transact in Bitcoin if they're higher than traditional payment. Bitcoin's "value" has also always been highly variable thanks in part to the fees so there's no good way to post prices in Bitcoin for vendors either. Shit the value can fluctuate wildly while your transaction is processing! Your agreed upon "price" is meaningless.
sinkholed 5y ago
Full disclaimer: I have no cryptocurrency assets, nor interest. > Since the “decentralized web” includes all sorts of things, it is important to also mention things that we will not be doing with this grant. We are not looking at NFTs, blockchain, DAOs, smart contracts or related aspects of “Web3”. This absolutely is Web3, and Filecoin is on a blockchain. This is a great use, though I understand why they wish to distance themselves (Web3 is popular amongst political opponents of the chattering class).
lazzlazzlazz 5y ago
It's funny to see a web3 project insisting it isn't one; developers begrudgingly accepting that centralizing a platform creates an infrastructural and economic risk. Great to see!
jancsika 5y ago
> Filecoin is built on top of IPFS, which is “a distributed system for storing and accessing files”. > We are not looking at NFTs, blockchain, DAOs, smart contracts or related aspects of “Web3”. Wow, blockchain is so nearly synonymous with "complicated digital pump and dump scam" that the author has to disavow the very technology they started with in their description. So Filecoin is blockchain, but not, you know, blockchain.
pabs3 5y ago
I note that F-Droid are looking for Android and Python contractors, probably related to this IPFS/Filecoin work: https://www.fossjobs.net/job/10840/seeking-androidpython-contractors-at-guardian-project/ https://www.fossjobs.net/job/10840/seeking-androidpython-con... https://guardianproject.info/contact/android-python-contractor/ https://guardianproject.info/contact/android-python-contract...
deleted 5y ago
[deleted]

119 points by marcodiego 5y ago

I am working on a very similar project (decentralized app distribution on Android) called SkyDroid (https://skydroid.app/ https://skydroid.app/), with the difference that it is based on Sia Skynet (a Filecoin/IPFS competitor) and uses the DNS system for global app discovery - so for example the SkyDroid app itself is available on the skydroid.app domain inside of it. I'm curious about how they will try to solve the discovery issue, because if they just have one global decentralized pool of apps, it will be very hard to ensure that no malicious apps get in. But if they keep it a central repository of apps trusted by F-Droid by default, there's not really much decentralization going on. Most developers would still publish their apps in the main repo directly.

deepstack 5y ago
> it will be very hard to ensure that no malicious apps get in Well isn't the whole idea of F-Droid is that the app can be build from the source that is available and the build checksum will match the source from anyone? Sure there can be malicious code, but code for that malicious code will also be available publicly.
eighthave 5y ago
We're taking a hybrid approach right now, where we incorporate IPFS support into our working stack. Then we can transition more and more to IPFS as it proves mature enough to replace the more centralized methods.

awinter-py 5y ago

interesting that it's 'veracity and distribution', not just distribution. makes sense that you need to hash the file, but I wonder about various higher-level ways to decentralize CI while maintaining a cheaply checkable 'affirmation' for example, can you cheaply verify repeatable builds? (on a phone even?). can you do intense linting for privacy violations like phone-home, then publish a 'lint payload' that's cheaply verifiable against the built binary? feels like if f-droid leads here, commercial software will have to adopt their tools to maintain their position of trust (also huge fan of f-droid project, I donate, I think it gets more important every year)

mhitza 5y ago
Somehow related take. When I was still interested in blockchains back in 2018 I was thinking that instead of proof of work, wouldn't it be nice to have a distributed CI where you run your builds & tests on end-user machines, it computes a hash of the resulting artifact. Which is validated by multiple nodes that run the same job for validation (then split the PoW fees among themselves). It's not an easy problem to solve of course, and didn't go too deep into what TPM (Trusted Platform Module) can do, but I think you would need an isolation unit from the kernel/OS to allow running untrusted code, and prevent tampering. There's definitely a fine line between such a functionality and spyware that would need serious safeguards.
- yjftsjthsd-h 5y ago
  Just make it fully isolated with WASM or whatever, and then port the necessary compilers to cross-compile back to whatever native platform you want to use the outputs binaries on. Simple! (Not easy, but conceptually simple and probably doable.) The big problem that I can't see an obvious solution for is proving that a given output corresponds to the work requested. I guess you can just run it on enough machines that you hope they're not collaborating?
  - mhitza 5y ago
    I think isolation itself is the harder problem, as you don't want to have the untrusted code interface with your primary OS, nor the computer owner tamper with the isolated environment. > I guess you can just run it on enough machines that you hope they're not collaborating? Right validation through numbers. Still susceptible to a 51% attack, if I recall correctly what Sybil attacks are about. This type of proof of work gives you a way to find collusion post fact. For example if your last commit on a PR (which triggers a decentralized CI), is the same commit on which you base you distribution build (assuming reproducible builds, same output), that's run on your fully trusted system, you would find out if jobs were faked.
yjftsjthsd-h 5y ago
I'm not sure if this is exactly what you want, but https://f-droid.org/en/docs/Reproducible_Builds/ https://f-droid.org/en/docs/Reproducible_Builds/ is a thing

yjftsjthsd-h 5y ago

> Expand outreach and assistance to software developers, media organization, tech companies and more to promote adoption of the F-Droid platform for their software, content and devices (example: Mozilla could easily run a Mozilla app store that includes all the Mozilla channels: releases, nightlies, etc. They could also include a curated collection of reproducibly built apps that Mozilla approves of. Someone who trusts Mozilla can then easily choose to only have access to the Mozilla-curated app store) That would be fantastic!

franga2000 5y ago
This would also finally remove the excuse the Signal devs have for not including the app in F-Droid. Third-party repos aren't very discoverable and the official repos don't let the devs use their own keys.
- eighthave 5y ago
  Yes, but f-droid.org actually does let devs use their own keys via the reproducible builds process (https://f-droid.org/docs/Build_Metadata_Reference/#Binaries https://f-droid.org/docs/Build_Metadata_Reference/#Binaries). The only blocker for getting Signal into f-droid.org is that it includes proprietary Google libs.

throwaway984393 5y ago

FWIW, Linux distributions are decentralized without any modern tech. It is all secured via signed checksum files you can download from hundreds of geographically-distributed independent sources. There is no dependence on any single protocol or technology, because it's just copied files. Anything that can copy files can spread the distribution. This makes them future-proof, backwards-compatible, and flexible enough to work in any situation. You can cache files, proxy connections, use any storage mechanism, any transfer method. The packaging tools support multiple transport protocols. Works offline and online. You can install a Linux distribution today the same way you did 30 years ago, from the same sources, even though the underlying servers and software have been replaced.

giantrobot 5y ago
No. Clearly old things are bad. Only new things are good. Everything but be ridiculously complicated and extremely fragile. Reliability is bad! /s The Linux distro model is awesome. It would be more awesome with reproducible builds but it's still awesome as-is. Not only can Linux distros easily distribute their software but even mix and match packages between different mediums and have completely consistent systems with the same level of security. Even better is anyone can freely set up a distro mirror and anyone using it can trust it the same as they trust an "official" mirror.
pabs3 5y ago
It would still be good to have Linux distros on IPFS too though.
boramalper 5y ago
> It is all secured via signed checksum files you can download from hundreds of geographically-distributed independent sources. It's not permissionless though, or put differently, discoverability is the problem. You can set up your own mirror, and through cryptographic protocols the authenticity of the data you are distributing can be verified all well, but you need the distro maintainers to manually add you to their list of announced mirrors before any traffic hits your servers. The point of BitTorrent, IPFS, Dat, etc is that they create a network of participants that does not need anyone to add them (nor, consequently, their permission) to start contributing. This also reduces the administrative burden of course, and provides load-balancing as well since the content is now distributed not in file-units but block-units (~ 16 KiB).

skyfaller 5y ago

Dammit, yet another project integrating/promoting cryptocurrency (Filecoin in this case). I like and use F-Droid, and I'm really unhappy about how difficult it's becoming to avoid web3 crap turning everything into a financial instrument to fuel speculation, energy use and e-waste.

BLKNSLVR 5y ago
Don't let the cloud of BS surrounding web3 (and cryptocurrency as a whole) obscure your view of the potential gold that lies at its core. Your comment reminds me of Letterman ridiculing Bill Gates' description of (very) early internet potential by saying "have you ever heard of the radio?". Edit: (the following was added after a couple of upvotes were received, so the following isn't necessarily "agreed with" by anyone) Uneducated opinion: The financial instrumentalisation of some of these web3 things is required for these two sides of the same coin (pun unintended): 1. A barrier to entry against those who would seek to sabotage the system / network / <thing> 2. Incentive to participate and donate resources to the system / network / <thing> The speculation side of this financial instrumentalisation is, to a greater or lesser degree, unintended consequences (potentially inflammatory example: Bitcoin was intended as an uncensorable way to exchange value, ie. a currency, and has now turned into a store of value to 'hodl' long-term rather than regularly exchange for goods and services). I put this down to human nature rather than design.
- sinkholed 5y ago
  I looked up the Letterman video expecting to see someone confidently dismissive about the Internet. Letterman came across as fine: https://m.youtube.com/watch?v=gipL_CEw-fk https://m.youtube.com/watch?v=gipL_CEw-fk The radio quote is taken out of context. Letterman doesn't come across as well as Bowie though: https://m.youtube.com/watch?v=8tCC9yxUIdw https://m.youtube.com/watch?v=8tCC9yxUIdw
- giantrobot 5y ago
  > I put this down to human nature rather than design. The design of Bitcoin plays a larger part in "hodl" than human nature. The cost of mining growth with growth of the network pretty quickly eliminated Bitcoin's utility as a value exchange. It didn't take long for Bitcoin transactions to get more expensive than traditional payment methods. There's no sense in paying fees to transact in Bitcoin if they're higher than traditional payment. Bitcoin's "value" has also always been highly variable thanks in part to the fees so there's no good way to post prices in Bitcoin for vendors either. Shit the value can fluctuate wildly while your transaction is processing! Your agreed upon "price" is meaningless.

sinkholed 5y ago

Full disclaimer: I have no cryptocurrency assets, nor interest. > Since the “decentralized web” includes all sorts of things, it is important to also mention things that we will not be doing with this grant. We are not looking at NFTs, blockchain, DAOs, smart contracts or related aspects of “Web3”. This absolutely is Web3, and Filecoin is on a blockchain. This is a great use, though I understand why they wish to distance themselves (Web3 is popular amongst political opponents of the chattering class).

lazzlazzlazz 5y ago

It's funny to see a web3 project insisting it isn't one; developers begrudgingly accepting that centralizing a platform creates an infrastructural and economic risk. Great to see!

jancsika 5y ago

> Filecoin is built on top of IPFS, which is “a distributed system for storing and accessing files”. > We are not looking at NFTs, blockchain, DAOs, smart contracts or related aspects of “Web3”. Wow, blockchain is so nearly synonymous with "complicated digital pump and dump scam" that the author has to disavow the very technology they started with in their description. So Filecoin is blockchain, but not, you know, blockchain.

pabs3 5y ago

I note that F-Droid are looking for Android and Python contractors, probably related to this IPFS/Filecoin work: https://www.fossjobs.net/job/10840/seeking-androidpython-contractors-at-guardian-project/ https://www.fossjobs.net/job/10840/seeking-androidpython-con... https://guardianproject.info/contact/android-python-contractor/ https://guardianproject.info/contact/android-python-contract...

deleted 5y ago

[deleted]