4 ms·

Well done. I've been frequently thinking over this past decade that an average consumer level distributed computing is the biggest causality of closed, proprie

Well done. I've been frequently thinking over this past decade that an average consumer level distributed computing is the biggest causality of closed, proprietary software/hardware. Consider compressing several terabytes of disjoint data to a zip file using compute resources of pc, smartphone, tablet etc. at the same time. I've seen some subtle attempts to make use of distributed computing in consumer space like Apple's Compressor for video encoding. Of course distributed computing has been a staple in scientific research for a long time. I personally use dask, modin for some data exploration activities when I feel the urge for some distributed computing. Wanted to checkout Julia's distributed computing capabilities but it required similar setup for all nodes but I'm interested in cross-platform(architecture) only.

nonameiguess 6y ago
This is sort of a plot point in Silicon Valley where the Pied Piper team is trying figure out where to physically store files they keep as a remote backup/sync service for a mobile app and end up going with a fully-distributed peer-to-peer solution that uses no servers at all and just stores file shards directly on the phones of all of their users, like the CERN shared CPU time project but for disk space.
- Abishek_Muthian 6y ago
  Thanks, I haven't watched that show. I'll try to watch that episode or season; don't know how that show is structured. Common network storage access removed most of the headache with distributed computing in my explorations on that subject(OP is using common network share to execute programs as well).
  - anaganisk 6y ago
    It goes like this, one of the programmers of the app, hack a smart fridge to irritate their roommate. Then one fine day, just during their biggest demo to an insurance data company, their garage looses power where their servers are located, they hastily build a mobile data centre on a truck. They try to transport it over to their friends lab at a college. Upon reaching there, they notice they forgot to lock the door, and find that their entire equipment got scattered all over the road. Cue some emotional scenes they get a call from insurance guy saying them the app perfomed perfectly well. Shockingly to these people they realise when one of the programmer hacked the smart fridge he accidentally linked their software to the smart fridge and the smart fridge via its network uploaded the "malware", there by the entire content into its network of all smart fridges there is on earth. Thats when they realise the power of distribution and they go P2P. Silicon valley season 2 last episode and season 3 first episode, should do, to get this.
    - Abishek_Muthian 6y ago
      Thank you!
astrange 6y ago
Distributed computing is not super useful for video encoding (although it's certainly kind of useful) because there's overhead to ship all the raw video around and put it back together again. If you have multiple files to encode, you might as well just run one on each machine.

nonameiguess 6y ago

This is sort of a plot point in Silicon Valley where the Pied Piper team is trying figure out where to physically store files they keep as a remote backup/sync service for a mobile app and end up going with a fully-distributed peer-to-peer solution that uses no servers at all and just stores file shards directly on the phones of all of their users, like the CERN shared CPU time project but for disk space.

Abishek_Muthian 6y ago
Thanks, I haven't watched that show. I'll try to watch that episode or season; don't know how that show is structured. Common network storage access removed most of the headache with distributed computing in my explorations on that subject(OP is using common network share to execute programs as well).
- anaganisk 6y ago
  It goes like this, one of the programmers of the app, hack a smart fridge to irritate their roommate. Then one fine day, just during their biggest demo to an insurance data company, their garage looses power where their servers are located, they hastily build a mobile data centre on a truck. They try to transport it over to their friends lab at a college. Upon reaching there, they notice they forgot to lock the door, and find that their entire equipment got scattered all over the road. Cue some emotional scenes they get a call from insurance guy saying them the app perfomed perfectly well. Shockingly to these people they realise when one of the programmer hacked the smart fridge he accidentally linked their software to the smart fridge and the smart fridge via its network uploaded the "malware", there by the entire content into its network of all smart fridges there is on earth. Thats when they realise the power of distribution and they go P2P. Silicon valley season 2 last episode and season 3 first episode, should do, to get this.
  - Abishek_Muthian 6y ago
    Thank you!

astrange 6y ago

Distributed computing is not super useful for video encoding (although it's certainly kind of useful) because there's overhead to ship all the raw video around and put it back together again. If you have multiple files to encode, you might as well just run one on each machine.