5 ms·

Yes, unfortunately both data and compute will probably become more and more centralized. At least the algorithmic components have a chance to becone available t

Yes, unfortunately both data and compute will probably become more and more centralized. At least the algorithmic components have a chance to becone available to everyone.

samstave 8y ago
Here is an off the cuff thought: What if there was (or maybe there already is?) a system which is distributed such as was SETI back in the day and its a massively distributed general AI that can be used - and people on a mass scale allow for slices of their compute to be part of the system?
- tlrobinson 8y ago
  So Skynet, but it lets you rent parts of itself? I'm sure someone is writing an ICO whitepaper for this right now, if they haven't already...
  - nostrademons 8y ago
    EOS: https://eos.io/ https://eos.io/ Raised $2.7B in its ICO, currently trading at a market cap of $10B. FileCoin: https://filecoin.io/ https://filecoin.io/ Raised $257M in its ICO. Tezos: https://tezos.com/ https://tezos.com/ Raised $232M in its ICO. Those are the 3 largest ICOs of all time, so yes, there is definitely a market for renting part of Skynet. The actual technology may or may not be vaporware or a scam. IMHO the way you build a decentralized P2P system is to give a single really smart programmer enough to live on for a couple years and see what he comes up with, not throw a billion dollars at a Cayman Islands corporation that may or may not use it for anything productive. Sorta like what Ethereum did.
    - tlrobinson 8y ago
      Are any of those platforms suitable for running deep learning algorithms? I think Golem is closer https://golem.network/ https://golem.network/ And some others: https://www.investinblockchain.com/distributed-computing-blockchain-projects/ https://www.investinblockchain.com/distributed-computing-blo... But I'm skeptical of distributed computing blockchains. I think a) it's unlikely a distributed compute network can compete with highly optimized datacenters running TPUs or whatever, b) people are unlikely to trust distributed compute networks with their proprietary data (maybe acceptable for CGI rendering and some other specific use-cases)
- PeterisP 8y ago
  The current machine learning paradigms are hard to distribute efficiently - they can be parallelized but require significant ongoing communication between nodes, you can't really split a problem into separate subtasks and merge them only in the end, you need to transfer all the calculated values after, say, each iteration. I'd guess that having ten extra machines in the same rack would be more valuable than a thousand remote machines with limited network bandwidth.
- TeMPOraL 8y ago
  > its a massively distributed general AI You've just described a centralized system. Centralization can happen at different layers - not all technical. The ultimate centralization is ownership, as defined legally.
- hodgesrm 8y ago
  One problem with the SETI model is that a lot of the increase in available compute capacity is due to specialized GPU or TPU processors, which aren't widely available outside of purpose-built data centers. Trying to offload ML workloads to general purpose CPUs would likely be quite wasteful in terms of power consumption unless you can somehow get access to graphics processors.

Yes, unfortunately both data and compute will probably become more and more centralized. At least the algorithmic components have a chance to becone available to everyone.

samstave 8y ago

Here is an off the cuff thought: What if there was (or maybe there already is?) a system which is distributed such as was SETI back in the day and its a massively distributed general AI that can be used - and people on a mass scale allow for slices of their compute to be part of the system?

tlrobinson 8y ago
So Skynet, but it lets you rent parts of itself? I'm sure someone is writing an ICO whitepaper for this right now, if they haven't already...
- nostrademons 8y ago
  EOS: https://eos.io/ https://eos.io/ Raised $2.7B in its ICO, currently trading at a market cap of $10B. FileCoin: https://filecoin.io/ https://filecoin.io/ Raised $257M in its ICO. Tezos: https://tezos.com/ https://tezos.com/ Raised $232M in its ICO. Those are the 3 largest ICOs of all time, so yes, there is definitely a market for renting part of Skynet. The actual technology may or may not be vaporware or a scam. IMHO the way you build a decentralized P2P system is to give a single really smart programmer enough to live on for a couple years and see what he comes up with, not throw a billion dollars at a Cayman Islands corporation that may or may not use it for anything productive. Sorta like what Ethereum did.
  - tlrobinson 8y ago
    Are any of those platforms suitable for running deep learning algorithms? I think Golem is closer https://golem.network/ https://golem.network/ And some others: https://www.investinblockchain.com/distributed-computing-blockchain-projects/ https://www.investinblockchain.com/distributed-computing-blo... But I'm skeptical of distributed computing blockchains. I think a) it's unlikely a distributed compute network can compete with highly optimized datacenters running TPUs or whatever, b) people are unlikely to trust distributed compute networks with their proprietary data (maybe acceptable for CGI rendering and some other specific use-cases)
PeterisP 8y ago
The current machine learning paradigms are hard to distribute efficiently - they can be parallelized but require significant ongoing communication between nodes, you can't really split a problem into separate subtasks and merge them only in the end, you need to transfer all the calculated values after, say, each iteration. I'd guess that having ten extra machines in the same rack would be more valuable than a thousand remote machines with limited network bandwidth.
TeMPOraL 8y ago
> its a massively distributed general AI You've just described a centralized system. Centralization can happen at different layers - not all technical. The ultimate centralization is ownership, as defined legally.
hodgesrm 8y ago
One problem with the SETI model is that a lot of the increase in available compute capacity is due to specialized GPU or TPU processors, which aren't widely available outside of purpose-built data centers. Trying to offload ML workloads to general purpose CPUs would likely be quite wasteful in terms of power consumption unless you can somehow get access to graphics processors.