3 ms·

Somebody comes up with a "useful proof of work" a few times a year, and so far there has always been a reason why the proposed mechanism is infeasible. In this

by drcode 7y ago

Somebody comes up with a "useful proof of work" a few times a year, and so far there has always been a reason why the proposed mechanism is infeasible. In this example, the reasons are (1) the ML tasks require large amounts of data, which makes them infeasible for a blockchain POW algo and (2) in their proposed scheme, there is no way for a regular node to validate the blockchain, because it has to trust a separate node they call a "verifier". (but I only glanced at the paper quickly, if someone has the time to do a thorough analysis, please feel free to explain why I'm wrong)

tastroder 7y ago
I'd like to add to this list: The related work section closes with "Our PoUW facilitates better AI through decentralisation, enhanced security and the right incentives.". Decentralisation: Their own paper states in 7.2 that "distributed solution will never be as efficient as running the whole training on a local machine". What incentive for buying into this would I have here then? I occasionally use cloud training if what I'm doing exceeds the capability I have locally. As a client I don't particularly care about the internal infrastructure of whoever I pay to do that (with small exceptions like the next point). Enhanced security: Security that's only required because of me using a blockchain solution? 7.3 starts out with "We assume that the majority of participants is honest.". That's an assumption I can make today with the cloud providers I pay for my training time, without any blockchain overhead. Nothing in that section seems to be related to real world security concerns, e.g. trusting random mining worker nodes with my - potentially proprietary - training data and intermittent results. 3.4, the outline of a task definition, also specifies that I basically provide the full model and really convenient formats to steal all of that away from me. I think the major risk factor of using the idea outlined in the paper doesn't have anything to do with a revolution of cryptocurrencies, but one of the corporate espionage in the ML world it would enable. Right incentives: For me, as a client, a theoretical 30% savings doesn't outweigh the above (and all the ancillary considerations like infrastructure, ...). If any of their economical analysis is off, they just generate more people going back to bitcoin mining on the other side of the equation. Not sure what the right incentive is supposed to be here. This might be a good system to proove useful work but somehow I don't see the text convincing the reader that the system itself is useful, maybe I missed it.
- onefuncman 7y ago
  assuming the participants are honest is useful because someone else can prove how to participate in a trust-less fashion, e.g. via homomorphic encryption
  - tastroder 7y ago
    That kind of sounds too "move fast and break things" for something that's essentially a systems description with an actual implementation they likely want to sell. At any rate, it might be useful for the Blockchain part of the system I don't care too much about. My comment was meant to address the actual work part of the system. It relies on client trust w.r.t. not only the network but also the models and data they put in, which I don't even see mentioned in the document. I don't see how the architecture described in there would be able to even attempt to break down training processes enough to prevent a malicious actor from extracting substantial parts of the system without crippling performance.
tromp 7y ago
Being cheap to verify is one of the defining characteristics of a good PoW. Only PoW verification allows you to identify the chain with the most work. Which is needed to identify how much money the verifiers have at stake. So trusting verifiers to help you identify the most worked chain has no solid footing.
- zozbot234 7y ago
  Any "search"-like problem can become cheap to verify simply by augmenting the solution with a verification certificate. It's quite likely that many ML training tasks are "search"-like in this sense, hence cheap to verify as well.
  - octocop 7y ago
    >Any "search"-like problem This is a pretty broad statement. Verifying a shortest path or something like this would not be cheap to verify
reagent_finder 7y ago
(1) Not all ML needs massive amounts of data. For instance Leela Zero and LCZero want a bunch of horsepower instead. So a task that features a simple set of rules but tries to find the best way to do it should work just fine here. (2) Having a 'verifier' is how mining pools work already, so it's not adding complexity or anything else. That said, I didn't read the paper either, but I would say thinking about how to harness the massive resources being used to mine crypto these days is nowhere near a wasted effort. At some point, someone is going to look at all the things that have been proposed and the way some network is set up and it will click.

by drcode 7y ago

tastroder 7y ago

I'd like to add to this list: The related work section closes with "Our PoUW facilitates better AI through decentralisation, enhanced security and the right incentives.". Decentralisation: Their own paper states in 7.2 that "distributed solution will never be as efficient as running the whole training on a local machine". What incentive for buying into this would I have here then? I occasionally use cloud training if what I'm doing exceeds the capability I have locally. As a client I don't particularly care about the internal infrastructure of whoever I pay to do that (with small exceptions like the next point). Enhanced security: Security that's only required because of me using a blockchain solution? 7.3 starts out with "We assume that the majority of participants is honest.". That's an assumption I can make today with the cloud providers I pay for my training time, without any blockchain overhead. Nothing in that section seems to be related to real world security concerns, e.g. trusting random mining worker nodes with my - potentially proprietary - training data and intermittent results. 3.4, the outline of a task definition, also specifies that I basically provide the full model and really convenient formats to steal all of that away from me. I think the major risk factor of using the idea outlined in the paper doesn't have anything to do with a revolution of cryptocurrencies, but one of the corporate espionage in the ML world it would enable. Right incentives: For me, as a client, a theoretical 30% savings doesn't outweigh the above (and all the ancillary considerations like infrastructure, ...). If any of their economical analysis is off, they just generate more people going back to bitcoin mining on the other side of the equation. Not sure what the right incentive is supposed to be here. This might be a good system to proove useful work but somehow I don't see the text convincing the reader that the system itself is useful, maybe I missed it.

onefuncman 7y ago
assuming the participants are honest is useful because someone else can prove how to participate in a trust-less fashion, e.g. via homomorphic encryption
- tastroder 7y ago
  That kind of sounds too "move fast and break things" for something that's essentially a systems description with an actual implementation they likely want to sell. At any rate, it might be useful for the Blockchain part of the system I don't care too much about. My comment was meant to address the actual work part of the system. It relies on client trust w.r.t. not only the network but also the models and data they put in, which I don't even see mentioned in the document. I don't see how the architecture described in there would be able to even attempt to break down training processes enough to prevent a malicious actor from extracting substantial parts of the system without crippling performance.

tromp 7y ago

Being cheap to verify is one of the defining characteristics of a good PoW. Only PoW verification allows you to identify the chain with the most work. Which is needed to identify how much money the verifiers have at stake. So trusting verifiers to help you identify the most worked chain has no solid footing.

zozbot234 7y ago
Any "search"-like problem can become cheap to verify simply by augmenting the solution with a verification certificate. It's quite likely that many ML training tasks are "search"-like in this sense, hence cheap to verify as well.
- octocop 7y ago
  >Any "search"-like problem This is a pretty broad statement. Verifying a shortest path or something like this would not be cheap to verify

reagent_finder 7y ago

(1) Not all ML needs massive amounts of data. For instance Leela Zero and LCZero want a bunch of horsepower instead. So a task that features a simple set of rules but tries to find the best way to do it should work just fine here. (2) Having a 'verifier' is how mining pools work already, so it's not adding complexity or anything else. That said, I didn't read the paper either, but I would say thinking about how to harness the massive resources being used to mine crypto these days is nowhere near a wasted effort. At some point, someone is going to look at all the things that have been proposed and the way some network is set up and it will click.