8 ms·

It’s interesting that it does something useful (training a LLM) without trust and in a decentralized way. Maybe this could be used as proof of work? To stop wa

by iTokio 1y ago

It’s interesting that it does something useful (training a LLM) without trust and in a decentralized way. Maybe this could be used as proof of work? To stop wasting computing resources in crypto currencies and get something useful as a byproduct.

proof_by_vibes 1y ago
There could be merit to this. Proofs are generally computationally hard, so it's possible that a currency could be created by quantifying verification.
littlestymaar 1y ago
> To stop wasting computing resources in crypto currencies and get something useful as a byproduct. Bitcoin is the only major cryptocurrency that still use proof of work today (others are either using “proof of stakes” or are “Layer 2” chains), and due to its (relative lack of) governance structure, it's very unlikely to ever change.
fastball 1y ago
The emphasis is indeed on "without trust" – as far as I can tell this project is unable to verify whether the decentralized training nodes are contributing productively. Without the ability to validate that training compute is heading in the globally desired direction, it is unlikely you could use it as the foundation of a (sound) cryptocurrency.
- mentalgear 1y ago
  The reward model could be used as a validation/reward for the client. Give the same nodes the same inferences to make, and the one with the highest reward (those could be short, or even partially calculated long-term) will also get the "currency" reward.
  - Philpax 1y ago
    That sounds like it'll lead to human-driven reward hacking [0]? [0]: https://en.wikipedia.org/wiki/Reward_hacking https://en.wikipedia.org/wiki/Reward_hacking
  - bastawhiz 1y ago
    Arguably that's worse than crypto proof of work: inference is extremely expensive and you're multiplying every operation by N. Which means the cost is multiplied by N. And like, what are you doing? You've managed to find a use case where you don't care that you're doing compute on some untrusted servers online (and no, there's no magic AI homomorphic encryption) but at the same time you're willing to accept the latency of doing the work multiple times AND it's probably all low end 4090s doing the work AND you're willing to pay for the wasted compute? I'm here shuddering at the thought of model setup times when one node in a cluster goes down and you're facing that on... well, probably most inferences? If you're not administering the infra, you get the lowest common denominator of performance.
mentalgear 1y ago
That would be indeed a very promising way of FINALLY making cryptocurrency useful!
_ink_ 1y ago
I read an argument, that proof of work needs to be useless and wasteful. If it would produce value in itself it would make 51% attacks more economic and thus the currency less secure.
- throwanem 1y ago
  Sure. The whole point of "proof of work" is to show (prove) you've lost energy to heat (work). That's what makes it costly and thus an honest signal. The model breaks where work can be counterfeited (usually impossible) or where energy prices go to zero, which is why "bitcoin colonialism" was briefly a thing last decade. Much of bitcoin's design, this aspect also, is intended to protect against the bare-fanged, red-eyed money weasels it was also designed to attract.
- ucha 1y ago
  It needs to not have economic value but it doesn't necessarily need to be useless and wasteful.
  - Xmd5a 1y ago
    For instance if the end product, in this case the LLM, is made available to anyone, publicly...
  - piiToo 1y ago
    If it improves the economic value of something else it has economic value just not on its own discrete value. Wrappers on candy don’t have value intrinsically but improve the quality of the candy.
- api 1y ago
  I’ve seen an argument that military power and credible threat are the proof of work mechanism for fiat currencies. That is also useless, but it does throw off secondary useful effects like inventions. Not totally convinced the analogy maps but interesting.
  - genewitch 1y ago
Geee 1y ago
No, this process doesn't produce "proof of work", i.e. verifiable proofs that energy has been used.
- deleted 1y ago
  [deleted]
- naasking 1y ago
  New weights that have lower loss than the input weights is proof that work has been done.
k__ 1y ago
Arweave and Filecoin use PoW algorithms that prove something useful.
bastawhiz 1y ago
> Maybe this could be used as proof of work There's nothing provable here. Crypto proof of work is easily verified (does the hash of this value look the way I expect?). How do you prove in ~O(1) time that someone did some operation with their GPU? You don't. You don't even know what the thing is that you're training (without a trained model you don't have the ability to know whether the model the was allegedly trained learned the thing you want it to learn).
- naasking 1y ago
  > How do you prove in ~O(1) time that someone did some operation with their GPU? You don't. The work in this case could be that the weights after the was done work have lower loss than the input weights. Applying the new weights to input to check that it's lower is much cheaper than calculating the weights, which is the same trend as proof of work (not sure about the magnitude of difficulty being enough to replace proof of work though).
  - refulgentis 1y ago
    [flagged]
  - refulgentis 1y ago
    Trying again, apologies: - Minimizing loss could be a useful heuristic on a base model. Here, we expect the distribution to be different as we are only doing RL. Measuring loss means we're measuring the difference against the base model inputs: a non-goal, we expect reasoning post RL-training to look quite different from a web scrape. Let's set that aside. Let's say lower loss = model improved. - Checking the loss requires the entire dataset used to train the base model + forward pass. That’s O(N·d) where N is samples, d is model size. This takes us from "cool demo of RL can be done on the edge with little benefit" to "we're shipping around terabytes of data constantly among clients" - Proof of work as a technical term is different from proof of work as a colloquial term: the former is a cryptographic puzzle whose solution is universally and instantly checkable, while the latter just means “I can show I did something,” with no strict guarantee or uniqueness. Randomly perturbing one parameter could show "proof of work" without the work we actually wanted to be done, being done. - Early in base model training, shaving 0.01 off the loss is easy. Later, impossible. In an RL environment, we're expecting some to go bad. In our interpretation of "loss decrease means model better means you did work", that would mean loss would increase -- that is how it learns in an RL environment. However, that does not mean no work is done.

by iTokio 1y ago

proof_by_vibes 1y ago

There could be merit to this. Proofs are generally computationally hard, so it's possible that a currency could be created by quantifying verification.

littlestymaar 1y ago

> To stop wasting computing resources in crypto currencies and get something useful as a byproduct. Bitcoin is the only major cryptocurrency that still use proof of work today (others are either using “proof of stakes” or are “Layer 2” chains), and due to its (relative lack of) governance structure, it's very unlikely to ever change.

fastball 1y ago

The emphasis is indeed on "without trust" – as far as I can tell this project is unable to verify whether the decentralized training nodes are contributing productively. Without the ability to validate that training compute is heading in the globally desired direction, it is unlikely you could use it as the foundation of a (sound) cryptocurrency.

mentalgear 1y ago
The reward model could be used as a validation/reward for the client. Give the same nodes the same inferences to make, and the one with the highest reward (those could be short, or even partially calculated long-term) will also get the "currency" reward.
- Philpax 1y ago
  That sounds like it'll lead to human-driven reward hacking [0]? [0]: https://en.wikipedia.org/wiki/Reward_hacking https://en.wikipedia.org/wiki/Reward_hacking
- bastawhiz 1y ago
  Arguably that's worse than crypto proof of work: inference is extremely expensive and you're multiplying every operation by N. Which means the cost is multiplied by N. And like, what are you doing? You've managed to find a use case where you don't care that you're doing compute on some untrusted servers online (and no, there's no magic AI homomorphic encryption) but at the same time you're willing to accept the latency of doing the work multiple times AND it's probably all low end 4090s doing the work AND you're willing to pay for the wasted compute? I'm here shuddering at the thought of model setup times when one node in a cluster goes down and you're facing that on... well, probably most inferences? If you're not administering the infra, you get the lowest common denominator of performance.

mentalgear 1y ago

That would be indeed a very promising way of FINALLY making cryptocurrency useful!

_ink_ 1y ago

I read an argument, that proof of work needs to be useless and wasteful. If it would produce value in itself it would make 51% attacks more economic and thus the currency less secure.

throwanem 1y ago
Sure. The whole point of "proof of work" is to show (prove) you've lost energy to heat (work). That's what makes it costly and thus an honest signal. The model breaks where work can be counterfeited (usually impossible) or where energy prices go to zero, which is why "bitcoin colonialism" was briefly a thing last decade. Much of bitcoin's design, this aspect also, is intended to protect against the bare-fanged, red-eyed money weasels it was also designed to attract.
ucha 1y ago
It needs to not have economic value but it doesn't necessarily need to be useless and wasteful.
- Xmd5a 1y ago
  For instance if the end product, in this case the LLM, is made available to anyone, publicly...
- piiToo 1y ago
  If it improves the economic value of something else it has economic value just not on its own discrete value. Wrappers on candy don’t have value intrinsically but improve the quality of the candy.
api 1y ago
I’ve seen an argument that military power and credible threat are the proof of work mechanism for fiat currencies. That is also useless, but it does throw off secondary useful effects like inventions. Not totally convinced the analogy maps but interesting.
- genewitch 1y ago

Geee 1y ago

No, this process doesn't produce "proof of work", i.e. verifiable proofs that energy has been used.

deleted 1y ago
[deleted]
naasking 1y ago
New weights that have lower loss than the input weights is proof that work has been done.

k__ 1y ago

Arweave and Filecoin use PoW algorithms that prove something useful.

bastawhiz 1y ago

> Maybe this could be used as proof of work There's nothing provable here. Crypto proof of work is easily verified (does the hash of this value look the way I expect?). How do you prove in ~O(1) time that someone did some operation with their GPU? You don't. You don't even know what the thing is that you're training (without a trained model you don't have the ability to know whether the model the was allegedly trained learned the thing you want it to learn).

naasking 1y ago
> How do you prove in ~O(1) time that someone did some operation with their GPU? You don't. The work in this case could be that the weights after the was done work have lower loss than the input weights. Applying the new weights to input to check that it's lower is much cheaper than calculating the weights, which is the same trend as proof of work (not sure about the magnitude of difficulty being enough to replace proof of work though).
- refulgentis 1y ago
  [flagged]
- refulgentis 1y ago
  Trying again, apologies: - Minimizing loss could be a useful heuristic on a base model. Here, we expect the distribution to be different as we are only doing RL. Measuring loss means we're measuring the difference against the base model inputs: a non-goal, we expect reasoning post RL-training to look quite different from a web scrape. Let's set that aside. Let's say lower loss = model improved. - Checking the loss requires the entire dataset used to train the base model + forward pass. That’s O(N·d) where N is samples, d is model size. This takes us from "cool demo of RL can be done on the edge with little benefit" to "we're shipping around terabytes of data constantly among clients" - Proof of work as a technical term is different from proof of work as a colloquial term: the former is a cryptographic puzzle whose solution is universally and instantly checkable, while the latter just means “I can show I did something,” with no strict guarantee or uniqueness. Randomly perturbing one parameter could show "proof of work" without the work we actually wanted to be done, being done. - Early in base model training, shaving 0.01 off the loss is easy. Later, impossible. In an RL environment, we're expecting some to go bad. In our interpretation of "loss decrease means model better means you did work", that would mean loss would increase -- that is how it learns in an RL environment. However, that does not mean no work is done.