4 ms·
The overlap between the hardware requirements for model training and web3 mining has me thinking could there be a coin where the proof of work could be trainin
by manquer 4y ago
The overlap between the hardware requirements for model training and web3 mining has me thinking could there be a coin where the proof of work could be training a model ?
- manigandham 4y agoPotentially, I was recently introduced to this: https://bws.xyz/ https://bws.xyz/ Not affiliated and haven't researched yet but looks interesting. There are a few other crypto mining companies offering general compute to manage changing power/price/hardware resources.
- londons_explore 4y agoUnfortunately ML model training is too easy to 'cheat' - ie. To look at the test data to fine-tune the model to overfit a lot, do really well at test time, but actually be useless for real world stuff. To work out, you'd need a way to keep some test data secret, which in turn kinda kills decentralisation.
- mejutoco 4y agoYou could do something like kaggle does, where you have some test data but the final score is determined by a more complete test data that you don't have access to.
- nonoesp 4y agoWouldn't you then be penalized when you train a model that yields bad results on the test data, even if you're doing what you're meant to be doing?
- mejutoco 4y agoAFAIK it works like this: you have a test data you develop against and some secret (bigger) test data that you only have access for the final score. While you are developing you can overfit if you want, but then probably you won't perform that well with the secret tests. What you are meant to be doing is to perform with the test data without overfiting. Even if it is not optimal, it mostly solves the overfiting problem. It might work for a cryptocurrency too.
- Sebb767 4y agoI think the problem described is the following: 1. A researcher puts out a model with bad initial parameters/data. 2. The chain workers/miners train the model as request. 3. The model fails at the test or verification dataset due to the bad setup. In this case, the miners would not get paid despite doing exactly what was asked of them.
- IanCal 4y agoYou don't need to test that they perform well, just that they perform the same (for algorithms that should be bit for bit identical) / similarly (for ones that are less so). If multiple people train the same thing and the results are the same you could trust they have faithfully run the training as asked. That rewards "train as asked" rather than "train and get a good result". Lack of trust there can be addressed with a few other things like staking, and a broad desire for all miners that people trust the system as a whole. Quite how to design those things is a complex problem but not insurmountable I think. Disclaimer, this kind of decentralised more useful kind of work is something I'm investigating now.
- manquer 4y agoMany coins are not really decentralized , there is some centralization anyways as we recently heard with the Ronin bridge attack or many examples from before. I am not sure that is a really big turn off.
- IshKebab 4y agoI don't think there's much overlap. Crypto stuff is all integer maths as far as I know (maybe there are some weird floating point coins?) but ML is all low precision floating point. ML training hardware has a ton of powerful FPUs but much less integer compute.
- toxik 4y agoInteresting idea for proof of work, a large nonlinear function where you find weights so that f(x_0) = x_1 < x_0 or something like that. To exploit ML advances.
- manquer 4y agoI meant the hardware not the math, while there are new GPUs which disable mining, and there are GPUs and ASICs designed for mining, the fact is last 5 years, the GPU market has been hogged by mining.