5 ms·
Distributed Inference and Fine-Tuning of Large Language Models over the Internet
- deleted 3y ago[deleted]
- amrb 3y agoCan check out their project at https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals
- knightoffaith 3y agoGreat that folks are starting to think about this, I think it's important to keep the power of AI decentralized. Thought it seems like for this to be practical, they need to flesh out the Security section in the appendix to deal with bad actors. I like the probabilistic approach in lieu of full byzantine fault tolerance. Would be interesting to see what kind of guarantees we could still have about correctness/convergence even with bad actors.
- 3abiton 3y agoStep in the right direction, finally distributed and decentralized technology that has value other than crypto.
- alexpotato 3y agoMakes Skynet harder to shutdown if it's distributed too.
- sjwhevvvvvsj 3y agoConversely makes it possible to fight some closed corpo-mil skynet with a community supported skynet.
- alexpotato 3y agoGood to own the means of production
- sjwhevvvvvsj 3y agoYou’ll take my GPU from my cold dead hands.
- pama 3y agoHere is the main arxiv page (html is a mess on lockdown mode): https://arxiv.org/abs/2312.08361 https://arxiv.org/abs/2312.08361
- bugglebeetle 3y agoA distributed network of GPUs for something other than funny-money sudoku puzzle-solving is a nice development.
- lifty 3y agoWould be nice to have some kind of proof of useful AI work.
- monkeydust 3y agoMade me think of Gridcoin and BOINC https://boinc.berkeley.edu/ https://boinc.berkeley.edu/
- sdenton4 3y agoIf you have to pay $X for AI inference, then it proves that the work is worth at least $X to you... Which suggests the obvious marketplace scheme: Compute providers are paid (or credited) $X/k to contribute their nodes to the network, and users pay $X for inference. (inevitably, someone will implement this with o.g. bitcoin and we can all cry.)
- csdvrx 3y ago> If you have to pay $X for AI inference, then it proves that the work is worth at least $X to you... Which suggests the obvious marketplace scheme: Compute providers are paid (or credited) $X/k to contribute their nodes to the network, and users pay $X for inference. Yes, and that seems good > (inevitably, someone will implement this with o.g. bitcoin and we can all cry.) Why should we cry? If the person providing the GPU in the distributed inference system can be "credited" on a ledger with some unit that will let them use other people's GPU to do their own inference in the future (and proportionally to what they themselves provided in the past), I see that as a way to arbitrate between present and future wants for GPU capacity. You may have little use for your GPU 24/7, so you could let it chew numbers to fulfil other people AI needs, but tou may not want to do that if you're left with the electricity bill! Yet if in return you would be provided with some credit that lets you use other people GPU when you need it, you may be more willing to be cooperative instead of keeping your computer turned off ur using your GPU just for yourself. There's also a mutual interest in cooperation: I wouldn't cry if I need in 1 second more than my GPU could do in 1 hour, and if this scheme existed, as it'd mean I wouldn't have to wait. You shouldn't cry either: if you have contributed to the network for say 2 days in the past, you can "redeem" the equivalent of these computations immediately in 2 seconds, which is better than waiting or paying to use a company API. The accounting may seem futile, but I think it ensures you can't waste more resources that what you provide: this is the proportionality between past/present/future wants for GPU (no leeching) Can you explain me what you mean by "cry", or why you imply it would be a bad thing?
- sjwhevvvvvsj 3y agoI’ve long assumed something like this existed in a corporate lab but hadn’t been made public (or, ahem, “open”). It’s an obvious move as open source models get bigger, really happy to see this out in the world - especially with a HF author attached.
- dekervin 3y agoWow ! I was basically mulling over it earlier today in the HN learn discord[1], for a practical project. If you want to tag along to do something useful, come say hi (the more the merrier ! ). [1] https://discord.gg/SAAn3xXC https://discord.gg/SAAn3xXC
- josemanuel 3y agoIs this skynet?!?