5 ms·

This is a nice effort if it allows you to run bloom 170B in 1s per token. Just for comparison sake. With a last gen Ryzen cpu (16core) it takes me about 90s to

This is a nice effort if it allows you to run bloom 170B in 1s per token. Just for comparison sake. With a last gen Ryzen cpu (16core) it takes me about 90s to run the model with 32gb ram (the entire model uses few GB of nvme storage too, as 32gb isn't enough ram). However, I wonder how they prevent abuse. The main page doesn't mention it. As they mentioned block chain I suspect there will be some sort of credits implemented. I'll definitely be watching where this project goes. Edit:just to clarify the 90s is not the 170b parameter model. It is 7b bloom version. I forgot to mention it and it puts the ability to run a 170B model in 1s in better perspective.

borzunov 4y ago
A Petals dev here. At the moment, we're working on a centralized incentive system, no blockchain involved. It will award points if someone is running a server that consistently stays online and returns correct results. Then, users will be able to spend these points for prioritized inference and (maybe) extra features like increased sequence length/batch size limits. This way, the swarm will prioritize people who actually contribute compute and serve others in the remaining idle time.
- narrator 4y ago
  Maybe you could get Bram Cohen to work on this. Seriously, reach out to him, he loves to work on these game theory sorts of things.
  - oefrha 4y ago
    I’d say his reputation suffered quite a bit after the whole Chia (that proof of SSD thrashing coin) BS.
- prettyStandard 4y ago
  Is it possible to have the server shutdown predictably when it finishes tasks periodically, and not get penalized? I would like my machine to run while I'm not using it.
  - borzunov 4y ago
    Sure! People who disconnect for a while (not necessarily predictably) won't be penalized - it's okay if you suddenly decide to use your GPU for something else, then get back to running a server.

borzunov 4y ago

A Petals dev here. At the moment, we're working on a centralized incentive system, no blockchain involved. It will award points if someone is running a server that consistently stays online and returns correct results. Then, users will be able to spend these points for prioritized inference and (maybe) extra features like increased sequence length/batch size limits. This way, the swarm will prioritize people who actually contribute compute and serve others in the remaining idle time.

narrator 4y ago
Maybe you could get Bram Cohen to work on this. Seriously, reach out to him, he loves to work on these game theory sorts of things.
- oefrha 4y ago
  I’d say his reputation suffered quite a bit after the whole Chia (that proof of SSD thrashing coin) BS.
prettyStandard 4y ago
Is it possible to have the server shutdown predictably when it finishes tasks periodically, and not get penalized? I would like my machine to run while I'm not using it.
- borzunov 4y ago
  Sure! People who disconnect for a while (not necessarily predictably) won't be penalized - it's okay if you suddenly decide to use your GPU for something else, then get back to running a server.