41 ms·

Dumb question from someone who knows not to much yet about LLMs. How can you trust the other computers? Will I end up with a bunch of swear words coming back fr

Dumb question from someone who knows not to much yet about LLMs. How can you trust the other computers? Will I end up with a bunch of swear words coming back from the other nodes that are playing a prank?

cjbprime 4y ago
I don't think that's a dumb question! I don't know if this project has an answer to that, but there are some techniques (Merkle tree hashes etc) that might work depending on how much of the model you want to download locally. I don't see how to securely scale the inference step, though.
sillysaurusx 4y ago
Sending tensors to two or more nodes is a partial solution, since you can reject ones that don’t match. But fundamentally the answer is yes — bad actors can overwhelm the swarm, and there’s not much you can do about it. I think the project should embrace this limitation. eBay had the same problem, but it’s in a seller’s interest to deliver correct items quickly. Make a social incentive and the rest will work itself out.
- flangola7 4y ago
  If you get banned from eBay your SSN/phone number/email/browser fingerprint/address/etc are prevented from coming back. What system would enforce that for computation nodes?
- 6510 4y ago
  > bad actors can overwhelm the swarm I don't think so. To simplify: You send out 1000 tasks, you perform them yourself, now you have 999 bad flags and 1 good one, you send out 10 tasks to the same including the good one, now you have 990 with 1 bad flag, 9 with 2 and 1 with 2 good ones, you continue sending tasks to the bad nodes and drop their response, if they send you a task you return garbage, you ask the good nodes (with say 100+ good flags) for their list of good nodes and test them one by one. You could build a system where bad nodes have to return so many good responses before getting booted that the joke is on them.
techwizrd 4y ago
I'm not entirely how the approach they're using works [0], but I study federated learning and one of the highly-cited survey papers has several chapters (5 and 6 in particular) addressing potential attacks, failure modes, and bias [1]. 0: https://github.com/learning-at-home/hivemind https://github.com/learning-at-home/hivemind 1: https://arxiv.org/abs/1912.04977 https://arxiv.org/abs/1912.04977
ftxbro 4y ago
They don't solve the problem completely, but they address the problem in their publication as follows. The inference step uses a lot of layers and the first and last layers have more 'interpretability' so some of these are the ones that you solve locally on your own computer. If they wanted to prank you, they would have to use some sophisticated thing that probably doesn't exist yet but it could still be possible. Also if it becomes a problem they could do usual methods like run on multiple other computers and use the majority solution.
- sebzim4500 4y ago
  >If they wanted to prank you, they would have to use some sophisticated thing that probably doesn't exist yet but it could still be possible. Isn't the attack straightforward? i) Take the model, freeze all the weights except the ones you expect to be responsible for ii) Finetune to produce whatever output you are looking for. iii) Profit. Or mainly just annoy people, but it could be funny.
  - ftxbro 4y ago
    OK sure, any time someone says on the internet that something is sophisticated someone will say it's straightforward. So I guess it's a continuum. The thing you describe is more sophisticated than what they would have to do if they had access to the final layer.
px43 4y ago
There's some really cool work being done using Zero Knowledge proofs to write a succinct proof that output from a given model is correct. This is going to be increasingly important not just for these types of distributed systems, but even for things like ChatGPT to make sure that you're actually getting results from the model that you're paying to use. Imagine a world where OpenAI or some other large API provider gets taken over by someone who wants to make money, so they start quietly using smaller, weaker models to respond to API calls, even for customers who are paying for the highest end model. Maybe this is just done at first to survive under massive load, but then someone realizes how many millions of dollars they can save by just forgetting to switch back to the more expensive models. Here's a great primer: https://0xparc.org/blog/zk-mnist https://0xparc.org/blog/zk-mnist
- deleted 4y ago
  [deleted]
winddude 4y ago
You can't, it's somewhere in the petals docs, but they recommend generating in parallel, and averaging, or selecting the response.

cjbprime 4y ago

I don't think that's a dumb question! I don't know if this project has an answer to that, but there are some techniques (Merkle tree hashes etc) that might work depending on how much of the model you want to download locally. I don't see how to securely scale the inference step, though.

sillysaurusx 4y ago

Sending tensors to two or more nodes is a partial solution, since you can reject ones that don’t match. But fundamentally the answer is yes — bad actors can overwhelm the swarm, and there’s not much you can do about it. I think the project should embrace this limitation. eBay had the same problem, but it’s in a seller’s interest to deliver correct items quickly. Make a social incentive and the rest will work itself out.

flangola7 4y ago
If you get banned from eBay your SSN/phone number/email/browser fingerprint/address/etc are prevented from coming back. What system would enforce that for computation nodes?
6510 4y ago
> bad actors can overwhelm the swarm I don't think so. To simplify: You send out 1000 tasks, you perform them yourself, now you have 999 bad flags and 1 good one, you send out 10 tasks to the same including the good one, now you have 990 with 1 bad flag, 9 with 2 and 1 with 2 good ones, you continue sending tasks to the bad nodes and drop their response, if they send you a task you return garbage, you ask the good nodes (with say 100+ good flags) for their list of good nodes and test them one by one. You could build a system where bad nodes have to return so many good responses before getting booted that the joke is on them.

techwizrd 4y ago

I'm not entirely how the approach they're using works [0], but I study federated learning and one of the highly-cited survey papers has several chapters (5 and 6 in particular) addressing potential attacks, failure modes, and bias [1]. 0: https://github.com/learning-at-home/hivemind https://github.com/learning-at-home/hivemind 1: https://arxiv.org/abs/1912.04977 https://arxiv.org/abs/1912.04977

ftxbro 4y ago

They don't solve the problem completely, but they address the problem in their publication as follows. The inference step uses a lot of layers and the first and last layers have more 'interpretability' so some of these are the ones that you solve locally on your own computer. If they wanted to prank you, they would have to use some sophisticated thing that probably doesn't exist yet but it could still be possible. Also if it becomes a problem they could do usual methods like run on multiple other computers and use the majority solution.

sebzim4500 4y ago
>If they wanted to prank you, they would have to use some sophisticated thing that probably doesn't exist yet but it could still be possible. Isn't the attack straightforward? i) Take the model, freeze all the weights except the ones you expect to be responsible for ii) Finetune to produce whatever output you are looking for. iii) Profit. Or mainly just annoy people, but it could be funny.
- ftxbro 4y ago
  OK sure, any time someone says on the internet that something is sophisticated someone will say it's straightforward. So I guess it's a continuum. The thing you describe is more sophisticated than what they would have to do if they had access to the final layer.

px43 4y ago

There's some really cool work being done using Zero Knowledge proofs to write a succinct proof that output from a given model is correct. This is going to be increasingly important not just for these types of distributed systems, but even for things like ChatGPT to make sure that you're actually getting results from the model that you're paying to use. Imagine a world where OpenAI or some other large API provider gets taken over by someone who wants to make money, so they start quietly using smaller, weaker models to respond to API calls, even for customers who are paying for the highest end model. Maybe this is just done at first to survive under massive load, but then someone realizes how many millions of dollars they can save by just forgetting to switch back to the more expensive models. Here's a great primer: https://0xparc.org/blog/zk-mnist https://0xparc.org/blog/zk-mnist

deleted 4y ago
[deleted]

winddude 4y ago

You can't, it's somewhere in the petals docs, but they recommend generating in parallel, and averaging, or selecting the response.