4 ms·

That's what I am afraid of the most. Do some devops mistake and get penalized. Are there any protection mechanisms to help you with that or are you totally on y

by e9 4y ago

That's what I am afraid of the most. Do some devops mistake and get penalized. Are there any protection mechanisms to help you with that or are you totally on your own?

dannyw 4y ago
There are slashing protection features built into all clients, but the general advice is to don't do devops. You only get slashed if you run malicious custom code, or if you run the same key on two machines at the same time. The latter is a mistake that's very easy to make if you try to build auto-failover. So don't build it. My validators operate with 'screen' and shell scripts. If I need to do anything (3-4 times a year), I do it manually.
- e9 4y ago
  I guess couple of questions out of curiosity: - machine dies at 4am, what happens? do you manually restart process when you wake up? does this down time cause slashing? - what mechanism prevents you from running 2 identical validators by accident?
  - 3np 4y ago
    An explicit goal is that running a validator at home for the enthusiast should be feasible. A couple of days total downtime per year for an individual validator should approximately only miss out on the rewards it would have generated by being available for the same period. Penalties increase non-linearly with downtime and number of validators unavailable at the same time (meaning assuming all else equal, being down for 100h at once is worse than being down 1h/d for 100 days, and a whale with 10 validators going down for 10h at the same time will lose more than an individual validator going down 10h at 10 different occasions. This is not contingent on those validators being attributable to the same entity) If you have monitoring that alerts you for updates and when your node is unresponsive and can respond to downtime requiting intervention within a day or two you should still end up ETH-positive.
    - e9 4y ago
      Thank you so much for this information
  - pa7x1 4y ago
    The rough breakeven time is 42% of the time. If your validator is online more than that you are making money, less than that you lose it. So the protocol is rather forgiving in uptime. That's why it's much better to not run failover of any kind. You can afford to be offline without problems but if your failover fails and you run two instances you get slashed. There is also a penalty that kicks in if many validators fail at the same time resulting in over 1/3 of the network not being available. In that case penalties are also increased significantly. So if over 1/3 of the network were to be run in AWS those validators would risk heavy penalties if the region were to suffer some downtime.
- bogomipz 4y ago
  >"There are slashing protection features built into all clients, but the general advice is to don't do devops." I'm curious how the big crypto companies are managing these large fleets of servers without doing some form of Devops? Looking at the docs there's a stack that needs to be deployed and maintained: https://ethereum.org/en/developers/docs/nodes-and-clients/run-a-node/ https://ethereum.org/en/developers/docs/nodes-and-clients/ru...
  - yokem55 4y ago
    They use special signing software (dirk or web3signer) that can have the keys split accross several physical machines. If they have separate access credentials for each system, it makes it difficult for anyone to have access to the complete keys. Then a central validator client can request signatures from each signer to complete attestations and block proposals.
    - bogomipz 4y ago
      This didn't really answer my question though. There's other software besides key management services that run on those nodes. How is that being managed if these orgs are not doing some form of Devops?

by e9 4y ago

That's what I am afraid of the most. Do some devops mistake and get penalized. Are there any protection mechanisms to help you with that or are you totally on your own?

dannyw 4y ago

There are slashing protection features built into all clients, but the general advice is to don't do devops. You only get slashed if you run malicious custom code, or if you run the same key on two machines at the same time. The latter is a mistake that's very easy to make if you try to build auto-failover. So don't build it. My validators operate with 'screen' and shell scripts. If I need to do anything (3-4 times a year), I do it manually.

e9 4y ago
I guess couple of questions out of curiosity: - machine dies at 4am, what happens? do you manually restart process when you wake up? does this down time cause slashing? - what mechanism prevents you from running 2 identical validators by accident?
- 3np 4y ago
  An explicit goal is that running a validator at home for the enthusiast should be feasible. A couple of days total downtime per year for an individual validator should approximately only miss out on the rewards it would have generated by being available for the same period. Penalties increase non-linearly with downtime and number of validators unavailable at the same time (meaning assuming all else equal, being down for 100h at once is worse than being down 1h/d for 100 days, and a whale with 10 validators going down for 10h at the same time will lose more than an individual validator going down 10h at 10 different occasions. This is not contingent on those validators being attributable to the same entity) If you have monitoring that alerts you for updates and when your node is unresponsive and can respond to downtime requiting intervention within a day or two you should still end up ETH-positive.
  - e9 4y ago
    Thank you so much for this information
- pa7x1 4y ago
  The rough breakeven time is 42% of the time. If your validator is online more than that you are making money, less than that you lose it. So the protocol is rather forgiving in uptime. That's why it's much better to not run failover of any kind. You can afford to be offline without problems but if your failover fails and you run two instances you get slashed. There is also a penalty that kicks in if many validators fail at the same time resulting in over 1/3 of the network not being available. In that case penalties are also increased significantly. So if over 1/3 of the network were to be run in AWS those validators would risk heavy penalties if the region were to suffer some downtime.
bogomipz 4y ago
>"There are slashing protection features built into all clients, but the general advice is to don't do devops." I'm curious how the big crypto companies are managing these large fleets of servers without doing some form of Devops? Looking at the docs there's a stack that needs to be deployed and maintained: https://ethereum.org/en/developers/docs/nodes-and-clients/run-a-node/ https://ethereum.org/en/developers/docs/nodes-and-clients/ru...
- yokem55 4y ago
  They use special signing software (dirk or web3signer) that can have the keys split accross several physical machines. If they have separate access credentials for each system, it makes it difficult for anyone to have access to the complete keys. Then a central validator client can request signatures from each signer to complete attestations and block proposals.
  - bogomipz 4y ago
    This didn't really answer my question though. There's other software besides key management services that run on those nodes. How is that being managed if these orgs are not doing some form of Devops?