3 ms·

> We do this with a large amount of random data. Many TBs or perhaps as much as a PB. What's the tradeoff in using a much smaller pool and severely rate limiti

by Canada 10y ago

> We do this with a large amount of random data. Many TBs or perhaps as much as a PB. What's the tradeoff in using a much smaller pool and severely rate limiting its network interface?

zaroth 10y ago
The protocol involves about 64 bytes in and 64 bytes response. Technically you could shrink it a bit if you needed, but let's start there. If you want 1,000 logins per second you limit yourself to 64,000 bytes / sec egress. This is hand wavy - there is some overhead but I'm not counting front-end traffic, TLS handshakes, etc. just the request/response against the internal data pool server. If you want it to take 100 days at full rate to steal the pool, then that works out to 64,000 * 60 * 60 * 24 * 100 = 552GB. You can try to set the limit even lower, to allow an even smaller pool but that's the basic math. Just 10 logins/sec --> 5.5GB. But if you depend too much on software limits and not PHY layer limits on the network speed you certainly run the risk of having your software limits blown away and the data pool slipping away much faster than you expected. You want a physical limitation to ensure the data will not get away from you over the network. When you get to the point where a successful attack would require physically removing a hundred disks which are self-encrypted SSDs then you sleep more soundly at night!
- kbaker 10y ago
  What happens if someone is sniffing the line? It wouldn't even need to be your fault, it could be some logger on your customer's server... i.e. do you have any protection against someone including this in the client code: DEBUG("testing external service... " + Hash1) Salt2 = call_external_salt_service_api(Hash1) DEBUG("service returned: " + Salt2) If someone was logging this traffic, wouldn't this allow your service to be treated as a black box and nullify any additional security? They wouldn't need to steal the full pool in this case, as you are sending back the exact hash/salt values needed for every user that logs in.

by Canada 10y ago

> We do this with a large amount of random data. Many TBs or perhaps as much as a PB. What's the tradeoff in using a much smaller pool and severely rate limiting its network interface?

zaroth 10y ago

The protocol involves about 64 bytes in and 64 bytes response. Technically you could shrink it a bit if you needed, but let's start there. If you want 1,000 logins per second you limit yourself to 64,000 bytes / sec egress. This is hand wavy - there is some overhead but I'm not counting front-end traffic, TLS handshakes, etc. just the request/response against the internal data pool server. If you want it to take 100 days at full rate to steal the pool, then that works out to 64,000 * 60 * 60 * 24 * 100 = 552GB. You can try to set the limit even lower, to allow an even smaller pool but that's the basic math. Just 10 logins/sec --> 5.5GB. But if you depend too much on software limits and not PHY layer limits on the network speed you certainly run the risk of having your software limits blown away and the data pool slipping away much faster than you expected. You want a physical limitation to ensure the data will not get away from you over the network. When you get to the point where a successful attack would require physically removing a hundred disks which are self-encrypted SSDs then you sleep more soundly at night!

kbaker 10y ago
What happens if someone is sniffing the line? It wouldn't even need to be your fault, it could be some logger on your customer's server... i.e. do you have any protection against someone including this in the client code: DEBUG("testing external service... " + Hash1) Salt2 = call_external_salt_service_api(Hash1) DEBUG("service returned: " + Salt2) If someone was logging this traffic, wouldn't this allow your service to be treated as a black box and nullify any additional security? They wouldn't need to steal the full pool in this case, as you are sending back the exact hash/salt values needed for every user that logs in.