3 ms·

This is my paper (along with Tao and Fan). It's a great feeling to have this published and available, and I'm super proud of the team behind Physalia. There's

by mjb 7y ago

This is my paper (along with Tao and Fan). It's a great feeling to have this published and available, and I'm super proud of the team behind Physalia. There's a lighter-weight introduction to the work here: https://www.amazon.science/blog/amazon-ebs-addresses-the-challenge-of-the-cap-theorem-at-scale https://www.amazon.science/blog/amazon-ebs-addresses-the-cha... and for those attending NSDI, I'll be talking about Physalia in the "Deployment Experience" session on Wednesday.

maxmcd 7y ago
Very cool. I haven't read the whole paper yet, but from a quick overview it seems somewhat similar to SLOG (which also deals with world-scale replication by trying to keep data closer to the nodes that use it): - http://www.vldb.org/pvldb/vol12/p1747-ren.pdf http://www.vldb.org/pvldb/vol12/p1747-ren.pdf - https://blog.acolyer.org/2019/09/04/slog/ https://blog.acolyer.org/2019/09/04/slog/ Any thoughts on this comparison?
- mjb 7y ago
  Very interesting, I hadn't seen SLOG before. It seems like there's a fairly similar core insight: placing data near where it's used can help with some system properties. They appear to be more latency-focused and we were (primarily) aiming at availability. The other different part of Physalia is our focus (again, for availability) on placement for 'blast radius'. That means we try limit the number of cells than any one failure (software, infrastructure, etc) can touch. Geo-replicated systems can have similar concerns, but I haven't seen the same level of focus on it as a key design goal.
revertts 7y ago
RE: NSDI - Is the Firecracker paper available somewhere, or not yet? Edit: I'm an idiot - https://www.amazon.science/publications/firecracker-lightweight-virtualization-for-serverless-applications https://www.amazon.science/publications/firecracker-lightwei...
pm90 7y ago
Thanks so much for doing this! I really like to read about interesting engineering problems that are encountered at scale and the way engineers think about them and come up with solutions to solve them. Probably tangential, but wanted to know what your thoughts are about the frequent case we often see where the interesting designs are made public without the actual implementation. e.g. the Map-Reduce paper-> Apache Hadoop. Dynamo-> Cassandra (and a few others), Spanner -> CockroachDB, Borg-> K8s etc. Having worked at corporations most of my life, I know that internal systems often have a lot of internal dependencies which makes it really hard to open source them easily; often a refactor is more expensive than writing from scratch. I can see it both ways; just wondering as the author, what your perspectives might be :).
stebann 7y ago
Congrats!

by mjb 7y ago

maxmcd 7y ago

Very cool. I haven't read the whole paper yet, but from a quick overview it seems somewhat similar to SLOG (which also deals with world-scale replication by trying to keep data closer to the nodes that use it): - http://www.vldb.org/pvldb/vol12/p1747-ren.pdf http://www.vldb.org/pvldb/vol12/p1747-ren.pdf - https://blog.acolyer.org/2019/09/04/slog/ https://blog.acolyer.org/2019/09/04/slog/ Any thoughts on this comparison?

mjb 7y ago
Very interesting, I hadn't seen SLOG before. It seems like there's a fairly similar core insight: placing data near where it's used can help with some system properties. They appear to be more latency-focused and we were (primarily) aiming at availability. The other different part of Physalia is our focus (again, for availability) on placement for 'blast radius'. That means we try limit the number of cells than any one failure (software, infrastructure, etc) can touch. Geo-replicated systems can have similar concerns, but I haven't seen the same level of focus on it as a key design goal.

revertts 7y ago

RE: NSDI - Is the Firecracker paper available somewhere, or not yet? Edit: I'm an idiot - https://www.amazon.science/publications/firecracker-lightweight-virtualization-for-serverless-applications https://www.amazon.science/publications/firecracker-lightwei...

pm90 7y ago

Thanks so much for doing this! I really like to read about interesting engineering problems that are encountered at scale and the way engineers think about them and come up with solutions to solve them. Probably tangential, but wanted to know what your thoughts are about the frequent case we often see where the interesting designs are made public without the actual implementation. e.g. the Map-Reduce paper-> Apache Hadoop. Dynamo-> Cassandra (and a few others), Spanner -> CockroachDB, Borg-> K8s etc. Having worked at corporations most of my life, I know that internal systems often have a lot of internal dependencies which makes it really hard to open source them easily; often a refactor is more expensive than writing from scratch. I can see it both ways; just wondering as the author, what your perspectives might be :).

stebann 7y ago

Congrats!