4 ms·

I would love to see some advancements into distributed, statically typed languages that can be run on across cluster, and that would support type-safe, rolling

by bjz_ 9y ago

I would love to see some advancements into distributed, statically typed languages that can be run on across cluster, and that would support type-safe, rolling deployments. One would have to ensure that state could be migrated safely, and that messaging can still happen between the nodes of different versions. Similar to thinking about this 'temporal' dimension of code, it would be cool to see us push versioning and library upgrades further, perhaps supporting automatic migrations.

doublec 9y ago
Have you seen Alice ML? It's a Standard ML variant that includes serialization of types and support for distributed programming. [1] http://www.ps.uni-saarland.de/alice/manual/tour.html#distribution http://www.ps.uni-saarland.de/alice/manual/tour.html#distrib...
- bjz_ 9y ago
  How does it handle heterogeneous clusters that might have different versions of code running? Or is it like Cloud Haskell where you have to bring everything down when deploying?
imtringued 9y ago
I would start a bit simpler. Are there any statically typed serialisation formats like flatbuffers or cap'n proto where you can statically verify that version 1.0 and version 1.2 are compatible but version 1.2 and 2.0 are not?
- kentonv 9y ago
  Not sure if I completely understand the question, but... Protobuf, Cap'n Proto, Flatbuffers, and anything else with a similar design gives you the ability to verify that two versions of the protocol are "compatible" insofar as: - If a new version of the program reads a message from an older version which is missing some newly-added field, that field will assigned a default value or "null", but the rest of the message will parse fine. - If an old version of the program reads a message from a newer version which has some new fields that the old version doesn't know about, it will ignore those fields, but can parse the rest of the message just fine. Of course, just because you've verified that the fields line up doesn't prove that the old and new versions of the program are actually compatible. The interpretation of the protocol could change in an incompatible way without any change to the protocol itself. Verifying compatibility at that level is an AI-complete problem. But in practice, it works very well. Enabling safe incremental, rolling updates of components in a distributed system is exactly what Protobuf was designed to do and has been doing for >15 years at Google.

by bjz_ 9y ago

doublec 9y ago

Have you seen Alice ML? It's a Standard ML variant that includes serialization of types and support for distributed programming. [1] http://www.ps.uni-saarland.de/alice/manual/tour.html#distribution http://www.ps.uni-saarland.de/alice/manual/tour.html#distrib...

bjz_ 9y ago
How does it handle heterogeneous clusters that might have different versions of code running? Or is it like Cloud Haskell where you have to bring everything down when deploying?

imtringued 9y ago

I would start a bit simpler. Are there any statically typed serialisation formats like flatbuffers or cap'n proto where you can statically verify that version 1.0 and version 1.2 are compatible but version 1.2 and 2.0 are not?

kentonv 9y ago
Not sure if I completely understand the question, but... Protobuf, Cap'n Proto, Flatbuffers, and anything else with a similar design gives you the ability to verify that two versions of the protocol are "compatible" insofar as: - If a new version of the program reads a message from an older version which is missing some newly-added field, that field will assigned a default value or "null", but the rest of the message will parse fine. - If an old version of the program reads a message from a newer version which has some new fields that the old version doesn't know about, it will ignore those fields, but can parse the rest of the message just fine. Of course, just because you've verified that the fields line up doesn't prove that the old and new versions of the program are actually compatible. The interpretation of the protocol could change in an incompatible way without any change to the protocol itself. Verifying compatibility at that level is an AI-complete problem. But in practice, it works very well. Enabling safe incremental, rolling updates of components in a distributed system is exactly what Protobuf was designed to do and has been doing for >15 years at Google.