3 ms·

These are great examples of difficulties people will encounter in all popular statically typed languages sooner or later. I find the solution presented interes

by beders 2y ago

These are great examples of difficulties people will encounter in all popular statically typed languages sooner or later. I find the solution presented interesting but it is limited to the 3 operations mentioned. The alternative to this runtime schema checks and treating your data as data - the data you are working with is coming from an external system and needs to be runtime-validated anyways. Throw in some nil-punting and you can have a very flexible system that is guarded by runtime checks.

mrkeen 2y ago
Agreed that it is naive to assume that static types will keep some kind of consistency on the bytes that come in on the wire, but it misses the point. With static types, you do one transformation from bytes to a proper data type at the boundary. The static type information propogates bidirectionally deep into the guts of the system. If you're 10 layers in, and want to invoke getUserId() on an object, you just add it to your type. Now your endpoint can return 400s without you needing to add extra nil-checks somewhere (or multiple places) from layers 1 to 10 (which realistically will turn into 500s).
- Vampiero 2y ago
  Realistically you can design a runtime-checked interface that uses option types instead of nulls, thereby guaranteeing static correctness and preventing common pitfalls at the cost of making the usage pattern a bit more verbose.
- delusional 2y ago
  > but it misses the point. I don't think it misses the point as much as it's an entirely different point. When I talk to my coworkers about the uses of these automated marshaling systems at the boundary, they're not talking about correctness or "evolution". They talk about how annoying they find it to type. In the fingers hitting keyboard sense of that word. They explicitly don't want a change 10 layers deep to result in any error in the parsing layer. They want the magical system to just do something at runtime.
manx 2y ago
I recommend reading: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
- specialist 2y ago
  IDK... I've done a lot of data processing. What the kids today call ETL. Plus some parsing (writing grammars). Here's my working taxonomy: Parsing for source code and protocols. Serialization when you control both ends. (Implicitly defined parsing of protocols.) Scrapping is data processing, more or less. Find and read a value, maybe transmogrify it, write new value. -- As for the OC, once the contract made a breaking change, the problem transmuted from serialization to scrapping. Maybe some future runtime system will successfully enable multiple live updatable schema revisions. (I've been pondering that gordian knot for decades. Anyone else remember "objected oriented databases" hype?) Until then, we're stuck in data processing hell.

by beders 2y ago

mrkeen 2y ago

Agreed that it is naive to assume that static types will keep some kind of consistency on the bytes that come in on the wire, but it misses the point. With static types, you do one transformation from bytes to a proper data type at the boundary. The static type information propogates bidirectionally deep into the guts of the system. If you're 10 layers in, and want to invoke getUserId() on an object, you just add it to your type. Now your endpoint can return 400s without you needing to add extra nil-checks somewhere (or multiple places) from layers 1 to 10 (which realistically will turn into 500s).

Vampiero 2y ago
Realistically you can design a runtime-checked interface that uses option types instead of nulls, thereby guaranteeing static correctness and preventing common pitfalls at the cost of making the usage pattern a bit more verbose.
delusional 2y ago
> but it misses the point. I don't think it misses the point as much as it's an entirely different point. When I talk to my coworkers about the uses of these automated marshaling systems at the boundary, they're not talking about correctness or "evolution". They talk about how annoying they find it to type. In the fingers hitting keyboard sense of that word. They explicitly don't want a change 10 layers deep to result in any error in the parsing layer. They want the magical system to just do something at runtime.

manx 2y ago

I recommend reading: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

specialist 2y ago
IDK... I've done a lot of data processing. What the kids today call ETL. Plus some parsing (writing grammars). Here's my working taxonomy: Parsing for source code and protocols. Serialization when you control both ends. (Implicitly defined parsing of protocols.) Scrapping is data processing, more or less. Find and read a value, maybe transmogrify it, write new value. -- As for the OC, once the contract made a breaking change, the problem transmuted from serialization to scrapping. Maybe some future runtime system will successfully enable multiple live updatable schema revisions. (I've been pondering that gordian knot for decades. Anyone else remember "objected oriented databases" hype?) Until then, we're stuck in data processing hell.