5 ms·

Static typing is fantastic in a closed system. It's predictable and reliable. My problem is nearly every type system breaks down when you try communicate with

Static typing is fantastic in a closed system. It's predictable and reliable. My problem is nearly every type system breaks down when you try communicate with something outside of the closed system (e.g. API calls, 3rd party integration, etc). An API returns an unexpected data type, Missing a field that's supposed to be required, Starts returning a number as a string because they changed their ID system, etc. It always seems to end up in types where every field is optional - defeating the purpose of having types in the first place.

carlmr 8y ago
If the API you rely on breaks, you have to change your code. This is exactly how catastrophes happen. It's more of an issue that you have unreliable 3rd party vendors that you buy from/use.
- SkyPuncher 8y ago
  > It's more of an issue that you have unreliable 3rd party vendors that you buy from/use. It happens very, very rarely - but the problem is when it happens it can happen suddenly and be very, very hard to address properly.
mjburgess 8y ago
Option types do not defy the purpose of typing. Optionality as a type constraint is a reason for static typing.
idrae 8y ago
I agree with you that static typing can create more work on the boundries of a system. However, the problems you describe also happen in dynamic languages and they have just as much potential to cause problems. The biggest issue with dynamic languages is that you don't get immediate errors when something changes. The system will just keep doing its thing, until it doesn't and you discover that something has changed and has already been propagated to other parts of the system. I don't think dynamic languages are always better in these cases. They basically let you defer the error/change handling, but with the potential of much bigger data integrity/quality problems when things eventualy do go wrong because of changes in an integration.
klmr 8y ago
The API problems you cite aren’t prevented in dynamically typed systems. If anything they are worse, because they could lead to silent failures, whereas a statically typed API statically checks and guarantees the contract. Contrary to what you imply, a statically typed API can’t return an unexpected type. That’s an advantage. > It always seems to end up in types where every field is optional “Always”? Not by a long shot. These situations where this is the case can be isolated and dealt with explicitly. You may object to the explicitness but — again, for reasons of robustness — proponents of static typing prefer this.
- rhacker 8y ago
  There's nothing stopping a dynamically typed system from using something like grpc or graphql that can check for missing fields. Or something like json schema to check for missing fields in a lot of those scenarios. A type system doesn't necessarily catch missing fields, but a communication framework of some kind, does.
  - klmr 8y ago
    I entirely agree, see my orthogonal comment reply.
- pmontra 8y ago
  > whereas a statically typed API statically checks and guarantees the contract I didn't work much with static typed languages lately, so I'm really puzzled by your comment. If I expect an int in the age field of a JSON response and the server suddenly sends me a string, maybe with value "NaN", how do you check for that at compile time? I expect the program to either crash or manage the error at run time (try/catch, die-and-restart a-la-Erlang, etc) Maybe JSON doesn't count as "statically typed API", but how many of them do common Internet services expose to the public? And how to we make sure that the server or any in between proxy doesn't misbehave and violates the contract?
  - steveklabnik 8y ago
    The way this works is, when you say “please turn this JSON string into a data structure”, it’s a fallible operation. So, if that JSON string does not have what you expect, the serialization operation will fail. You can “check at compile time” by the system ensuring that you handle the success and failure cases. Additionally, it means that if you have an instance of that data structure, it is always valid; it is checked (at runtime) upon creation. Does that make sense? I can show you some code if you’d like.
lewisf 8y ago
This is likely going to lead to an error in both statically and dynamically typed languages. In statically typed language, at least when troubleshooting it's an easy guess that the error is highly likely to be at the external boundary of your application code since the rest of it type checked.
phaedrus 8y ago
The problem here is not static typing or its interface to the outside world. The problem is most mainstream static typed languages have shit handling for Sum Types. With proper support and proper syntax static typing would actually help you ensure you handle both possibilities of your "option" type. (Any language which defaults to any reference type may be null, at any time, is the opposite of properly handling option types.) Let me turn this around: if you're saying possibly-missing fields inevitably propagate throughout the types in your codebase, it sounds like you're saying malformed input propagates arbitrarily deeply into your core logic. Wouldn't you rather catch that at the boundary?
- SkyPuncher 8y ago
  > if you're saying possibly-missing fields inevitably propagate throughout the types in your codebase, it sounds like you're saying malformed input propagates arbitrarily deeply into your core logic. Wouldn't you rather catch that at the boundary? Possibly. If it's a key part of the application, like Credit Card processing or auth - absolutely. If it's simply auxiliary or supplemental information - probably not. The thing for me is the type system shouldn't decide whether or not a certain piece of information is critical to the system. That typically falls to deeper logic in the system. If the type system isn't deciding, then it's pretty much meaningless (as everything would be optional). For example, I'm using a Stripe JS integration to allow customers to manage credit cards in app. Stripe messes up or changes something without me paying attention. Instead of returning a complete object, Stripe starts returning only the credit card ID. This isn't a big deal if we simply need to know that a record exists. We can still tell the user has a card and we have an ID to attempt a transaction against server side. We might not be able to display details about the card, but users could still submit a transaction (likely against their last used card). However, it is a big deal in a credit card management interface. Without supplementary data, all of the cards would look exactly the same and the UI might break if expected fields aren't present. ---- I don't want my type system deciding what the use case is or how important one case is compared to the other. The actual implementation needs to decide if it has the information it needs to do its job.
  - pacala 8y ago
pacala 8y ago
Static type systems are just a tool. The type signature describes what _your_ module is capable of handling. If other modules pass incorrect data, your module will fail, whether statically or dynamically typed, albeit harder to debug in the dynamically typed case. If every field ends up being truly optional, it's going to be hard to change your module, because of the sheer weight of the combinatorial explosion of field subsets that may or maynot be set when third party modules interact with your module. Most of the time optional fields: * Encode that your module doesn't care about such-and-such fields, so the parser / type checker should just ignore them. * Are used to encode variants of the same data that slightly changed in another module [id changed from number to string], so you should use an union type. In practice, designing a statically typed ecosystem for wire data is hard [0]. Grpc has basically gave up on validating field presence altogether [1]. These failures are directly caused by the immaturity of the ecosystem [deep decode batches of unrelated messages in middleware, WTF?]. It would be a mistake to throw the baby with the bathwater and blame the mathematically sound idea that your module can only process data of a certain shape, encoded by a suitable type signature. [0] https://capnproto.org/faq.html#how-do-i-make-a-field-required-like-in-protocol-buffers https://capnproto.org/faq.html#how-do-i-make-a-field-require... [1] https://github.com/protocolbuffers/protobuf/issues/2497#issuecomment-267422550 https://github.com/protocolbuffers/protobuf/issues/2497#issu...

carlmr 8y ago

If the API you rely on breaks, you have to change your code. This is exactly how catastrophes happen. It's more of an issue that you have unreliable 3rd party vendors that you buy from/use.

SkyPuncher 8y ago
> It's more of an issue that you have unreliable 3rd party vendors that you buy from/use. It happens very, very rarely - but the problem is when it happens it can happen suddenly and be very, very hard to address properly.

mjburgess 8y ago

Option types do not defy the purpose of typing. Optionality as a type constraint is a reason for static typing.

idrae 8y ago

I agree with you that static typing can create more work on the boundries of a system. However, the problems you describe also happen in dynamic languages and they have just as much potential to cause problems. The biggest issue with dynamic languages is that you don't get immediate errors when something changes. The system will just keep doing its thing, until it doesn't and you discover that something has changed and has already been propagated to other parts of the system. I don't think dynamic languages are always better in these cases. They basically let you defer the error/change handling, but with the potential of much bigger data integrity/quality problems when things eventualy do go wrong because of changes in an integration.

klmr 8y ago

The API problems you cite aren’t prevented in dynamically typed systems. If anything they are worse, because they could lead to silent failures, whereas a statically typed API statically checks and guarantees the contract. Contrary to what you imply, a statically typed API can’t return an unexpected type. That’s an advantage. > It always seems to end up in types where every field is optional “Always”? Not by a long shot. These situations where this is the case can be isolated and dealt with explicitly. You may object to the explicitness but — again, for reasons of robustness — proponents of static typing prefer this.

rhacker 8y ago
There's nothing stopping a dynamically typed system from using something like grpc or graphql that can check for missing fields. Or something like json schema to check for missing fields in a lot of those scenarios. A type system doesn't necessarily catch missing fields, but a communication framework of some kind, does.
- klmr 8y ago
  I entirely agree, see my orthogonal comment reply.
pmontra 8y ago
> whereas a statically typed API statically checks and guarantees the contract I didn't work much with static typed languages lately, so I'm really puzzled by your comment. If I expect an int in the age field of a JSON response and the server suddenly sends me a string, maybe with value "NaN", how do you check for that at compile time? I expect the program to either crash or manage the error at run time (try/catch, die-and-restart a-la-Erlang, etc) Maybe JSON doesn't count as "statically typed API", but how many of them do common Internet services expose to the public? And how to we make sure that the server or any in between proxy doesn't misbehave and violates the contract?
- steveklabnik 8y ago
  The way this works is, when you say “please turn this JSON string into a data structure”, it’s a fallible operation. So, if that JSON string does not have what you expect, the serialization operation will fail. You can “check at compile time” by the system ensuring that you handle the success and failure cases. Additionally, it means that if you have an instance of that data structure, it is always valid; it is checked (at runtime) upon creation. Does that make sense? I can show you some code if you’d like.

lewisf 8y ago

This is likely going to lead to an error in both statically and dynamically typed languages. In statically typed language, at least when troubleshooting it's an easy guess that the error is highly likely to be at the external boundary of your application code since the rest of it type checked.

phaedrus 8y ago

The problem here is not static typing or its interface to the outside world. The problem is most mainstream static typed languages have shit handling for Sum Types. With proper support and proper syntax static typing would actually help you ensure you handle both possibilities of your "option" type. (Any language which defaults to any reference type may be null, at any time, is the opposite of properly handling option types.) Let me turn this around: if you're saying possibly-missing fields inevitably propagate throughout the types in your codebase, it sounds like you're saying malformed input propagates arbitrarily deeply into your core logic. Wouldn't you rather catch that at the boundary?

SkyPuncher 8y ago
> if you're saying possibly-missing fields inevitably propagate throughout the types in your codebase, it sounds like you're saying malformed input propagates arbitrarily deeply into your core logic. Wouldn't you rather catch that at the boundary? Possibly. If it's a key part of the application, like Credit Card processing or auth - absolutely. If it's simply auxiliary or supplemental information - probably not. The thing for me is the type system shouldn't decide whether or not a certain piece of information is critical to the system. That typically falls to deeper logic in the system. If the type system isn't deciding, then it's pretty much meaningless (as everything would be optional). For example, I'm using a Stripe JS integration to allow customers to manage credit cards in app. Stripe messes up or changes something without me paying attention. Instead of returning a complete object, Stripe starts returning only the credit card ID. This isn't a big deal if we simply need to know that a record exists. We can still tell the user has a card and we have an ID to attempt a transaction against server side. We might not be able to display details about the card, but users could still submit a transaction (likely against their last used card). However, it is a big deal in a credit card management interface. Without supplementary data, all of the cards would look exactly the same and the UI might break if expected fields aren't present. ---- I don't want my type system deciding what the use case is or how important one case is compared to the other. The actual implementation needs to decide if it has the information it needs to do its job.
- pacala 8y ago

pacala 8y ago

Static type systems are just a tool. The type signature describes what _your_ module is capable of handling. If other modules pass incorrect data, your module will fail, whether statically or dynamically typed, albeit harder to debug in the dynamically typed case. If every field ends up being truly optional, it's going to be hard to change your module, because of the sheer weight of the combinatorial explosion of field subsets that may or maynot be set when third party modules interact with your module. Most of the time optional fields: * Encode that your module doesn't care about such-and-such fields, so the parser / type checker should just ignore them. * Are used to encode variants of the same data that slightly changed in another module [id changed from number to string], so you should use an union type. In practice, designing a statically typed ecosystem for wire data is hard [0]. Grpc has basically gave up on validating field presence altogether [1]. These failures are directly caused by the immaturity of the ecosystem [deep decode batches of unrelated messages in middleware, WTF?]. It would be a mistake to throw the baby with the bathwater and blame the mathematically sound idea that your module can only process data of a certain shape, encoded by a suitable type signature. [0] https://capnproto.org/faq.html#how-do-i-make-a-field-required-like-in-protocol-buffers https://capnproto.org/faq.html#how-do-i-make-a-field-require... [1] https://github.com/protocolbuffers/protobuf/issues/2497#issuecomment-267422550 https://github.com/protocolbuffers/protobuf/issues/2497#issu...