4 ms·

> In a dynamic language like JavaScript, it can be hard to know what the shape of your data is. You don't need static typing to program defensively. Static ty

> In a dynamic language like JavaScript, it can be hard to know what the shape of your data is. You don't need static typing to program defensively. Static typing instead ensures a certain defensive position, but as a developer you can choose to program in such a way even in a dynamic language. That being said it is foolish to make a blanket assumption that data is doomed to confusion. The state and shape of data is determined by the authoring application and reinforced by the receiving application. If data is shaped improperly then a defect is present in the system, so fix the defect. If a dose of common sense is still not enough... then program in TypeScript.

arcticbull 8y ago
It's nothing to do with "a dose of common sense" so much as it is the reality of dealing with humans. Humans make mistakes. Static typing is automated checks to make sure you didn't. When doctors start using checklists their patients do better [1]. Static typing is checklists for programmers. In theory weak dynamic typing works great. In practice humans suck. Hence checklists. [1] https://www.newyorker.com/magazine/2007/12/10/the-checklist https://www.newyorker.com/magazine/2007/12/10/the-checklist
- austincheney 8y ago
  This completely ignores my comment in its entirety. Do people make mistakes: yes. Does software ever have defects: yes. Typically, people making mistakes in software is called a software defect. It is absolutely critical in a discussion like this to understand that defects in software are not necessarily defects in data. This conversation is about data, and checks for processing data are already present in various forms. For example when was the last time you complained about weak type systems when working with JSON, XML, or SQL tables?
  - Quekid5 8y ago
    > For example when was the last time you complained about weak type systems when working with JSON, XML, or SQL tables? Well, I don't know about the person you're replying to, but I certainly complain about these all the time. Interestingly and counter to your point, I think, XML grew a "type system" of sorts with XML Schema (which is widely used IME, esp. where interfacing with external services as in SOA), and SQL actually has a reasonably strong type system for most implementations albeit dynamically checked at query planning time. (SQLite seems to be the outlier here.) JSON is actually the outlier here, but even with JSON there's been attempts at a sort-of type system with JSON Schema. Why? Well, because beyond a certain size the "just chuck some data somewhere in there" ceases to be workable when you want stable and maintainable interfaces between software components. Ultimately type systems are invented and used for almost entirely pragmatic reasons.
    - spiralx 8y ago
      > ... XML grew a "type system" of sorts with XML Schema (which is widely used IME, esp. where interfacing with external services as in SOA) ... The type system came from both XML Schema and XPath and was codified with XSLT 2, then expanded when XQuery was defined. It's not been a type system "of sorts" since XSLT 2, which filled in a lot of gaps and undefined behaviours.
  - arcticbull 8y ago
    SQL has a schema, and the way I use it, I generate interfaces with strict types from my schema, specifically because I've run into trouble not doing so. XML, well, there's the schema language for that, but that's fairly "outdated" and I haven't touched it in ages, but if you've ever looked at an HTML parser, it's borderline sentient because of weak typing. JSON? That's a huge source of issues. I've run into them professionally and personally, and they cause outages all the time. Thats why protobufs are great (schema) and GraphQL with generated interface is great.
  - arcticbull 8y ago
    I didn't really ignore your comment either though. >> You don't need static typing to program defensively. Static typing instead ensures a certain defensive position, but as a developer you can choose to program in such a way even in a dynamic language. Correct, but in a statically typed language defensive programming is not required on your part to the same extent (because the compiler enforces a whole set of defenses on your behalf). You'll also forget sometimes, mistype sometimes, check the wrong things sometimes, forget to update some defenses. The compiler won't. >> That being said it is foolish to make a blanket assumption that data is doomed to confusion. The state and shape of data is determined by the authoring application and reinforced by the receiving application. If data is shaped improperly then a defect is present in the system, so fix the defect. The type system enforces you checked the data (by for instance, providing a conversion function `fn convert(input: String) -> Result<StructuredData, Error>` which then forces you to handle your error there, and then you don't need to check anything else later. >> If a dose of common sense is still not enough... then program in TypeScript. That's half a solution because unless your entire ecosystem is typescript those types are at best aspirational. An improvement to be sure, but still aspirational. Which, IMO, can all be summed up by "we have automated checks so humans don't have to write them and make mistakes doing so as they go". Outsource the checks to an automated system.
  - crimsonalucard 8y ago
    Data in itself can be typed. For JSON objects the equivalent type is usually called a Record. Within a single strictly typed program, data is transferred between methods and functions and the shape of this data would be described by a type. There is not much of a difference, theoretically speaking, when data is passed between functions inside an app, or between apps via some network api. What you describe is actually a real issue in typing. Types are lost across systems. JSON is not strictly typed, neither is XML or SQL. So if I have a typed application that spits out data to these formats I lose type checking. If you think about it, even jumping across abstractions from one typed language (say rust on the backend) to another typed language (say typescript on the frontend) you will lose type safety due to incompatibilities between languages. I don't know if there's research into or thought going into solving this issue. But a universal type language/syntax that can be translated into respective type systems across api abstractions could lend a lot to safety of entire ecosystems. There is no type safety across http services and designing a system to overcome this issue is very possible and to my knowledge not something people have thought about yet. New idea? Imagine a language/framework that compiles into web apps and microservices on some cloud provider. The language has syntax that allows me to decorate functions to specify which microservice it runs on. Data transfer between microservices can now be placed in some binary format or maybe the language has syntax that allows me to specify human readability. Either way the language abstracts away the devops and webapp layer and combines it into a single source code... this would allow type checking of the shape of data to not happen just within an app, but across services. Is there such a framework/language that does this? I'm positive it can be done. I would totally love to work on an idea like this. The microservice fad talked up modularity between services but in practice people realized that microservices are too complicated. Perhaps an entire language framework that combines the web app layer and dev ops layer and adds type checking across services would solve this issue.
- Kiro 8y ago
  How would static typing help you if it's two different applications like the post you reply to is talking about?
  - pka 8y ago
    You try to parse the received data into a predefined data type, if this fails you are forced by the type system to handle the failure scenario, if it parses correctly you now have a 100% valid data structure you can work with in the rest of your codebase, no defensive checks necessary.
    - Kiro 8y ago
      How does that work? I thought static typing was only checked when compiling.
      - dbattaglia 8y ago
        Imagine some language with the following type and functions: type Person { name: String } function parsePerson(json: String): Person { ... } function usePerson(person: Person) { ... } The compiler will guarantee that usePerson will always get the correct shape of data for it's person variable. The only function you need to worry about is parsePerson, which will generally throw a runtime error if the incoming JSON does not conform to the expected shape. And in most web frameworks, the JSON to type parsing is done for you so once the framework hands you data in the type you are asking for you can expect it to be correct.
      - dfgdghdf 8y ago
        You write code at the entry point that might fail. Perhaps it returns either structured data or an error. From that point on, you can work only with structured data and the type-system will help you. You might be able to generate a parser from a schema.
      - int_19h 8y ago
        In most modern languages with static types, types are reified - i.e. actual values have some runtime type associated with them, and it's impossible (or, at least, exceedingly difficult) to create a value that has the wrong representation for its type. In practice, with parsing, this usually means that runtime type metadata is used. Effectively, types become your schema, and you can validate against them as needed.
crimsonalucard 8y ago
If you think common sense can ensure safety then you are super human. I have a 10 million line program that outputs an extremely complicated dataset with hundreds of parameters. Occasionally one parameter disappears... at a rate of 1/1000 runs. This is a bug, but the bug bypassed everything. The system has 100 e2e tests of "defense" but because the bug happens 1 out of 1000 times, 100 integration tests did not have enough coverage to find the error... so the program made it to production... Now with static type checking. That program wouldn't even compile. The compiler can prove that the program is wrong rather than you going through hundreds of unit tests to try to catch an error while proving nothing. Proving a program is wrong is better than writing billions of unit tests that cover the entire domain of a function. Your strategy of using common sense to go through 10 million (exaggeration of course) lines of code is an effective strategy, but not an intelligent one.

arcticbull 8y ago

It's nothing to do with "a dose of common sense" so much as it is the reality of dealing with humans. Humans make mistakes. Static typing is automated checks to make sure you didn't. When doctors start using checklists their patients do better [1]. Static typing is checklists for programmers. In theory weak dynamic typing works great. In practice humans suck. Hence checklists. [1] https://www.newyorker.com/magazine/2007/12/10/the-checklist https://www.newyorker.com/magazine/2007/12/10/the-checklist

austincheney 8y ago
This completely ignores my comment in its entirety. Do people make mistakes: yes. Does software ever have defects: yes. Typically, people making mistakes in software is called a software defect. It is absolutely critical in a discussion like this to understand that defects in software are not necessarily defects in data. This conversation is about data, and checks for processing data are already present in various forms. For example when was the last time you complained about weak type systems when working with JSON, XML, or SQL tables?
- Quekid5 8y ago
  > For example when was the last time you complained about weak type systems when working with JSON, XML, or SQL tables? Well, I don't know about the person you're replying to, but I certainly complain about these all the time. Interestingly and counter to your point, I think, XML grew a "type system" of sorts with XML Schema (which is widely used IME, esp. where interfacing with external services as in SOA), and SQL actually has a reasonably strong type system for most implementations albeit dynamically checked at query planning time. (SQLite seems to be the outlier here.) JSON is actually the outlier here, but even with JSON there's been attempts at a sort-of type system with JSON Schema. Why? Well, because beyond a certain size the "just chuck some data somewhere in there" ceases to be workable when you want stable and maintainable interfaces between software components. Ultimately type systems are invented and used for almost entirely pragmatic reasons.
  - spiralx 8y ago
    > ... XML grew a "type system" of sorts with XML Schema (which is widely used IME, esp. where interfacing with external services as in SOA) ... The type system came from both XML Schema and XPath and was codified with XSLT 2, then expanded when XQuery was defined. It's not been a type system "of sorts" since XSLT 2, which filled in a lot of gaps and undefined behaviours.
- arcticbull 8y ago
  SQL has a schema, and the way I use it, I generate interfaces with strict types from my schema, specifically because I've run into trouble not doing so. XML, well, there's the schema language for that, but that's fairly "outdated" and I haven't touched it in ages, but if you've ever looked at an HTML parser, it's borderline sentient because of weak typing. JSON? That's a huge source of issues. I've run into them professionally and personally, and they cause outages all the time. Thats why protobufs are great (schema) and GraphQL with generated interface is great.
- arcticbull 8y ago
  I didn't really ignore your comment either though. >> You don't need static typing to program defensively. Static typing instead ensures a certain defensive position, but as a developer you can choose to program in such a way even in a dynamic language. Correct, but in a statically typed language defensive programming is not required on your part to the same extent (because the compiler enforces a whole set of defenses on your behalf). You'll also forget sometimes, mistype sometimes, check the wrong things sometimes, forget to update some defenses. The compiler won't. >> That being said it is foolish to make a blanket assumption that data is doomed to confusion. The state and shape of data is determined by the authoring application and reinforced by the receiving application. If data is shaped improperly then a defect is present in the system, so fix the defect. The type system enforces you checked the data (by for instance, providing a conversion function `fn convert(input: String) -> Result<StructuredData, Error>` which then forces you to handle your error there, and then you don't need to check anything else later. >> If a dose of common sense is still not enough... then program in TypeScript. That's half a solution because unless your entire ecosystem is typescript those types are at best aspirational. An improvement to be sure, but still aspirational. Which, IMO, can all be summed up by "we have automated checks so humans don't have to write them and make mistakes doing so as they go". Outsource the checks to an automated system.
- crimsonalucard 8y ago
  Data in itself can be typed. For JSON objects the equivalent type is usually called a Record. Within a single strictly typed program, data is transferred between methods and functions and the shape of this data would be described by a type. There is not much of a difference, theoretically speaking, when data is passed between functions inside an app, or between apps via some network api. What you describe is actually a real issue in typing. Types are lost across systems. JSON is not strictly typed, neither is XML or SQL. So if I have a typed application that spits out data to these formats I lose type checking. If you think about it, even jumping across abstractions from one typed language (say rust on the backend) to another typed language (say typescript on the frontend) you will lose type safety due to incompatibilities between languages. I don't know if there's research into or thought going into solving this issue. But a universal type language/syntax that can be translated into respective type systems across api abstractions could lend a lot to safety of entire ecosystems. There is no type safety across http services and designing a system to overcome this issue is very possible and to my knowledge not something people have thought about yet. New idea? Imagine a language/framework that compiles into web apps and microservices on some cloud provider. The language has syntax that allows me to decorate functions to specify which microservice it runs on. Data transfer between microservices can now be placed in some binary format or maybe the language has syntax that allows me to specify human readability. Either way the language abstracts away the devops and webapp layer and combines it into a single source code... this would allow type checking of the shape of data to not happen just within an app, but across services. Is there such a framework/language that does this? I'm positive it can be done. I would totally love to work on an idea like this. The microservice fad talked up modularity between services but in practice people realized that microservices are too complicated. Perhaps an entire language framework that combines the web app layer and dev ops layer and adds type checking across services would solve this issue.
Kiro 8y ago
How would static typing help you if it's two different applications like the post you reply to is talking about?
- pka 8y ago
  You try to parse the received data into a predefined data type, if this fails you are forced by the type system to handle the failure scenario, if it parses correctly you now have a 100% valid data structure you can work with in the rest of your codebase, no defensive checks necessary.
  - Kiro 8y ago
    How does that work? I thought static typing was only checked when compiling.
    - dbattaglia 8y ago
      Imagine some language with the following type and functions: type Person { name: String } function parsePerson(json: String): Person { ... } function usePerson(person: Person) { ... } The compiler will guarantee that usePerson will always get the correct shape of data for it's person variable. The only function you need to worry about is parsePerson, which will generally throw a runtime error if the incoming JSON does not conform to the expected shape. And in most web frameworks, the JSON to type parsing is done for you so once the framework hands you data in the type you are asking for you can expect it to be correct.
    - dfgdghdf 8y ago
      You write code at the entry point that might fail. Perhaps it returns either structured data or an error. From that point on, you can work only with structured data and the type-system will help you. You might be able to generate a parser from a schema.
    - int_19h 8y ago
      In most modern languages with static types, types are reified - i.e. actual values have some runtime type associated with them, and it's impossible (or, at least, exceedingly difficult) to create a value that has the wrong representation for its type. In practice, with parsing, this usually means that runtime type metadata is used. Effectively, types become your schema, and you can validate against them as needed.

crimsonalucard 8y ago

If you think common sense can ensure safety then you are super human. I have a 10 million line program that outputs an extremely complicated dataset with hundreds of parameters. Occasionally one parameter disappears... at a rate of 1/1000 runs. This is a bug, but the bug bypassed everything. The system has 100 e2e tests of "defense" but because the bug happens 1 out of 1000 times, 100 integration tests did not have enough coverage to find the error... so the program made it to production... Now with static type checking. That program wouldn't even compile. The compiler can prove that the program is wrong rather than you going through hundreds of unit tests to try to catch an error while proving nothing. Proving a program is wrong is better than writing billions of unit tests that cover the entire domain of a function. Your strategy of using common sense to go through 10 million (exaggeration of course) lines of code is an effective strategy, but not an intelligent one.