4 ms·

If you're not using a bunch of generics, check out typescript-is [1]. It takes little work to get it setup, but it generates run time type checks for you. I un

by joshAg 6y ago

If you're not using a bunch of generics, check out typescript-is [1]. It takes little work to get it setup, but it generates run time type checks for you. I understand why typescript decided to not add this functionality to the core of the language, but it's starting to feel like the largest missing piece of typescript is a built-in way to generate run-time type-checks for user-defined types from just the type definition. The happy medium we've found with that module is using the runtime type-check on anything "unsafe" to bless the result using typescript-is's equals functionality, but still allowing programmers to use casting with a comment justifying its necessity. For us our list of unsafe is results pulled from the db, anything parsed from JSON, and incoming request bodies (which can be a special case of parsing from JSON, but not always). [1]: https://github.com/woutervh-/typescript-is https://github.com/woutervh-/typescript-is

Shacklz 6y ago
Doing runtime-typechecks really is a blessing, and it would be great if typescript would provide this as a language-feature. There are two more libraries I know which are great for that: * Zod: https://github.com/colinhacks/zod https://github.com/colinhacks/zod * io-ts: https://github.com/gcanti/io-ts https://github.com/gcanti/io-ts (for the more functional-programming-oriented)
- joshAg 6y ago
  We decided against io-ts because we'd still have to write the validator ourself (unless we used their type system as the source of truth, which wouldn't easily work for our project because we generate most of our types from an openAPI schema), which is exactly what we wanted to avoid doing. I hadn't heard of Zod before, but i really liked the "parse don't validate" blog post linked. Turns out that's exactly how we've been using typescript-is. But it still suffers from the problem of expecting to be the source of truth and thus not working well with generated types.
- eropple 6y ago
  I like io-ts, but I've moved mostly to runtypes: https://github.com/pelotom/runtypes https://github.com/pelotom/runtypes I forget why Zod didn't pass muster for me now, but I've looked at it before and didn't find a compelling reason to jump to it. Thought it seemed good, though. I really wish at least one of these let you emit JSON schema back out, though. That'd be way easier for dealing with stuff like OpenAPI.
  - joshAg 6y ago
    We go the other way. We start with openAPI, use dtsgenerator to spit out typescript types, and then use a library (typescript-is) that will generate run-time type check function as part of compilation. A downside to this is that you need to compile type changes before you can use the change in code, but it gives us a single source of truth for our type definitions and let's us generate a more exacting request and response verification functions that use the (sometimes) more precise open-api definitions (eg a an array with a maximum number of elements and specific string formats), and it's really easy to add a run-time type check to the result of calling into a 3rd party library as well. For runtypes, you could write a script to use its Static function to spit out the interfaces in pure typescript and then use something like typeconv or ts-to-openapi to make your openapi definition, and then use that script inside your build script to output the openapi schema.
    - eropple 6y ago
      Interesting! How do you wire it into validation, etc. for your web layer? If you've got an example I'd love to see it. I've long been not a fan of OAS-first, but I'm always down to learn something new.
      - joshAg 6y ago
        We like OpenAPI first because it's more exact than pure typescript types. I suppose there's libraries we could use to give us a similar level of exactness that would also get us typescript types and OpenAPI with similar exactness to openAPI first, but then that's another dependency that's not giving us anythign extra compared to openAPI first. So the way we do it is a little meh because we have two separate run-time type validators, but it's because of the path we took where we adopted request and response validation almost from day 1, and other runtime validation came much later. We're using express and express-openapi as the base of our project. Express-openapi has built-in request validation and type-coercion, and an example of how to quickly bolt on your own response validation, so for request and response validation we're using that. And it works. It's damn solid. But the docs for how to use the underlying tech to do similar run-time validation for say pulling values out of the db is lacking and I don't have enough expertise in the underlying tech to stand it up quickly, especially compared just dropping in typescript-is changing tsc to ttsc and adding an tsc plugin to our tsconfig. If I were to do it again I'd either bite the bullet and spend the time figuring out how to get express-openapi's request validation infra to also do general type validation on just the schema, or i wouldn't use their run-time validation at all and just make my own middleware to do it, just so that I've only got a single run-time validation library. The nice thing about express-openapi is that since it "knows" the open-api definitions it can programmatically apply the types needed for validation, so rolling my own would require that functionality, because I don't want our team to have to remember to add the same middleware to every endpoint using a factory function that takes the typescript-is validator function as an argument, and the nice thing about typescript-is is that i can give it a typescript type and get a validator function exactly where i want it with no muss. Anyway, our build looks like this: 1) compile typescript 2) compile the openAPI to a single file (we put the endpoint definitions in the endpoint file) using express-openapi (just initialize and then write out the JSON.stringify of the apiDoc field) 3) run dtsgen against the output of step 2, and pipe that through `sed 's/declare//g' | sed 's/namespace/export namespace/g'` and into some file.ts 4) compile the file from step 3 with typescript step 1 gives us typescript compiled to js step 2 gives us a single openAPI.json file we can pass around or import into postman step 3 gives us all the types from openAPI in a ts file with a bunch of recursive exported namespaces step 4 gives us those types in a .d.ts file We do 3 and 4 in a subdirectory so that we can publish the types as an internal package, which means we can give front-end a types package to consume for all the calls we want to make to backend. Well, we actually publish an sdk package that wraps the types and encapsulates the logic for making the HTTP request to the right endpoint, so we don't have to worry about calling POST when the endpoint is PUT or fat-fingering the response type cast through the entire front-end and can instead just call a function with well-defined types. Plus, now it's super easy to test the interface, because we have a single thing we need to test. So we write an exhaustive happy-path (ie 2xx status codes) integration test against the sdk that actually calls into backend gets the response and checks the response against the type and then the values we expect for each field.

by joshAg 6y ago

Shacklz 6y ago

Doing runtime-typechecks really is a blessing, and it would be great if typescript would provide this as a language-feature. There are two more libraries I know which are great for that: * Zod: https://github.com/colinhacks/zod https://github.com/colinhacks/zod * io-ts: https://github.com/gcanti/io-ts https://github.com/gcanti/io-ts (for the more functional-programming-oriented)

joshAg 6y ago
We decided against io-ts because we'd still have to write the validator ourself (unless we used their type system as the source of truth, which wouldn't easily work for our project because we generate most of our types from an openAPI schema), which is exactly what we wanted to avoid doing. I hadn't heard of Zod before, but i really liked the "parse don't validate" blog post linked. Turns out that's exactly how we've been using typescript-is. But it still suffers from the problem of expecting to be the source of truth and thus not working well with generated types.
eropple 6y ago
I like io-ts, but I've moved mostly to runtypes: https://github.com/pelotom/runtypes https://github.com/pelotom/runtypes I forget why Zod didn't pass muster for me now, but I've looked at it before and didn't find a compelling reason to jump to it. Thought it seemed good, though. I really wish at least one of these let you emit JSON schema back out, though. That'd be way easier for dealing with stuff like OpenAPI.
- joshAg 6y ago
  We go the other way. We start with openAPI, use dtsgenerator to spit out typescript types, and then use a library (typescript-is) that will generate run-time type check function as part of compilation. A downside to this is that you need to compile type changes before you can use the change in code, but it gives us a single source of truth for our type definitions and let's us generate a more exacting request and response verification functions that use the (sometimes) more precise open-api definitions (eg a an array with a maximum number of elements and specific string formats), and it's really easy to add a run-time type check to the result of calling into a 3rd party library as well. For runtypes, you could write a script to use its Static function to spit out the interfaces in pure typescript and then use something like typeconv or ts-to-openapi to make your openapi definition, and then use that script inside your build script to output the openapi schema.
  - eropple 6y ago
    Interesting! How do you wire it into validation, etc. for your web layer? If you've got an example I'd love to see it. I've long been not a fan of OAS-first, but I'm always down to learn something new.
    - joshAg 6y ago
      We like OpenAPI first because it's more exact than pure typescript types. I suppose there's libraries we could use to give us a similar level of exactness that would also get us typescript types and OpenAPI with similar exactness to openAPI first, but then that's another dependency that's not giving us anythign extra compared to openAPI first. So the way we do it is a little meh because we have two separate run-time type validators, but it's because of the path we took where we adopted request and response validation almost from day 1, and other runtime validation came much later. We're using express and express-openapi as the base of our project. Express-openapi has built-in request validation and type-coercion, and an example of how to quickly bolt on your own response validation, so for request and response validation we're using that. And it works. It's damn solid. But the docs for how to use the underlying tech to do similar run-time validation for say pulling values out of the db is lacking and I don't have enough expertise in the underlying tech to stand it up quickly, especially compared just dropping in typescript-is changing tsc to ttsc and adding an tsc plugin to our tsconfig. If I were to do it again I'd either bite the bullet and spend the time figuring out how to get express-openapi's request validation infra to also do general type validation on just the schema, or i wouldn't use their run-time validation at all and just make my own middleware to do it, just so that I've only got a single run-time validation library. The nice thing about express-openapi is that since it "knows" the open-api definitions it can programmatically apply the types needed for validation, so rolling my own would require that functionality, because I don't want our team to have to remember to add the same middleware to every endpoint using a factory function that takes the typescript-is validator function as an argument, and the nice thing about typescript-is is that i can give it a typescript type and get a validator function exactly where i want it with no muss. Anyway, our build looks like this: 1) compile typescript 2) compile the openAPI to a single file (we put the endpoint definitions in the endpoint file) using express-openapi (just initialize and then write out the JSON.stringify of the apiDoc field) 3) run dtsgen against the output of step 2, and pipe that through `sed 's/declare//g' | sed 's/namespace/export namespace/g'` and into some file.ts 4) compile the file from step 3 with typescript step 1 gives us typescript compiled to js step 2 gives us a single openAPI.json file we can pass around or import into postman step 3 gives us all the types from openAPI in a ts file with a bunch of recursive exported namespaces step 4 gives us those types in a .d.ts file We do 3 and 4 in a subdirectory so that we can publish the types as an internal package, which means we can give front-end a types package to consume for all the calls we want to make to backend. Well, we actually publish an sdk package that wraps the types and encapsulates the logic for making the HTTP request to the right endpoint, so we don't have to worry about calling POST when the endpoint is PUT or fat-fingering the response type cast through the entire front-end and can instead just call a function with well-defined types. Plus, now it's super easy to test the interface, because we have a single thing we need to test. So we write an exhaustive happy-path (ie 2xx status codes) integration test against the sdk that actually calls into backend gets the response and checks the response against the type and then the values we expect for each field.