5 ms·

This is my biggest issue with the language. Fetch returns “any” meaning you can’t trust the data you received is actually the data you expected. Bugs from this

This is my biggest issue with the language. Fetch returns “any” meaning you can’t trust the data you received is actually the data you expected. Bugs from this mismatch will be many lines away (on first use) and more difficult to find. Because of this “goal of the language” you cited, there’s no built-in way to validate any data at runtime. In nearly any other typed language I have some deserialization mechanism. Not so in Typescript! This decision led to more bugs in our codebase than any other. The compiler actively lies to you about the types you’ll have at runtime. The only solutions are codegen or writing validators to poorly approximate what Typescript should give us for free.

madeofpalk 3y ago
The TS devs have mentioned that they wish JSON.parse returned unknown, but the change is too disruptive now.
- iainmerrick 3y ago
  It would be a lot nicer if it instead returned some JsonType that’s a union of all the possible JSON values. Anyone know if there’s a good reason why it doesn’t do that?
  - striking 3y ago
    You can pass an arbitrary rehydration function, which can return non-JSON-representable types
    - Dylan16807 3y ago
      It could look at the return type of your reviver function, or at least whether you passed one in.
  - kristiandupont 3y ago
    There's a big discussion about this: https://github.com/microsoft/TypeScript/issues/1897 https://github.com/microsoft/TypeScript/issues/1897. The benefit seems extremely limited to me. Valid JSON is obviously a subset of `any`, but I can't think of a situation where that particular specificity provides any value. Can you?
    - iainmerrick 3y ago
      The value is when you’re parsing the JSON afterwards. It’s good to know you can match it exhaustively -- each value is either a Record<string, Json>, Json[], string, number, Boolean or null, and nothing else. Edit to add: I think “any” is almost always a big cop-out because you couldn’t be bothered figuring out the correct type, and it often causes problems (loss of type coverage) further down the line. I admit I do use “any” in my own code when I really need to, but a library should work harder to avoid it, and the standard platform typings should work harder still.
  - rcfox 3y ago
    What are the common operations you can perform on that union?
    - consilient 3y ago
      You can narrow it exhaustively.
  - madeofpalk 3y ago
    That's effectively what unknown would be - at least the outcome would mostly be the same. You'll end up narrowing values in just the same way.
mnahkies 3y ago
You should take a look at https://zod.dev/ https://zod.dev/ if you haven't already - it's a library for runtime parsing that works really well for your use case. Types are inferred from the schema though personally I like to handwrite types as well to sense check that the schema describes the type I think it does
- hellcow 3y ago
  I’ve used zod and every other schema validator available for this. Some problems: 1. Types are not written in typescript anymore. Or you have to define them twice and manually ensure they match. ReturnType<typeof MyType> pollutes the codebase. 2. Types have to be defined in order, since they’re now consts. If you have a lot of types which embed other types, good luck determining that order by hand. 3. Recursive types need to be treated specially because a const variable can’t reference itself without some lazy evaluation mechanism. TS could solve all of this by baking this into the language.
  - jamescostian 3y ago
    1. You can just use `export type Foo = z.infer<typeof fooParser>` in one place and then import Foo everywhere else, without using z.infer everywhere else 2. Use let and modify your types as new ones become available - union them with a new object that contains the new property you need 3. How often are you making recursive types? I agree that all of this could be made easier, but zod is the best we have and great for most normal usage. The reason TS doesn't want to make this available at runtime is that it means so many changes they make will become breaking changes. Perhaps one day when there's less development on TS we'll see this get added
    - IggleSniggle 3y ago
      Including runtime checks would also have performance implications. I really enjoyed using myzod (more performative, simple, zod) for awhile, but recently I’ve been using Typia, which is a codegen approach. I have mixed feelings about it, and from my own benchmarking it’s performance seems overstated, but the idea is sound: because we know the type, we can compile better, type-optimized serialize/deserialize functions. As for not littering the codebase with runtime checks, it may be worth reiterating to the person above that you really should only do type determinations at the I/O edges: you parse your input, and it becomes known from then onwards. You runtime type-check your output, and its requirements propagate upwards through your program.
deleted 3y ago
[deleted]
__ryan__ 3y ago
Yes, “any” is a wart. And it’s a bad one. The correct type for values you don’t know the type of (like the response of an API call) is “unknown”. TypeScript does not provide the facilities you describe because there is not a one-size-fits-all solution to the cases that are possible and common in JavaScript. It is left to the developer to decide how to validate unknown data at the boundaries of the API. There are third party libraries that facilitate this in different ways with different trade-offs. The compiler actively lies to you about the types you’ll have at runtime. I find this to be rare if you are using strict mode with proper TypeScript definition files for your platform and dependencies. Usually the lie is in your own code or bad dependencies when an “unknown” type (including “any”) is cast to a concrete type without being validated. In nearly any other typed language I have some deserialization mechanism. Could you provide examples? I either don’t understand or I disagree.
- hellcow 3y ago
  > Usually the lie is in your own code or bad dependencies The lie is almost always in an external API response from fetch (hence the complaint about “any” above). > Could you provide examples? Off the top of my head… Go’s stdlib json.Unmarshal and Rust’s Serde derive Deserialize.
  - __ryan__ 3y ago
    The lie is when your code uses* the “any” value where a concrete type is expected. I was misunderstanding your point with the deserialize. Edit: “using” -> “uses”
- consilient 3y ago
  > Usually the lie is in your own code or bad dependencies when an “unknown” type (including “any”) is cast to a concrete type without being validated. Yes, but one of those bad dependencies is the standard library.
  - __ryan__ 3y ago
    When does the standard library lie in this case?
kristiandupont 3y ago
You might like TS Reset: https://github.com/total-typescript/ts-reset https://github.com/total-typescript/ts-reset, which fixes this particular problem. I don't personally find it to be a big issue though. Regarding runtime type checking, if you were to write something that can handle the total space of possible TS types, you would end up with incredibly complex machinery. It would be hard to make it perform, both in terms of speed and bundle size, and it would be hard to predict. I think Zod or perhaps https://arktype.io/ https://arktype.io/ which target a reasonable subset are the only way to go.
DangitBobby 3y ago
This was driving me nuts in a project with lots of backend churn. Runtime type validation libraries like typebox and zod (I like typebox) can really save your bacon. The downside is the underlying types tend to be more complex when viewed in your IDE, but I think it's worth it.
- eyelidlessness 3y ago
  Here’s a neat trick for those complex types: type Identity<T> = T // This can be made recursive to an extent, alas I’m on mobile type Merge<T> = { [K in keyof T]: Identity<T[K]> } type ReadableFoo = Merge<UnreadableFoo>
mirekrusin 3y ago
This can't be solved by static analysis - anything that crosses i/o boundary has to be asserted, refuted or predicated at runtime and you have libraries for it ie. [0] which doesn't throw (based on refutations which can be mapped to predicates without much cost and assertions) or [1] which throws (based on assertions). Predicates are the most performant but won't give you any indication on why it failed (ie. some nested field was null but was expected to be number etc). Refutations is great sweet spot as it's fast while giving information about error. Assertions are slow, but more often than not you don't care. You can map between any of them, but it doesn't make much sense for mapping ie. assertion to predicate as you'd be paying cost for nested try/catch while dropping error information. Refutation is great base for all 3. [0] https://github.com/preludejs/refute https://github.com/preludejs/refute [1] https://github.com/appliedblockchain/assert-combinators https://github.com/appliedblockchain/assert-combinators
- hellcow 3y ago
  The complaint is that Typescript not emitting any of the type information for the runtime means every library must reimplement the whole TS type system.
  - mirekrusin 3y ago
    Yes, that's true, they could support emitting metadata with explicit keyword which would help and wouldn't bloat anything implicitly, they already do emit code for enums for example. Personally I'm fan of not introducing new language that runs at comp time, just use the same language to have macros and operations on types for free - just like Zig does it. Typescript type system is already turing complete so it's not like they'd be loosing anything there.

madeofpalk 3y ago

The TS devs have mentioned that they wish JSON.parse returned unknown, but the change is too disruptive now.

iainmerrick 3y ago
It would be a lot nicer if it instead returned some JsonType that’s a union of all the possible JSON values. Anyone know if there’s a good reason why it doesn’t do that?
- striking 3y ago
  You can pass an arbitrary rehydration function, which can return non-JSON-representable types
  - Dylan16807 3y ago
    It could look at the return type of your reviver function, or at least whether you passed one in.
- kristiandupont 3y ago
  There's a big discussion about this: https://github.com/microsoft/TypeScript/issues/1897 https://github.com/microsoft/TypeScript/issues/1897. The benefit seems extremely limited to me. Valid JSON is obviously a subset of `any`, but I can't think of a situation where that particular specificity provides any value. Can you?
  - iainmerrick 3y ago
    The value is when you’re parsing the JSON afterwards. It’s good to know you can match it exhaustively -- each value is either a Record<string, Json>, Json[], string, number, Boolean or null, and nothing else. Edit to add: I think “any” is almost always a big cop-out because you couldn’t be bothered figuring out the correct type, and it often causes problems (loss of type coverage) further down the line. I admit I do use “any” in my own code when I really need to, but a library should work harder to avoid it, and the standard platform typings should work harder still.
- rcfox 3y ago
  What are the common operations you can perform on that union?
  - consilient 3y ago
    You can narrow it exhaustively.
- madeofpalk 3y ago
  That's effectively what unknown would be - at least the outcome would mostly be the same. You'll end up narrowing values in just the same way.

mnahkies 3y ago

You should take a look at https://zod.dev/ https://zod.dev/ if you haven't already - it's a library for runtime parsing that works really well for your use case. Types are inferred from the schema though personally I like to handwrite types as well to sense check that the schema describes the type I think it does

hellcow 3y ago
I’ve used zod and every other schema validator available for this. Some problems: 1. Types are not written in typescript anymore. Or you have to define them twice and manually ensure they match. ReturnType<typeof MyType> pollutes the codebase. 2. Types have to be defined in order, since they’re now consts. If you have a lot of types which embed other types, good luck determining that order by hand. 3. Recursive types need to be treated specially because a const variable can’t reference itself without some lazy evaluation mechanism. TS could solve all of this by baking this into the language.
- jamescostian 3y ago
  1. You can just use `export type Foo = z.infer<typeof fooParser>` in one place and then import Foo everywhere else, without using z.infer everywhere else 2. Use let and modify your types as new ones become available - union them with a new object that contains the new property you need 3. How often are you making recursive types? I agree that all of this could be made easier, but zod is the best we have and great for most normal usage. The reason TS doesn't want to make this available at runtime is that it means so many changes they make will become breaking changes. Perhaps one day when there's less development on TS we'll see this get added
  - IggleSniggle 3y ago
    Including runtime checks would also have performance implications. I really enjoyed using myzod (more performative, simple, zod) for awhile, but recently I’ve been using Typia, which is a codegen approach. I have mixed feelings about it, and from my own benchmarking it’s performance seems overstated, but the idea is sound: because we know the type, we can compile better, type-optimized serialize/deserialize functions. As for not littering the codebase with runtime checks, it may be worth reiterating to the person above that you really should only do type determinations at the I/O edges: you parse your input, and it becomes known from then onwards. You runtime type-check your output, and its requirements propagate upwards through your program.

deleted 3y ago

[deleted]

__ryan__ 3y ago

Yes, “any” is a wart. And it’s a bad one. The correct type for values you don’t know the type of (like the response of an API call) is “unknown”. TypeScript does not provide the facilities you describe because there is not a one-size-fits-all solution to the cases that are possible and common in JavaScript. It is left to the developer to decide how to validate unknown data at the boundaries of the API. There are third party libraries that facilitate this in different ways with different trade-offs. The compiler actively lies to you about the types you’ll have at runtime. I find this to be rare if you are using strict mode with proper TypeScript definition files for your platform and dependencies. Usually the lie is in your own code or bad dependencies when an “unknown” type (including “any”) is cast to a concrete type without being validated. In nearly any other typed language I have some deserialization mechanism. Could you provide examples? I either don’t understand or I disagree.

hellcow 3y ago
> Usually the lie is in your own code or bad dependencies The lie is almost always in an external API response from fetch (hence the complaint about “any” above). > Could you provide examples? Off the top of my head… Go’s stdlib json.Unmarshal and Rust’s Serde derive Deserialize.
- __ryan__ 3y ago
  The lie is when your code uses* the “any” value where a concrete type is expected. I was misunderstanding your point with the deserialize. Edit: “using” -> “uses”
consilient 3y ago
> Usually the lie is in your own code or bad dependencies when an “unknown” type (including “any”) is cast to a concrete type without being validated. Yes, but one of those bad dependencies is the standard library.
- __ryan__ 3y ago
  When does the standard library lie in this case?

kristiandupont 3y ago

You might like TS Reset: https://github.com/total-typescript/ts-reset https://github.com/total-typescript/ts-reset, which fixes this particular problem. I don't personally find it to be a big issue though. Regarding runtime type checking, if you were to write something that can handle the total space of possible TS types, you would end up with incredibly complex machinery. It would be hard to make it perform, both in terms of speed and bundle size, and it would be hard to predict. I think Zod or perhaps https://arktype.io/ https://arktype.io/ which target a reasonable subset are the only way to go.

DangitBobby 3y ago

This was driving me nuts in a project with lots of backend churn. Runtime type validation libraries like typebox and zod (I like typebox) can really save your bacon. The downside is the underlying types tend to be more complex when viewed in your IDE, but I think it's worth it.

eyelidlessness 3y ago
Here’s a neat trick for those complex types: type Identity<T> = T // This can be made recursive to an extent, alas I’m on mobile type Merge<T> = { [K in keyof T]: Identity<T[K]> } type ReadableFoo = Merge<UnreadableFoo>

mirekrusin 3y ago

This can't be solved by static analysis - anything that crosses i/o boundary has to be asserted, refuted or predicated at runtime and you have libraries for it ie. [0] which doesn't throw (based on refutations which can be mapped to predicates without much cost and assertions) or [1] which throws (based on assertions). Predicates are the most performant but won't give you any indication on why it failed (ie. some nested field was null but was expected to be number etc). Refutations is great sweet spot as it's fast while giving information about error. Assertions are slow, but more often than not you don't care. You can map between any of them, but it doesn't make much sense for mapping ie. assertion to predicate as you'd be paying cost for nested try/catch while dropping error information. Refutation is great base for all 3. [0] https://github.com/preludejs/refute https://github.com/preludejs/refute [1] https://github.com/appliedblockchain/assert-combinators https://github.com/appliedblockchain/assert-combinators

hellcow 3y ago
The complaint is that Typescript not emitting any of the type information for the runtime means every library must reimplement the whole TS type system.
- mirekrusin 3y ago
  Yes, that's true, they could support emitting metadata with explicit keyword which would help and wouldn't bloat anything implicitly, they already do emit code for enums for example. Personally I'm fan of not introducing new language that runs at comp time, just use the same language to have macros and operations on types for free - just like Zig does it. Typescript type system is already turing complete so it's not like they'd be loosing anything there.