6 ms·

An Inconsistent Truth: Next.js and Typesafety

103 points by theobr 5y ago

theobr 5y ago
Hey y’all! I’ve been a big nerd about all things React, Next and Typescript for awhile and wanted to formalize some of my thoughts and frustrations into this article. I was lucky enough to get some awesome eyes on it early, including a handful of people from Vercel. I hope this sparks some good discussion!
- markharrisuk999 5y ago
  Is there any reason why you didn't use BlitzJS? Where, I think this issue is largely solved with Typescript?
moksly 5y ago
Don’t you need some pretty solid source data for you to trust them? I only recently picked up typescript, coming from decades of .Net and later a Python, and the way you can turn things like json results into typed data via interfaces while also having the vast open source libraries I’ve done to love from Python, I gotta say it’s just been so easy to work in an enterprise environment where the source data is often really terrible. I can certainly follow the points of this article. It is sort of silly to type an uuid as a string, but when does it actually break?
- throwanem 5y ago
  When you don't get warned that you assigned or passed something that's a string but not a UUID, because the type checker can't tell them apart. Nominal types fix this. Typescript doesn't precisely implement them, but its "branded types" seem to be close enough for most purposes. For further discussion and a sandbox example, see https://www.typescriptlang.org/play#example/nominal-typing https://www.typescriptlang.org/play#example/nominal-typing
mirekrusin 5y ago
Types can be asserted at runtime (parsed) at IO boundaries (reading http request or response, websocket message, parsing json file etc). Once they enter statically type system they don't need to be asserted again. The difference it makes is illusion of type-safety vs type-safety this article touches on. It's basically mapping of an `unknown` type into known one, at runtime. You can try to bind service with client (types) somehow but in many cases this will fail in production as you can't guarantee paired versioning, due to normal situations by design of your architecture or temporary mid-deployment state or other team doing something they were not suppose to do (new client is deployed connecting to old service or vice versa) etc. It's hard to avoid runtime parsing/type assertion in general. Functional combinator approaches like [0] or faster [1] with predicate/assert semantics work very well with typescript, which is very pleasant language to work with. [0] https://github.com/appliedblockchain/assert-combinators https://github.com/appliedblockchain/assert-combinators [1] https://github.com/preludejs/refute https://github.com/preludejs/refute
- the_duke 5y ago
  That's only partially true for Typescript and requires a lot of discipline. TS is continuously getting better, but it's still possible to lose type safety with some constructs without the compiler warning you about it at all. The third party typings for JS dependencies are also often not a 100% correct, especially for rare code paths and edge cases. And due to how easy it is to fall back to any, there are also Typescript native libraries that don't uphold the guarantees they seem to give. Typescript is great, but it's far from bullet proof, and it's extremely frustrating to deal with bugs that the type system supposedly should have prevented.
  - frenchyatwork 5y ago
    Some of those issues can be remedied by linting rules (preventing unsafe use of any), but typing of dependencies can be a headache. Even 1st party typing and Typescript dependencies can regularly be wrong if type safety was not important to whoever wrote them.
    - 5y ago
jensneuse 5y ago
Very interesting topic, actually it's one of my favourite ones. I've been working for the last year or so on the problem to improve type-safety for NextJS. My solution works like this: Step 1: introspect your DataSources (Postgres, MySQL, GraphQL, REST, etc...) Step 2: combine them into a virtual Graph (virtual GraphQL API) Step 3: write GraphQL Operations and "compile" them into JSON RPC + generate a 100% type-safe client Result? No more double declaration of types. No manual typing. More info on how this works with NextJS and Typescript here: [0] Note, I'm the founder of WunderGraph and obsessed with creating the perfect developer experience for working with APIs. I appreciate any feedback. [0]: https://wundergraph.com/docs/overview/features/generated_clients https://wundergraph.com/docs/overview/features/generated_cli...
- theobr 5y ago
  This is super cool and I’m excited to dig in more Any chance there are some demos building a full stack app on this tech?
  - jensneuse 5y ago
    Hey, thanks for you feedback. Yes absolutely! Here's an example using Apollo Federation, REST APIs and a standalone GraphQL API: [0] Alternatively, you can also just run "wunderctl init --template nextjs-starter" to start with the NextJS template. (Obviously you need to install it first: yarn global add @wundergraph/wunderctl@latest) We're going open source with this solution soon. So, any feedback is appreciated! You can also join our discord and shoot questions. =) [0]: https://github.com/wundergraph/wundergraph-demo https://github.com/wundergraph/wundergraph-demo
- SlickStef11 5y ago
  Super cool! Will be checking out the documentation!
- kansface 5y ago
  How do you deal with multiple versions on the back end - like during a migration, or a partial migration, deploy, etc?
crate_barre 5y ago
What is the value proposition of all this for simple apis? The author is using one of the simplest examples, and almost all of the proposed solutions other than the first problematic one read like obtuse technical literature to me. All for what, so I don’t fuck up the ‘id’ param on a hello-world fetch request? What does this stuff look like on anything seemingly more complex (my best guess, it’ll look more complex).
- theobr 5y ago
  You should check out the video I link at the end - the most complex problems have the exact same solutions when your types are inferred :)
  - crate_barre 5y ago
    Yeah, I still don’t get it. This all seems stupid, sorry. But nice post, keep at it, I don’t want anyone on any of my teams bringing this nonsense to work. Type safety is not even close to delivering good software.
terracottage 5y ago
The one thing I would ask this person is how big and complex a system they've built like this. Because if you optimize all the slack out of a system, you have no room to manoeuvre. In this case, you need to update all your code and dbs all at once if any type changes. Because it's all linked. In my experience this is not feasible one you reach a certain size. You need to be able to upgrade parts in isolation while keeping the majority working without touching it.
- chii 5y ago
  > You need to be able to upgrade parts in isolation while keeping the majority working without touching it. having no slack means you get told about this problem during compile time, rather than having the need for an experienced developer who understands the entire system and is able to do the above without the help of a compiler.
- theobr 5y ago
  Without a strongly inferred type system, all of the things you’re describing around “changing everything to change one thing” are just as much a problem. In a well typed system, you can choose how deep you want your changes to go. Let’s say you choose to rename a field in db, I.e. “imageUrl” -> “profilePicURL”. Upon making this change, you will receive type errors on the client consuming it. At this point, you can make a choice. You can go address all the furthest end consumers, which is what you imply is necessary here. You can just as easily re-shape the data being returned though. return { …user, imageUrl: user.profilePicUrl } I firmly believe we’re nearing the “best of both worlds” here :)
- eurasiantiger 5y ago
  With GraphQL, the idea is to never change existing behaviour, but to use directives to flag them deprecated and direct devs to change to using the new type/field/query/whatever. This gives a guarantee about not breaking existing things.
SavantIdiot 5y ago
It will happen. Eventually runtime / io typesafety will happen, just not yet. A good example of how this was solved in the past was Microsofts IUnknown interface where the type of a blob is determined after it is downloaded. It basically converts unknown to a type on access, but is just too heavy duty for bandwidth requirements in a runtime/JIT environment.
- DaiPlusPlus 5y ago
  IUnknown was the base type for all COM interfaces. What does that have to do with downloads?
  - SavantIdiot 5y ago
    I was referring to memory bandwidth and processing bandwidth, not download bandwidth. Re-reading I can see how that is confusing. It was also the base type for all DirectX interfaces. It effectively provides type info for a binary blob with implicit version information that could be used for extremely robust runtime data-typing. However there's all kinds of overhead associated with it, especially if the interface needs to be parsed constantly.
klibertp 5y ago
> with an implicit type contract (potentially generated) through the creation of these files Racket is able to automatically convert static types of Typed Racket into contracts when values flow between typed and untyped worlds. This happens automatically and transparently, which means you don't have to worry about almost at all. One advantage Racket has over JS is the module system (well, it holds the same advantage over almost all the other languages), which allows typed and untyped code to reside in the the same file, yet have a clear boundary between them. I can't find it right now, but there was a paper describing how it works. It's probably somewhere here: https://github.com/samth/gradual-typing-bib https://github.com/samth/gradual-typing-bib (if you're curious enough to read many tens of abstracts...)
crate_barre 5y ago
- theobr 5y ago
  This reply alone makes all the effort I’ve put into my stack worth it. Thank you :)
  - crate_barre 5y ago
    You’d have to explain, I was certainly being vitriolic, but I hope HN hasn’t lost its poor taste for theatrics and poetry - granted, as a moron, I believe everything I’ve said. Edit: is it so much to ask to not want to write code that looks like gibberish? I don’t care what the benefits are, it looks like gibberish code. Can type safety be accomplished without gibberish code? I think in Java it goes something like this: Function (String s, int num) Ok, I’m on board. You Typescript bastards are writing gibberish code and I don’t care who disagrees.
    - bmeski 5y ago
      I agree that this stuff is getting out of hand. The most important part of all that code is the part in the prisma calls, which would handle all the caching and request logic. The rest is literally just making sure you name your stuff correctly and render it. If you want actual type safety for your large client-side app, use rust/wasm. If your client side app is small, why do any of this?
      - theobr 5y ago
        Point me at one fast moving “large client side app” that uses Rust via WASM over Typescript and I will move my entire company over
        techpression 5y ago
        Not Rust, but Figma (which everyone seem to love) is using C++ and Web Assembly and they seem to be moving quite fast. I guess it’s not the entire stack but still, seems viable for them.
Sophistifunk 5y ago
That's nothing, there's also implicit boundaries across which NextJS will convert your objects to JSON, and NOT CONVERT THEM BACK AFTER, while pretending there's no change. Have a date in your object? Not any more pal, it's a string. You return an instance with methods in it? Wanna bet? How about a Map or Set? Guess what? Nah.
- diroussel 5y ago
  Which boundaries are these? I’ve not hit them. But then I don’t use the api features.
  - Sophistifunk 5y ago
    getInitialProps and the various slightly-differently-worded replacements thereof. Anything returned from there must be transparently JSONable or you're gonna have a bad time. And there's no way to enforce this other than warning comments and code review, so it gets violated all the time.
gitgud 5y ago
I recommend "next-rpc" [1] which is a small library allows the nextjs client-side pages to call the API function using a type-checked function interface. [1] https://github.com/Janpot/next-rpc https://github.com/Janpot/next-rpc
kaufmae 5y ago
I‘m sorry but this is exactly how you should NOT do it unless you’ve lost your mind in type limbo. It‘s ideological overarching for totalitarian TS. After that, the code looks cluttered and the amount of typing syntax compared to the benefits is ridiculous, even with inference. Don‘t waste your time typing the shit out of your js, use ts selectively, use it for the signatures of your own pure functions and move on.
merrywhether 5y ago
My biggest gripe with inferred types is that you wind up with errors at random usage sites instead of at where the actual type “error” is. For example, with inferred return types you can wind up returning `a | b` when you thought you were returning `a` and this might actually be coincidentally fine for a bit until you use it somewhere that only takes `a` and you get a type mismatch. You can then trace backwards until you find the source of the bad assignment, which can be a few layers after a bit of inference, and this always felt antithetical to the improved ergonomics of types. As a result I always enforce explicit return type annotations with inference only within scopes. But TS happily supports a spectrum!
- captainmuon 5y ago
  Yeah, type interference makes me paranoid. I'm always afraid of this kind of action at a distance. Say I have a function with inferred argument types, and I normally use it with an `int`. Then in one place I call it with a `float` argument. I worry that this will propagate back and cause a lot of variables to change to float, and thus change the semantics of my program. Now I think inference works in one direction only (at least in languages with HM type interference), so this is a non-issue. But I'm often uncertain. Anyway, in languages with optional typing like TS, I usually specify the types in function signatures manually to be 100% sure I understand what is going on.
epolanski 5y ago
As someone who writes fp-ts regularly: TypeScript has way too many ways to be type unsafe. Discipline and parsing API boundaries can give you typings you can 99% trust. Discipline is very needed because of many factors, last but not least the fact that there is an abundance of type unsafe apis (e.g. JSON.parse). The worse part isn't even that you cannot have total type safety imho, there has to be compromise for a language willing to be a JS superset. It's how difficult it is the type system when more advanced patterns are required, often the things are completely impossible. Still, I would never do front end on a different language, and I do love elm and other type safe alternatives.

103 points by theobr 5y ago

Hey y’all! I’ve been a big nerd about all things React, Next and Typescript for awhile and wanted to formalize some of my thoughts and frustrations into this article. I was lucky enough to get some awesome eyes on it early, including a handful of people from Vercel. I hope this sparks some good discussion!

markharrisuk999 5y ago
Is there any reason why you didn't use BlitzJS? Where, I think this issue is largely solved with Typescript?

moksly 5y ago

Don’t you need some pretty solid source data for you to trust them? I only recently picked up typescript, coming from decades of .Net and later a Python, and the way you can turn things like json results into typed data via interfaces while also having the vast open source libraries I’ve done to love from Python, I gotta say it’s just been so easy to work in an enterprise environment where the source data is often really terrible. I can certainly follow the points of this article. It is sort of silly to type an uuid as a string, but when does it actually break?

throwanem 5y ago
When you don't get warned that you assigned or passed something that's a string but not a UUID, because the type checker can't tell them apart. Nominal types fix this. Typescript doesn't precisely implement them, but its "branded types" seem to be close enough for most purposes. For further discussion and a sandbox example, see https://www.typescriptlang.org/play#example/nominal-typing https://www.typescriptlang.org/play#example/nominal-typing

mirekrusin 5y ago

Types can be asserted at runtime (parsed) at IO boundaries (reading http request or response, websocket message, parsing json file etc). Once they enter statically type system they don't need to be asserted again. The difference it makes is illusion of type-safety vs type-safety this article touches on. It's basically mapping of an `unknown` type into known one, at runtime. You can try to bind service with client (types) somehow but in many cases this will fail in production as you can't guarantee paired versioning, due to normal situations by design of your architecture or temporary mid-deployment state or other team doing something they were not suppose to do (new client is deployed connecting to old service or vice versa) etc. It's hard to avoid runtime parsing/type assertion in general. Functional combinator approaches like [0] or faster [1] with predicate/assert semantics work very well with typescript, which is very pleasant language to work with. [0] https://github.com/appliedblockchain/assert-combinators https://github.com/appliedblockchain/assert-combinators [1] https://github.com/preludejs/refute https://github.com/preludejs/refute

the_duke 5y ago
That's only partially true for Typescript and requires a lot of discipline. TS is continuously getting better, but it's still possible to lose type safety with some constructs without the compiler warning you about it at all. The third party typings for JS dependencies are also often not a 100% correct, especially for rare code paths and edge cases. And due to how easy it is to fall back to any, there are also Typescript native libraries that don't uphold the guarantees they seem to give. Typescript is great, but it's far from bullet proof, and it's extremely frustrating to deal with bugs that the type system supposedly should have prevented.
- frenchyatwork 5y ago
  Some of those issues can be remedied by linting rules (preventing unsafe use of any), but typing of dependencies can be a headache. Even 1st party typing and Typescript dependencies can regularly be wrong if type safety was not important to whoever wrote them.
  - 5y ago

jensneuse 5y ago

Very interesting topic, actually it's one of my favourite ones. I've been working for the last year or so on the problem to improve type-safety for NextJS. My solution works like this: Step 1: introspect your DataSources (Postgres, MySQL, GraphQL, REST, etc...) Step 2: combine them into a virtual Graph (virtual GraphQL API) Step 3: write GraphQL Operations and "compile" them into JSON RPC + generate a 100% type-safe client Result? No more double declaration of types. No manual typing. More info on how this works with NextJS and Typescript here: [0] Note, I'm the founder of WunderGraph and obsessed with creating the perfect developer experience for working with APIs. I appreciate any feedback. [0]: https://wundergraph.com/docs/overview/features/generated_clients https://wundergraph.com/docs/overview/features/generated_cli...

theobr 5y ago
This is super cool and I’m excited to dig in more Any chance there are some demos building a full stack app on this tech?
- jensneuse 5y ago
  Hey, thanks for you feedback. Yes absolutely! Here's an example using Apollo Federation, REST APIs and a standalone GraphQL API: [0] Alternatively, you can also just run "wunderctl init --template nextjs-starter" to start with the NextJS template. (Obviously you need to install it first: yarn global add @wundergraph/wunderctl@latest) We're going open source with this solution soon. So, any feedback is appreciated! You can also join our discord and shoot questions. =) [0]: https://github.com/wundergraph/wundergraph-demo https://github.com/wundergraph/wundergraph-demo
SlickStef11 5y ago
Super cool! Will be checking out the documentation!
kansface 5y ago
How do you deal with multiple versions on the back end - like during a migration, or a partial migration, deploy, etc?

crate_barre 5y ago

What is the value proposition of all this for simple apis? The author is using one of the simplest examples, and almost all of the proposed solutions other than the first problematic one read like obtuse technical literature to me. All for what, so I don’t fuck up the ‘id’ param on a hello-world fetch request? What does this stuff look like on anything seemingly more complex (my best guess, it’ll look more complex).

theobr 5y ago
You should check out the video I link at the end - the most complex problems have the exact same solutions when your types are inferred :)
- crate_barre 5y ago
  Yeah, I still don’t get it. This all seems stupid, sorry. But nice post, keep at it, I don’t want anyone on any of my teams bringing this nonsense to work. Type safety is not even close to delivering good software.

terracottage 5y ago

The one thing I would ask this person is how big and complex a system they've built like this. Because if you optimize all the slack out of a system, you have no room to manoeuvre. In this case, you need to update all your code and dbs all at once if any type changes. Because it's all linked. In my experience this is not feasible one you reach a certain size. You need to be able to upgrade parts in isolation while keeping the majority working without touching it.

chii 5y ago
> You need to be able to upgrade parts in isolation while keeping the majority working without touching it. having no slack means you get told about this problem during compile time, rather than having the need for an experienced developer who understands the entire system and is able to do the above without the help of a compiler.
theobr 5y ago
Without a strongly inferred type system, all of the things you’re describing around “changing everything to change one thing” are just as much a problem. In a well typed system, you can choose how deep you want your changes to go. Let’s say you choose to rename a field in db, I.e. “imageUrl” -> “profilePicURL”. Upon making this change, you will receive type errors on the client consuming it. At this point, you can make a choice. You can go address all the furthest end consumers, which is what you imply is necessary here. You can just as easily re-shape the data being returned though. return { …user, imageUrl: user.profilePicUrl } I firmly believe we’re nearing the “best of both worlds” here :)
eurasiantiger 5y ago
With GraphQL, the idea is to never change existing behaviour, but to use directives to flag them deprecated and direct devs to change to using the new type/field/query/whatever. This gives a guarantee about not breaking existing things.

SavantIdiot 5y ago

It will happen. Eventually runtime / io typesafety will happen, just not yet. A good example of how this was solved in the past was Microsofts IUnknown interface where the type of a blob is determined after it is downloaded. It basically converts unknown to a type on access, but is just too heavy duty for bandwidth requirements in a runtime/JIT environment.

DaiPlusPlus 5y ago
IUnknown was the base type for all COM interfaces. What does that have to do with downloads?
- SavantIdiot 5y ago
  I was referring to memory bandwidth and processing bandwidth, not download bandwidth. Re-reading I can see how that is confusing. It was also the base type for all DirectX interfaces. It effectively provides type info for a binary blob with implicit version information that could be used for extremely robust runtime data-typing. However there's all kinds of overhead associated with it, especially if the interface needs to be parsed constantly.

klibertp 5y ago

> with an implicit type contract (potentially generated) through the creation of these files Racket is able to automatically convert static types of Typed Racket into contracts when values flow between typed and untyped worlds. This happens automatically and transparently, which means you don't have to worry about almost at all. One advantage Racket has over JS is the module system (well, it holds the same advantage over almost all the other languages), which allows typed and untyped code to reside in the the same file, yet have a clear boundary between them. I can't find it right now, but there was a paper describing how it works. It's probably somewhere here: https://github.com/samth/gradual-typing-bib https://github.com/samth/gradual-typing-bib (if you're curious enough to read many tens of abstracts...)

crate_barre 5y ago

theobr 5y ago
This reply alone makes all the effort I’ve put into my stack worth it. Thank you :)
- crate_barre 5y ago
  You’d have to explain, I was certainly being vitriolic, but I hope HN hasn’t lost its poor taste for theatrics and poetry - granted, as a moron, I believe everything I’ve said. Edit: is it so much to ask to not want to write code that looks like gibberish? I don’t care what the benefits are, it looks like gibberish code. Can type safety be accomplished without gibberish code? I think in Java it goes something like this: Function (String s, int num) Ok, I’m on board. You Typescript bastards are writing gibberish code and I don’t care who disagrees.
  - bmeski 5y ago
    I agree that this stuff is getting out of hand. The most important part of all that code is the part in the prisma calls, which would handle all the caching and request logic. The rest is literally just making sure you name your stuff correctly and render it. If you want actual type safety for your large client-side app, use rust/wasm. If your client side app is small, why do any of this?
    - theobr 5y ago
      Point me at one fast moving “large client side app” that uses Rust via WASM over Typescript and I will move my entire company over
      - techpression 5y ago
        Not Rust, but Figma (which everyone seem to love) is using C++ and Web Assembly and they seem to be moving quite fast. I guess it’s not the entire stack but still, seems viable for them.

Sophistifunk 5y ago

That's nothing, there's also implicit boundaries across which NextJS will convert your objects to JSON, and NOT CONVERT THEM BACK AFTER, while pretending there's no change. Have a date in your object? Not any more pal, it's a string. You return an instance with methods in it? Wanna bet? How about a Map or Set? Guess what? Nah.

diroussel 5y ago
Which boundaries are these? I’ve not hit them. But then I don’t use the api features.
- Sophistifunk 5y ago
  getInitialProps and the various slightly-differently-worded replacements thereof. Anything returned from there must be transparently JSONable or you're gonna have a bad time. And there's no way to enforce this other than warning comments and code review, so it gets violated all the time.

gitgud 5y ago

I recommend "next-rpc" [1] which is a small library allows the nextjs client-side pages to call the API function using a type-checked function interface. [1] https://github.com/Janpot/next-rpc https://github.com/Janpot/next-rpc

kaufmae 5y ago

I‘m sorry but this is exactly how you should NOT do it unless you’ve lost your mind in type limbo. It‘s ideological overarching for totalitarian TS. After that, the code looks cluttered and the amount of typing syntax compared to the benefits is ridiculous, even with inference. Don‘t waste your time typing the shit out of your js, use ts selectively, use it for the signatures of your own pure functions and move on.

merrywhether 5y ago

My biggest gripe with inferred types is that you wind up with errors at random usage sites instead of at where the actual type “error” is. For example, with inferred return types you can wind up returning `a | b` when you thought you were returning `a` and this might actually be coincidentally fine for a bit until you use it somewhere that only takes `a` and you get a type mismatch. You can then trace backwards until you find the source of the bad assignment, which can be a few layers after a bit of inference, and this always felt antithetical to the improved ergonomics of types. As a result I always enforce explicit return type annotations with inference only within scopes. But TS happily supports a spectrum!

captainmuon 5y ago
Yeah, type interference makes me paranoid. I'm always afraid of this kind of action at a distance. Say I have a function with inferred argument types, and I normally use it with an `int`. Then in one place I call it with a `float` argument. I worry that this will propagate back and cause a lot of variables to change to float, and thus change the semantics of my program. Now I think inference works in one direction only (at least in languages with HM type interference), so this is a non-issue. But I'm often uncertain. Anyway, in languages with optional typing like TS, I usually specify the types in function signatures manually to be 100% sure I understand what is going on.

epolanski 5y ago

As someone who writes fp-ts regularly: TypeScript has way too many ways to be type unsafe. Discipline and parsing API boundaries can give you typings you can 99% trust. Discipline is very needed because of many factors, last but not least the fact that there is an abundance of type unsafe apis (e.g. JSON.parse). The worse part isn't even that you cannot have total type safety imho, there has to be compromise for a language willing to be a JS superset. It's how difficult it is the type system when more advanced patterns are required, often the things are completely impossible. Still, I would never do front end on a different language, and I do love elm and other type safe alternatives.