7 ms·

> To simplify the migration process both for users and Zod's ecosystem of associated libraries, Zod 4 is being published alongside Zod 3 as part of the zod@3.25

by rafram 1y ago

> To simplify the migration process both for users and Zod's ecosystem of associated libraries, Zod 4 is being published alongside Zod 3 as part of the zod@3.25 release. [...] import Zod 4 from the "/v4" subpath npm is an absolute disaster of a dependency management system. Peer dependencies are so broken that they had to make v4 pretend it's v3.

fhd2 1y ago
I feel like both Node.js and NPM were initially vibe coded before LLMs existed. Just a quick hack, kind of, that got hugely popular somewhat by accident. Edit: Thinking about it, that's the origin story of JavaScript as well, so rather fitting.
derN3rd 1y ago
We've been using zod 4 beta already with great improvements but due to our huge codebase not being able to handle the required moduleResolution settings, we cannot upgrade... They could at least also publish it as a major version without the legacy layer EDIT: I've just seen the reason described here: https://github.com/colinhacks/zod/issues/4371 https://github.com/colinhacks/zod/issues/4371 TLDR: He doesn't want to trigger a "version bump avalanche" across the ecosystem. (Which I believe, wouldn't happen as they could still backport fixes and support the v3 for a time, as they do it right now)
- sroussey 1y ago
  Would not be hard for you or anyone else to do that as well.
- 827a 1y ago
  Yeah this seems like very poor reasoning tbh.
gejose 1y ago
> Peer dependencies are so broken that they had to make v4 pretend it's v3 I'm not sure this is the right conclusion here. I think zod v4 is being included within v3 so consumers can migrate over incrementally. I.e refactor all usages, one by one to `import ... from 'zod/v4'`, and once that's done, upgrade to v4 entirely.
- skydhash 1y ago
  How do you even migrate incrementally? why keep both old and new code together like a Frankeinstein project? Especially on a codebase you own. It's a library, not a platform.
  - rafram 1y ago
    Yeah, importing two versions could allow you to pass v3 objects to v4 methods and vice versa, and that seems like an extremely bad idea (if it would even type-check / run).
    - colinmcd 1y ago
      It won't typecheck, which is good in this case, because as you say that's a very bad idea :)
  - diggan 1y ago
    > why keep both old and new code together like a Frankeinstein project? It seems like Zod is a library that provides schema to your (domain) objects in JS/TS projects, so if you're all-in with this library, it's probably a base-layer for a ton of stuff in the entire codebase. Now imagine that the codebase is worked on by multiple parties, in different areas, and you're dealing with 100K-1M lines of code. Realistically, you can't coordinate such a huge change with just one "Migrated to Zod 4" commit and call it a day, and you most likely want to do it in pieces. I'm not convinced publishing V4 as version 3 on npm is the best way of achieving this, but to make that process easier seems to be the goal of that as far as I understand.
    - skydhash 1y ago
      If it's part of your core domain, I'd say that one of the reasons you want the feature set and the API to be as stable as possible. And your core domain should probably be isolated from the things that depends on it in such way that only business logic ripples out. One of the strange things from the NPM/JS culture is the focus to write everything in one language leading to everyone forgetting about modularization and principles like SOLID. We link everything together with strange mechanisms for "DX", which really means "make it easy to write the first version, maintenance be damned".
herrkanin 1y ago
I might be blindsided by using npm exclusively for years by this point, but what would be a better way to support iteratively migrating from v3 to v4 without having to do it all in one large batch?
- strken 1y ago
  Using npm's built in support for package aliases to simultaneously install zod@3 as zod and zod@4 as zod-next? Edit: reading the rationale, it's about peer dependencies rather than direct dependencies. I am still a little confused.
  - colinmcd 1y ago
    As you allude to: your aliased "zod-next" dependency wouldn't be able to satisfy the requirements of any packages with a peer dep on Zod. But this approach has a more fundamental flaw. My goal is to let ecosystem libraries support Zod 3 and Zod 4 simultaneously. There's no reliable way to do that if they aren't in the same package.[0] [0] https://github.com/colinhacks/zod/issues/4371 https://github.com/colinhacks/zod/issues/4371
- diggan 1y ago
  One possible solution: Publish a new package with a new name. I've personally been thinking of doing that for some libraries, where I'd commit to never change the public API, and if I figure out a nicer way of doing something, I'd create a new library for it instead, and people who wanna use/maintain the old API can continue to do so.
  - Sammi 1y ago
    People wouldn't change to the new one, because names are extremely sticky, so what's the point?
    - skydhash 1y ago
      If they need the new one, they will switch to it. And you have the right to drop support for the old one after a while (hopefully giving everyone the time to migrate).
swyx 1y ago
apart from the other things people raised, as leerob also points out it hints to an llm that the api may be different https://x.com/leerob/status/1924504939818008791 https://x.com/leerob/status/1924504939818008791
Trufa 1y ago
I feel like people will upvote anything negative. This is not much a limitation of npm, there's nothing intrinsically wrong with npm that lead to this decision, this is more a pragmatic way of allowing progressive change of a library that introduced a lot of braking changes.
- danenania 1y ago
  Right, you have the same issues to consider when shipping a breaking major version upgrade to a new library in any language/ecosystem. That said, you do see a cultural difference in node-land vs. many other ecosystems where library maintainers are much quicker to go for the new major version vs. iterating on the existing version and maintaining backward compatibility. I think that's what people are mostly really referring to when they complain about node/npm. Webpack is a good example—what's it on now, version 5? If it was a Go project, it would probably still be on v1 or maybe v2. While the API arguably does get 'better' with each major version, in practice a lot of the changes are cosmetic and unnecessary, and that's what frustrates people I think. Very few people really care how nice the API of their bundler is. They just want it to work, and to receive security updates etc. Any time you spend on upgrading and converting APIs is basically a waste—it's not adding value for users of your software.
  - skydhash 1y ago
    This very much. If I'm using your library, I've already committed to it's architecture and API with all its flaws. And my users don't care for the technology I use. Even if they're not that good, I can build wrapper over the worst aspects and then just focus on building features. New features are nice, but I'm more interested in getting security updates for the current version that I have. When it's time to go for a refactoring, the trade-off between costs and returns are worth it as you can go for years between those huge refactors.
colinmcd 1y ago
Author here. I wrote a fairly detailed writeup here[0] for those who are interested in the reasons for this approach. Ultimately you're right that npm doesn't work well to manage the situation Zod finds itself in. But Zod is subject to a bunch of constraints that virtually no other libraries are subject to. There are dozens or hundreds of libraries that directly import interfaces/classes from "zod" and use them in their own public-facing API. Since these libraries are directly coupled to Zod, they would need to publish a new major version whenever Zod does. That's ultimately reasonable in isolation, but in Zod's case it would trigger a "version avalanche" would just be painful for everyone involved. Selfishly, I suspect it would result in a huge swath of the ecosystem pinning on v3 forever. The approach I ended up using is analogous to what Golang does. In essence a given package never publishes new breaking versions: they just add a new subpath when a new breaking release is made. In the TypeScript ecosystem, this means libraries can configure a single peer dependency on zod@^3.25.0 and support both versions simultaneously by importing what they need from "zod/v3" and "zod/v4". It provides a nice opt-in incremental upgrade path for end-users of Zod too. [0] https://github.com/colinhacks/zod/issues/4371 https://github.com/colinhacks/zod/issues/4371
- codeflo 1y ago
  Is there any advantage to this approach over publishing a separate "zod4" package? That would be just as opt-in and incremental, not bloat download sizes forever, and make it easier to not accidentally import the wrong thing.
  - colinmcd 1y ago
    Ecosystem libraries would need to switch from a single peer dependency on Zod to two optional peer dependencies. Despite "optional peer dependencies" technicall being a thing, its functionally impossible for a library to determine which of the two packages your user is actually bringing to the table. Let's say a library is trying to implement an `acceptSchema` function that can accepts `Zod3Type | Zod4Type`. For starters: those two interfaces won't both be available if Zod 3 and Zod 4 are in separate packages. So that's already a non-starter. And differentiating them at runtime requires knowing which package is installed, which is impossible in the general case (mostly because frontend bundlers generally have no affordance for optional peer dependencies). I describe this in more detail here: https://x.com/colinhacks/status/1922101292849410256 https://x.com/colinhacks/status/1922101292849410256
bilalq 1y ago
This isn't an npm exclusive issue. A dependency having some other transitive deps also depend on an older version is a problem that happens in literally every other ecosystem. If anything, npm gives you more escape hatches by actually allowing you to run multiple versions concurrently if you need to or selectively overriding parts of your transitive dependency graph. What package management system has a solution to this? Even so called "stable" platforms like Maven deal with this nonsense by publishing new versions under a new namespace (like Apache Commons did from v2 to v3).
aiiizzz 1y ago
Horrible, not going to use it now. My ide will import whichever interchangeably, and it'll be a mess

by rafram 1y ago

fhd2 1y ago

I feel like both Node.js and NPM were initially vibe coded before LLMs existed. Just a quick hack, kind of, that got hugely popular somewhat by accident. Edit: Thinking about it, that's the origin story of JavaScript as well, so rather fitting.

derN3rd 1y ago

We've been using zod 4 beta already with great improvements but due to our huge codebase not being able to handle the required moduleResolution settings, we cannot upgrade... They could at least also publish it as a major version without the legacy layer EDIT: I've just seen the reason described here: https://github.com/colinhacks/zod/issues/4371 https://github.com/colinhacks/zod/issues/4371 TLDR: He doesn't want to trigger a "version bump avalanche" across the ecosystem. (Which I believe, wouldn't happen as they could still backport fixes and support the v3 for a time, as they do it right now)

sroussey 1y ago
Would not be hard for you or anyone else to do that as well.
827a 1y ago
Yeah this seems like very poor reasoning tbh.

gejose 1y ago

> Peer dependencies are so broken that they had to make v4 pretend it's v3 I'm not sure this is the right conclusion here. I think zod v4 is being included within v3 so consumers can migrate over incrementally. I.e refactor all usages, one by one to `import ... from 'zod/v4'`, and once that's done, upgrade to v4 entirely.

skydhash 1y ago
How do you even migrate incrementally? why keep both old and new code together like a Frankeinstein project? Especially on a codebase you own. It's a library, not a platform.
- rafram 1y ago
  Yeah, importing two versions could allow you to pass v3 objects to v4 methods and vice versa, and that seems like an extremely bad idea (if it would even type-check / run).
  - colinmcd 1y ago
    It won't typecheck, which is good in this case, because as you say that's a very bad idea :)
- diggan 1y ago
  > why keep both old and new code together like a Frankeinstein project? It seems like Zod is a library that provides schema to your (domain) objects in JS/TS projects, so if you're all-in with this library, it's probably a base-layer for a ton of stuff in the entire codebase. Now imagine that the codebase is worked on by multiple parties, in different areas, and you're dealing with 100K-1M lines of code. Realistically, you can't coordinate such a huge change with just one "Migrated to Zod 4" commit and call it a day, and you most likely want to do it in pieces. I'm not convinced publishing V4 as version 3 on npm is the best way of achieving this, but to make that process easier seems to be the goal of that as far as I understand.
  - skydhash 1y ago
    If it's part of your core domain, I'd say that one of the reasons you want the feature set and the API to be as stable as possible. And your core domain should probably be isolated from the things that depends on it in such way that only business logic ripples out. One of the strange things from the NPM/JS culture is the focus to write everything in one language leading to everyone forgetting about modularization and principles like SOLID. We link everything together with strange mechanisms for "DX", which really means "make it easy to write the first version, maintenance be damned".

herrkanin 1y ago

I might be blindsided by using npm exclusively for years by this point, but what would be a better way to support iteratively migrating from v3 to v4 without having to do it all in one large batch?

strken 1y ago
Using npm's built in support for package aliases to simultaneously install zod@3 as zod and zod@4 as zod-next? Edit: reading the rationale, it's about peer dependencies rather than direct dependencies. I am still a little confused.
- colinmcd 1y ago
  As you allude to: your aliased "zod-next" dependency wouldn't be able to satisfy the requirements of any packages with a peer dep on Zod. But this approach has a more fundamental flaw. My goal is to let ecosystem libraries support Zod 3 and Zod 4 simultaneously. There's no reliable way to do that if they aren't in the same package.[0] [0] https://github.com/colinhacks/zod/issues/4371 https://github.com/colinhacks/zod/issues/4371
diggan 1y ago
One possible solution: Publish a new package with a new name. I've personally been thinking of doing that for some libraries, where I'd commit to never change the public API, and if I figure out a nicer way of doing something, I'd create a new library for it instead, and people who wanna use/maintain the old API can continue to do so.
- Sammi 1y ago
  People wouldn't change to the new one, because names are extremely sticky, so what's the point?
  - skydhash 1y ago
    If they need the new one, they will switch to it. And you have the right to drop support for the old one after a while (hopefully giving everyone the time to migrate).

swyx 1y ago

apart from the other things people raised, as leerob also points out it hints to an llm that the api may be different https://x.com/leerob/status/1924504939818008791 https://x.com/leerob/status/1924504939818008791

Trufa 1y ago

I feel like people will upvote anything negative. This is not much a limitation of npm, there's nothing intrinsically wrong with npm that lead to this decision, this is more a pragmatic way of allowing progressive change of a library that introduced a lot of braking changes.

danenania 1y ago
Right, you have the same issues to consider when shipping a breaking major version upgrade to a new library in any language/ecosystem. That said, you do see a cultural difference in node-land vs. many other ecosystems where library maintainers are much quicker to go for the new major version vs. iterating on the existing version and maintaining backward compatibility. I think that's what people are mostly really referring to when they complain about node/npm. Webpack is a good example—what's it on now, version 5? If it was a Go project, it would probably still be on v1 or maybe v2. While the API arguably does get 'better' with each major version, in practice a lot of the changes are cosmetic and unnecessary, and that's what frustrates people I think. Very few people really care how nice the API of their bundler is. They just want it to work, and to receive security updates etc. Any time you spend on upgrading and converting APIs is basically a waste—it's not adding value for users of your software.
- skydhash 1y ago
  This very much. If I'm using your library, I've already committed to it's architecture and API with all its flaws. And my users don't care for the technology I use. Even if they're not that good, I can build wrapper over the worst aspects and then just focus on building features. New features are nice, but I'm more interested in getting security updates for the current version that I have. When it's time to go for a refactoring, the trade-off between costs and returns are worth it as you can go for years between those huge refactors.

colinmcd 1y ago

Author here. I wrote a fairly detailed writeup here[0] for those who are interested in the reasons for this approach. Ultimately you're right that npm doesn't work well to manage the situation Zod finds itself in. But Zod is subject to a bunch of constraints that virtually no other libraries are subject to. There are dozens or hundreds of libraries that directly import interfaces/classes from "zod" and use them in their own public-facing API. Since these libraries are directly coupled to Zod, they would need to publish a new major version whenever Zod does. That's ultimately reasonable in isolation, but in Zod's case it would trigger a "version avalanche" would just be painful for everyone involved. Selfishly, I suspect it would result in a huge swath of the ecosystem pinning on v3 forever. The approach I ended up using is analogous to what Golang does. In essence a given package never publishes new breaking versions: they just add a new subpath when a new breaking release is made. In the TypeScript ecosystem, this means libraries can configure a single peer dependency on zod@^3.25.0 and support both versions simultaneously by importing what they need from "zod/v3" and "zod/v4". It provides a nice opt-in incremental upgrade path for end-users of Zod too. [0] https://github.com/colinhacks/zod/issues/4371 https://github.com/colinhacks/zod/issues/4371

codeflo 1y ago
Is there any advantage to this approach over publishing a separate "zod4" package? That would be just as opt-in and incremental, not bloat download sizes forever, and make it easier to not accidentally import the wrong thing.
- colinmcd 1y ago
  Ecosystem libraries would need to switch from a single peer dependency on Zod to two optional peer dependencies. Despite "optional peer dependencies" technicall being a thing, its functionally impossible for a library to determine which of the two packages your user is actually bringing to the table. Let's say a library is trying to implement an `acceptSchema` function that can accepts `Zod3Type | Zod4Type`. For starters: those two interfaces won't both be available if Zod 3 and Zod 4 are in separate packages. So that's already a non-starter. And differentiating them at runtime requires knowing which package is installed, which is impossible in the general case (mostly because frontend bundlers generally have no affordance for optional peer dependencies). I describe this in more detail here: https://x.com/colinhacks/status/1922101292849410256 https://x.com/colinhacks/status/1922101292849410256

bilalq 1y ago

This isn't an npm exclusive issue. A dependency having some other transitive deps also depend on an older version is a problem that happens in literally every other ecosystem. If anything, npm gives you more escape hatches by actually allowing you to run multiple versions concurrently if you need to or selectively overriding parts of your transitive dependency graph. What package management system has a solution to this? Even so called "stable" platforms like Maven deal with this nonsense by publishing new versions under a new namespace (like Apache Commons did from v2 to v3).

aiiizzz 1y ago

Horrible, not going to use it now. My ide will import whichever interchangeably, and it'll be a mess