4 ms·

I'm curious if anyone here can answer a question I've wondered about for a long time. I've heard Zod might be in the right ballpark, but from reading the docume

by johnfn 1y ago

I'm curious if anyone here can answer a question I've wondered about for a long time. I've heard Zod might be in the right ballpark, but from reading the documentation, I'm not sure how I would go about it. Say I have a type returned by the server that might have more sophisticated types than the server API can represent. For instance, api/:postid/author returns a User, but it could either be a normal User or an anonymous User, in which case fields like `username`, `location`, etc come back null. So in this case I might want to use a discriminated union to represent my User object. And other objects coming back from other endpoints might also need some type alterations done to them as well. For instance, a User might sometimes have Post[] on them, and if the Post is from a moderator, it might have special attributes, etc - another discriminated union. In the past, I've written functions like normalizeUser() and normalizePost() to solve this, but this quickly becomes really messy. Since different endpoints return different subsets of the User/Post model, I would end up writing like 5 different versions of normalizePost for each endpoint, which seems like a mess. How do people solve this problem?

peab 1y ago
Either Unions, or optional fields
adpirz 1y ago
It's hard to unpack without knowing more about the use case, but adding discriminant properties (e.g. "user_type") to all the types in the union can make it easier to handle the general and specific case. E.g. if (user.user_type === 'authenticated') { // do something with user.name because the type system knows we have that now }
probabletrain 1y ago
In an ideal world you'd have one source of truth for what the shape of a User could be (which may well be a discriminated union of User and AnonymousUser or similar). Without fullstack TS this could look something like: (for a Python backend) Pydantic models+union for the various shapes of `User`, and then OpenAPI/GraphQL schema generation+codegen for the TS client.
- johnfn 1y ago
  The problem with this is that your One True User shape tends to have a bunch of optional properties on it. e.g., in the user profile you fetch Post[], but in the user directory you don't, and so on with other joined properties. If every endpoint returns the One True User, then you end up needing to write conditional logic to guard against (say) `.posts` when you fetch users in the profile, even though you know that `.posts` exists.
  - iainmerrick 1y ago
    In Typescript at least, if the discriminated union is set up correctly, you just need a single check of the type field. That lets TS narrow the type so it knows whether e.g. `.posts` is present or not.
    - johnfn 1y ago
      I think I can do this, yes, but it becomes a very large amount of repetitive work. Let's say I have api/profile (which has `.posts`) and api/user-directory (which does not). I define User as a discriminated union - one has type `user-with-posts` and one has type `user-no-posts`. OK, good so far. But now say I have a photo gallery, which returns photos on user. Now I have to add that into my discriminated union type, e.g. `user-with-posts-and-photos`, `user-with-posts-but-not-photos`, `user-with-no-posts-but-yes-photos`... and it gets worse if I also have another page with DMs...
      - joshfee 1y ago
        You can use a string union to discriminate when it makes sense, but that's not the only way to discriminate and in this case you'd instead use the presence of the items themselves (essentially duck-typing with strong type guarantees) Typescript playground: https://www.typescriptlang.org/play/?#code/C4TwDgpgBACg9gZ2FAvFA3lAthBCCGA5hAFxRIBOAlgHaFQC+AUC6JFAOpXAAW8SCVBiZRRUMImAIy-YAG0AugBoRYiUgDCcAK41gZGtqwAjCBSbMmbaF14wecYIiHpVosA6fTywanUUWrODQALL4IKYAPAAqAHxC0VAAPhiW1lAAqghmLlBUACZklLT0DFAAZFBhERCRtnySCPGV1VH19o6IsSxMAGa6AMbAVHA0UPlwAMpwOLwl9VlmABTa2RRkixQAlMJieb1QSwDk6lJHeWOrZjuue3sA9PdQA-hj+AMDuIKngsbayDRHOJPIg3HcrhQAHQ-SG9OAUACi7x4S1OqHiA1GCDgABsIJCcXBCKjJFstiw7mJHlAIAA3CBjXg6Qg8KAAd2goxxIGePAgAwA1hB8lA4RQoAADH4SpRQXjQdJs-CCGj4CgUOAckXY9nQF5vD5fKD4HE4qBwA7yqAUCA4-DAYXiDWQCjDXByuBwMF7TE0bF4glElZraGSLS6YDksGWPZUA7HDydBDnWhQCE3b1Up7642GvDApNQP4AoE-TOiCHQkEIWHwpEDFGJpzo55Y3H4wnEptwMnRiliOOHE6NFOXNYVSonaujtNrDOU0TUnPvT75n7Gmgi7sIcuzsyhgS1xHIklIFu+-0doOnXsLvdQ7dH+uNkHntsBzuokG3sTMBhAA https://www.typescriptlang.org/play/?#code/C4TwDgpgBACg9gZ2F...
stephen_hn 1y ago
I think you would use discrimated unions. const MyResult = z.discriminatedUnion("status", [ z.object({ status: z.literal("success"), data: z.string() }), z.object({ status: z.literal("failed"), error: z.string() }), ]); You can define passthrough behavior if there are a bunch of special attributes for a moderator but you don't want to list/check them all. With different methods that have different schema- If they share part of the same schema with alterations, you can define an object for the shared part and create objects that contain the shared object schema with additional fields. If you have a lot of different possibilities, it will be messy, but it sounds like it already is for you, so validators could still at least validate the messines.
- johnfn 1y ago
  Hmm, this is pretty interesting, but I still have issues. While Zod seems to be able to transform my server-returned object into a discriminated union, it doesn't seem to be able to tell which of the unions it is. For instance, say that api/user/mod-panel (e.g.) returns only moderator user objects. My API endpoint already knows that it will have moderator-related fields, but Zod forgets that when I parse it.
  - stephen_hn 1y ago
    I am not totally sure what your API's look like but if you know the API only has a certain type (or subset of types), you would only validate that type on the method. Don't use a single union for everything. Here is some pseudocode. Person = { name: string, height: number } Animal = {name: string, capability: string} A = { post: object, methodType: string, person: Person } ModeratorA = { post: object, moderatorField1: string, moderatorField2: string, person: Person } UnionA = A && ModeratorA (There's probably a better way of defining A and ModeratorA to share the shared fields) B = { post: object, animal: Animal } endpoint person parses UnionA endpoint animal parses B You don't put all of your types in one big Union.
  - vjerancrnjak 1y ago
    You can use an ‘is’ method to pick out a type from union. Although, if union is discriminated, it’s discriminated with a ‘kind’ so you should be able to know which kind you received.
gavinray 1y ago
This doesn't help in your case, but this is what GraphQL was invented for. TS libraries for GQL queries can dynamically infer the response shape from the shape of the selected fields. Such that the query query { user { username posts { text } } } Would be type Response = { user: { username: string posts: Array<{ text: string }> } } And a query with just { user { username } } would have the posts property omitted entirely.
mmcnl 1y ago
Your server should export the types. Don't write types by hand in your client, that makes no sense. The server knows what data it is sending and should provide that information as metadata to the client. Practically this can mean that a Python backend uses Pydantic schemas which can be used to automatically generate an OpenAPI specification that your client can use to generate types.
- johnfn 1y ago
  This answer is how I wish the world would work, but I am stuck with a server with a poor type system that can't refine the types nearly as accurately as TS can. :(

by johnfn 1y ago

peab 1y ago

Either Unions, or optional fields

adpirz 1y ago

It's hard to unpack without knowing more about the use case, but adding discriminant properties (e.g. "user_type") to all the types in the union can make it easier to handle the general and specific case. E.g. if (user.user_type === 'authenticated') { // do something with user.name because the type system knows we have that now }

probabletrain 1y ago

In an ideal world you'd have one source of truth for what the shape of a User could be (which may well be a discriminated union of User and AnonymousUser or similar). Without fullstack TS this could look something like: (for a Python backend) Pydantic models+union for the various shapes of `User`, and then OpenAPI/GraphQL schema generation+codegen for the TS client.

johnfn 1y ago
The problem with this is that your One True User shape tends to have a bunch of optional properties on it. e.g., in the user profile you fetch Post[], but in the user directory you don't, and so on with other joined properties. If every endpoint returns the One True User, then you end up needing to write conditional logic to guard against (say) `.posts` when you fetch users in the profile, even though you know that `.posts` exists.
- iainmerrick 1y ago
  In Typescript at least, if the discriminated union is set up correctly, you just need a single check of the type field. That lets TS narrow the type so it knows whether e.g. `.posts` is present or not.
  - johnfn 1y ago
    I think I can do this, yes, but it becomes a very large amount of repetitive work. Let's say I have api/profile (which has `.posts`) and api/user-directory (which does not). I define User as a discriminated union - one has type `user-with-posts` and one has type `user-no-posts`. OK, good so far. But now say I have a photo gallery, which returns photos on user. Now I have to add that into my discriminated union type, e.g. `user-with-posts-and-photos`, `user-with-posts-but-not-photos`, `user-with-no-posts-but-yes-photos`... and it gets worse if I also have another page with DMs...
    - joshfee 1y ago
      You can use a string union to discriminate when it makes sense, but that's not the only way to discriminate and in this case you'd instead use the presence of the items themselves (essentially duck-typing with strong type guarantees) Typescript playground: https://www.typescriptlang.org/play/?#code/C4TwDgpgBACg9gZ2FAvFA3lAthBCCGA5hAFxRIBOAlgHaFQC+AUC6JFAOpXAAW8SCVBiZRRUMImAIy-YAG0AugBoRYiUgDCcAK41gZGtqwAjCBSbMmbaF14wecYIiHpVosA6fTywanUUWrODQALL4IKYAPAAqAHxC0VAAPhiW1lAAqghmLlBUACZklLT0DFAAZFBhERCRtnySCPGV1VH19o6IsSxMAGa6AMbAVHA0UPlwAMpwOLwl9VlmABTa2RRkixQAlMJieb1QSwDk6lJHeWOrZjuue3sA9PdQA-hj+AMDuIKngsbayDRHOJPIg3HcrhQAHQ-SG9OAUACi7x4S1OqHiA1GCDgABsIJCcXBCKjJFstiw7mJHlAIAA3CBjXg6Qg8KAAd2goxxIGePAgAwA1hB8lA4RQoAADH4SpRQXjQdJs-CCGj4CgUOAckXY9nQF5vD5fKD4HE4qBwA7yqAUCA4-DAYXiDWQCjDXByuBwMF7TE0bF4glElZraGSLS6YDksGWPZUA7HDydBDnWhQCE3b1Up7642GvDApNQP4AoE-TOiCHQkEIWHwpEDFGJpzo55Y3H4wnEptwMnRiliOOHE6NFOXNYVSonaujtNrDOU0TUnPvT75n7Gmgi7sIcuzsyhgS1xHIklIFu+-0doOnXsLvdQ7dH+uNkHntsBzuokG3sTMBhAA https://www.typescriptlang.org/play/?#code/C4TwDgpgBACg9gZ2F...

stephen_hn 1y ago

I think you would use discrimated unions. const MyResult = z.discriminatedUnion("status", [ z.object({ status: z.literal("success"), data: z.string() }), z.object({ status: z.literal("failed"), error: z.string() }), ]); You can define passthrough behavior if there are a bunch of special attributes for a moderator but you don't want to list/check them all. With different methods that have different schema- If they share part of the same schema with alterations, you can define an object for the shared part and create objects that contain the shared object schema with additional fields. If you have a lot of different possibilities, it will be messy, but it sounds like it already is for you, so validators could still at least validate the messines.

johnfn 1y ago
Hmm, this is pretty interesting, but I still have issues. While Zod seems to be able to transform my server-returned object into a discriminated union, it doesn't seem to be able to tell which of the unions it is. For instance, say that api/user/mod-panel (e.g.) returns only moderator user objects. My API endpoint already knows that it will have moderator-related fields, but Zod forgets that when I parse it.
- stephen_hn 1y ago
  I am not totally sure what your API's look like but if you know the API only has a certain type (or subset of types), you would only validate that type on the method. Don't use a single union for everything. Here is some pseudocode. Person = { name: string, height: number } Animal = {name: string, capability: string} A = { post: object, methodType: string, person: Person } ModeratorA = { post: object, moderatorField1: string, moderatorField2: string, person: Person } UnionA = A && ModeratorA (There's probably a better way of defining A and ModeratorA to share the shared fields) B = { post: object, animal: Animal } endpoint person parses UnionA endpoint animal parses B You don't put all of your types in one big Union.
- vjerancrnjak 1y ago
  You can use an ‘is’ method to pick out a type from union. Although, if union is discriminated, it’s discriminated with a ‘kind’ so you should be able to know which kind you received.

gavinray 1y ago

This doesn't help in your case, but this is what GraphQL was invented for. TS libraries for GQL queries can dynamically infer the response shape from the shape of the selected fields. Such that the query query { user { username posts { text } } } Would be type Response = { user: { username: string posts: Array<{ text: string }> } } And a query with just { user { username } } would have the posts property omitted entirely.

mmcnl 1y ago

Your server should export the types. Don't write types by hand in your client, that makes no sense. The server knows what data it is sending and should provide that information as metadata to the client. Practically this can mean that a Python backend uses Pydantic schemas which can be used to automatically generate an OpenAPI specification that your client can use to generate types.

johnfn 1y ago
This answer is how I wish the world would work, but I am stuck with a server with a poor type system that can't refine the types nearly as accurately as TS can. :(