4 ms·

Author here. One thing that we do at Svix that I alluded to in one paragraph but I should probably have elaborated on further: thanks to libraries like Serde an

by tasn 3y ago

Author here. One thing that we do at Svix that I alluded to in one paragraph but I should probably have elaborated on further: thanks to libraries like Serde and Pydantic, we actually follow deserialization is validation (is that a term?), which means that we validate all of the JSON data before even creating the structures in our code. I guess that's similar to the redis example I gave, but it essentially means that even though we get sent JSON over the wire, we validate it fully and when it gets to our code we know it's a well formatted type. So our code can assume an email type is a valid email, an ID type is a valid ID, etc.

taeric 3y ago
Right, but you should also take pains to keep the original. All too often the deserialization/validation can also alter the data. In many places, this is fine, I think. In many others, it leads to a ton of confusion. Edit: I wrote "but you should", I really should have worded that as "and you should". I am wanting to add to your point, not contradict it.
- tasn 3y ago
  Oh yeah, I agree, and I learned it the hard way. In many cases you want to keep the customer format rather than the canonical format. This is why I stopped using Postgres's JSONB for customer data, and exclusively use String/JSON nowadays.
davedx 3y ago
It’s absolutely a term, and the way to go
williamdclt 3y ago
> deserialization is validation (is that a term?) “Parse, don’t validate » is a common Googleable idiom
AdamN 3y ago
What I like that you're doing and what I used to do with Python annotations is that you strongly define types for your higher level business logic - which may be encode quite complex formats. In many ways, your point is correct and kind of difficult to argue against (i.e. I agree!). What catches alot of people is they say how important types are but then just use 3 or 4 primitive types and never build custom types on top of them so they don't get much value out of those types aside from preventing the absolute worst runtime errors. The real value of types is having lots of them and evolving them over time as your systems and capabilities mature (US street address, US state, latitude, ... the list goes on for any piece of software).
- digging 3y ago
  Huh! That's given me a lot of insight into why I've had to argue about typing in the past. Thanks for encapsulating that for me. I've had teammates "agree" and use types only to use the most basic possible types and then confuse themselves later (like, days later). But to me, because I like static typing, it comes very naturally to define types based on business logic, which makes things very easy to work with. And they're just not thinking in that way. Now I wonder how I can bring them around.
manicennui 3y ago
This seems to be a common practice in Go and I've seen it cause requests to be rejected because one unimportant field is wrong. Were we to do this for truly important requests, like lead traffic from third parties, we would lose a lot of business.
- mcesch 3y ago
  If the field is unimportant you'd represent it as such. `Option` or `Maybe` or whatever the equivalent wrapper is in your language of choice.
cbdumas 3y ago
> we actually follow deserialization is validation (is that a term?) Perhaps you are thinking of this great blog post on the subject https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
lucasyvas 3y ago
This is correct - if it's going to error or panic, it should do it at the boundary, not at a random place in the program. It's also much faster to debug where the issue is this way.
funnymony 3y ago
> deserialization is validation I saw similar, catchy phrase: “parse, don’t validate”. https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
DanielHB 3y ago
In my opinion this is the single biggest benefit of GraphQL, forced validation of data server-side. Sure you can do that with swagger in REST apis as well but nobody ever bothers to do it strictly and correctly. Technically you can have more complex and custom validation (email is a valid email address) with annotations but the client-side will not have this information.

by tasn 3y ago

taeric 3y ago

Right, but you should also take pains to keep the original. All too often the deserialization/validation can also alter the data. In many places, this is fine, I think. In many others, it leads to a ton of confusion. Edit: I wrote "but you should", I really should have worded that as "and you should". I am wanting to add to your point, not contradict it.

tasn 3y ago
Oh yeah, I agree, and I learned it the hard way. In many cases you want to keep the customer format rather than the canonical format. This is why I stopped using Postgres's JSONB for customer data, and exclusively use String/JSON nowadays.

davedx 3y ago

It’s absolutely a term, and the way to go

williamdclt 3y ago

> deserialization is validation (is that a term?) “Parse, don’t validate » is a common Googleable idiom

AdamN 3y ago

What I like that you're doing and what I used to do with Python annotations is that you strongly define types for your higher level business logic - which may be encode quite complex formats. In many ways, your point is correct and kind of difficult to argue against (i.e. I agree!). What catches alot of people is they say how important types are but then just use 3 or 4 primitive types and never build custom types on top of them so they don't get much value out of those types aside from preventing the absolute worst runtime errors. The real value of types is having lots of them and evolving them over time as your systems and capabilities mature (US street address, US state, latitude, ... the list goes on for any piece of software).

digging 3y ago
Huh! That's given me a lot of insight into why I've had to argue about typing in the past. Thanks for encapsulating that for me. I've had teammates "agree" and use types only to use the most basic possible types and then confuse themselves later (like, days later). But to me, because I like static typing, it comes very naturally to define types based on business logic, which makes things very easy to work with. And they're just not thinking in that way. Now I wonder how I can bring them around.

manicennui 3y ago

This seems to be a common practice in Go and I've seen it cause requests to be rejected because one unimportant field is wrong. Were we to do this for truly important requests, like lead traffic from third parties, we would lose a lot of business.

mcesch 3y ago
If the field is unimportant you'd represent it as such. `Option` or `Maybe` or whatever the equivalent wrapper is in your language of choice.

cbdumas 3y ago

> we actually follow deserialization is validation (is that a term?) Perhaps you are thinking of this great blog post on the subject https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

lucasyvas 3y ago

This is correct - if it's going to error or panic, it should do it at the boundary, not at a random place in the program. It's also much faster to debug where the issue is this way.

funnymony 3y ago

> deserialization is validation I saw similar, catchy phrase: “parse, don’t validate”. https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

DanielHB 3y ago

In my opinion this is the single biggest benefit of GraphQL, forced validation of data server-side. Sure you can do that with swagger in REST apis as well but nobody ever bothers to do it strictly and correctly. Technically you can have more complex and custom validation (email is a valid email address) with annotations but the client-side will not have this information.