5 ms·

This is obviously good advice almost all of the time. However, I have had to deal occasionally with http libraries that tried to parse everything and would not

This is obviously good advice almost all of the time. However, I have had to deal occasionally with http libraries that tried to parse everything and would not give you access to anything that they could not parse. This was incredibly frustrating for corner cases that the library authors hadn't considered. If you are the one who is going to take action on the data, parse don't validate is the correct approach. If you are writing a library that deals with data that it doesn't fully understand, and you're handing that data to someone else to take action with, then it may not always be the right approach.

tizzy 4y ago
This seems like good library design. As annoying as it is, it means the things you can use are well tested and supported. What was your solution to this? Parse the things the library didn't?
- kybernetikos 4y ago
  The library didn't allow you to see the things (e.g. particular headers or options for those headers) that it didn't know to parse. Ultimately we had to migrate to a different library that didn't restrict us to just what the library knew. The decision not to let us even see things that the library didn't know about is particularly egregious where best practices are changing over time. In my view it's a very bad design for an http library, although it would have been a lot less frustrating if it had at least provided an escape hatch.
  - jimbokun 4y ago
    Sounds like its model is not the HTTP RFC, but something more specific to some domain. Which I agree, is a poor design choice. A type modeling an HTTP request should model the RFC definition as closely as possible.
    - aidenn0 4y ago
      > Which I agree, is a poor design choice. A type modeling an HTTP request should model the RFC definition as closely as possible. I couldn't disagree more. A type modeling an HTTP request should model HTTP requests. Not some theoretical description of an HTTP request.
      - recursive 4y ago
        HTTP requests are not things that humans discovered in nature. They are abstractions, created entirely by specification. In some sense, an HTTP request is exactly that which conforms to the specification.
        aidenn0 4y ago
        To an extent, that sounds like saying the thing I am sitting in is not a chair since it has 5 legs.
        recursive 4y ago
        I mean, if chairs were things with formal specifications, and that specification said so, yeah. But in this universe, no.
        girvo 4y ago
        If you can completely ignore HTTP request data that happens to not perfectly meet the RFC at your work (or, more specifically for us, Modbus RTU responses), I salute you. Sadly, I can’t, we get some wild stuff that we still need to attempt to handle. Both HTTP and Modbus!
        kybernetikos 4y ago
        You don't have to run an HTTP server very long on the internet to start discovering HTTP requests (including malformed HTTP requests, which are also a kind of HTTP request), 'in nature'.
        4y ago
  - ParetoOptimal 4y ago
    > The library didn't allow you to see the things (e.g. particular headers or options for those headers) that it didn't know to parse. I typically design code around things like this with a sum type like: data Header = KnownHeader1 | KnownHeader2 | UnknownHeader String String Then I typically don't offer any extra support or extended functionality for the cases where the type is `UnknownHeader`.
lexi-lambda 4y ago
I discussed how/why the point of this article is very much not to “parse everything” in this followup: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-type-systems-are-not-inherently-more-open/ https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ... In particular, it articulates precisely why it is fine to use relatively wide types for any input which the program treats opaquely, and it gives an example of how the same techniques can still be useful even in that context.

tizzy 4y ago

This seems like good library design. As annoying as it is, it means the things you can use are well tested and supported. What was your solution to this? Parse the things the library didn't?

kybernetikos 4y ago
The library didn't allow you to see the things (e.g. particular headers or options for those headers) that it didn't know to parse. Ultimately we had to migrate to a different library that didn't restrict us to just what the library knew. The decision not to let us even see things that the library didn't know about is particularly egregious where best practices are changing over time. In my view it's a very bad design for an http library, although it would have been a lot less frustrating if it had at least provided an escape hatch.
- jimbokun 4y ago
  Sounds like its model is not the HTTP RFC, but something more specific to some domain. Which I agree, is a poor design choice. A type modeling an HTTP request should model the RFC definition as closely as possible.
  - aidenn0 4y ago
    > Which I agree, is a poor design choice. A type modeling an HTTP request should model the RFC definition as closely as possible. I couldn't disagree more. A type modeling an HTTP request should model HTTP requests. Not some theoretical description of an HTTP request.
    - recursive 4y ago
      HTTP requests are not things that humans discovered in nature. They are abstractions, created entirely by specification. In some sense, an HTTP request is exactly that which conforms to the specification.
      - aidenn0 4y ago
        To an extent, that sounds like saying the thing I am sitting in is not a chair since it has 5 legs.
        recursive 4y ago
        I mean, if chairs were things with formal specifications, and that specification said so, yeah. But in this universe, no.
        girvo 4y ago
        If you can completely ignore HTTP request data that happens to not perfectly meet the RFC at your work (or, more specifically for us, Modbus RTU responses), I salute you. Sadly, I can’t, we get some wild stuff that we still need to attempt to handle. Both HTTP and Modbus!
      - kybernetikos 4y ago
        You don't have to run an HTTP server very long on the internet to start discovering HTTP requests (including malformed HTTP requests, which are also a kind of HTTP request), 'in nature'.
      - 4y ago
- ParetoOptimal 4y ago
  > The library didn't allow you to see the things (e.g. particular headers or options for those headers) that it didn't know to parse. I typically design code around things like this with a sum type like: data Header = KnownHeader1 | KnownHeader2 | UnknownHeader String String Then I typically don't offer any extra support or extended functionality for the cases where the type is `UnknownHeader`.

lexi-lambda 4y ago

I discussed how/why the point of this article is very much not to “parse everything” in this followup: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-type-systems-are-not-inherently-more-open/ https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ... In particular, it articulates precisely why it is fine to use relatively wide types for any input which the program treats opaquely, and it gives an example of how the same techniques can still be useful even in that context.