3 ms·

It's really hard to implement a HTTP client or server properly. It takes years to stabilize. That's why if you can use an existing solution. cURL is your friend

by zimbatm 13y ago

It's really hard to implement a HTTP client or server properly. It takes years to stabilize. That's why if you can use an existing solution. cURL is your friend.

zzzcpan 13y ago
I disagree. It's actually pretty easy. It might seem hard at first, but in reality there are just a few things one needs to know to support all clients correctly. Like disabling keepalive in some scenarios for some user-agents, choosing proper compression, padding small non-200 responses, etc. And you can look up all of those things in nginx or apache.
- MostAwesomeDude 13y ago
  As far as I can tell, nobody gets chunked encoding right the first time. Or the second time. The CVEs for chunked encoding are enlightening.
- zimbatm 13y ago
  That's exactly the kind of attitude I am trying to warn against. I have encountered times and times again people with that exact same attitude but I have yet to see HTTP implementation that don't get bug fixes years after their start. From my experience, even an implementation with limited scope will need at least a year to fully stabilize. Just point me to one and I'll be happy to revise my judgement. EDIT: As an example the http_parser in nodejs. It's just a small part of the whole but still had bug fixes a year after nodejs started. I don't have access to the repo right now (GH is down) but here is the source: https://github.com/joyent/http-parser https://github.com/joyent/http-parser
  - zzzcpan 13y ago
    No, you are still wrong. I have written http parsers a few times in different languages and probably can do it in my sleep. It was never hard. If you work on web servers a lot you tend to remember all the quirks, but if you don't you can always look into nginx. And you are probably thinking about different kinds of bugs when talking about stabilization. The reason it takes a long time for some implementations to stabilize is usually a poor language choice, not a variety of incorrect protocol implementations. Sure, there are some people who have no clue and try to make their own implementation without looking into nginx or apache, so your concerns are right, but they don't apply to everyone.
    - zimbatm 13y ago
      What kind of language are you using ? The most successful implementation of an HTTP parser that I have seen in terms of correctness/time-to-implement was done using Ragel for the mongrel ruby web server (see http://rubygems.org/gems/mongrel http://rubygems.org/gems/mongrel ). I have to look back again but if I remember well, the parser was written and validated with fuzzy testing in a week or so and then was pretty much stable. That was just for the parsing and the rest of the web server took a couple more months to stabilize.
      - zzzcpan 13y ago
        > What kind of language are you using ? Go, C, Perl and others.

by zimbatm 13y ago

It's really hard to implement a HTTP client or server properly. It takes years to stabilize. That's why if you can use an existing solution. cURL is your friend.

zzzcpan 13y ago

I disagree. It's actually pretty easy. It might seem hard at first, but in reality there are just a few things one needs to know to support all clients correctly. Like disabling keepalive in some scenarios for some user-agents, choosing proper compression, padding small non-200 responses, etc. And you can look up all of those things in nginx or apache.

MostAwesomeDude 13y ago
As far as I can tell, nobody gets chunked encoding right the first time. Or the second time. The CVEs for chunked encoding are enlightening.
zimbatm 13y ago
That's exactly the kind of attitude I am trying to warn against. I have encountered times and times again people with that exact same attitude but I have yet to see HTTP implementation that don't get bug fixes years after their start. From my experience, even an implementation with limited scope will need at least a year to fully stabilize. Just point me to one and I'll be happy to revise my judgement. EDIT: As an example the http_parser in nodejs. It's just a small part of the whole but still had bug fixes a year after nodejs started. I don't have access to the repo right now (GH is down) but here is the source: https://github.com/joyent/http-parser https://github.com/joyent/http-parser
- zzzcpan 13y ago
  No, you are still wrong. I have written http parsers a few times in different languages and probably can do it in my sleep. It was never hard. If you work on web servers a lot you tend to remember all the quirks, but if you don't you can always look into nginx. And you are probably thinking about different kinds of bugs when talking about stabilization. The reason it takes a long time for some implementations to stabilize is usually a poor language choice, not a variety of incorrect protocol implementations. Sure, there are some people who have no clue and try to make their own implementation without looking into nginx or apache, so your concerns are right, but they don't apply to everyone.
  - zimbatm 13y ago
    What kind of language are you using ? The most successful implementation of an HTTP parser that I have seen in terms of correctness/time-to-implement was done using Ragel for the mongrel ruby web server (see http://rubygems.org/gems/mongrel http://rubygems.org/gems/mongrel ). I have to look back again but if I remember well, the parser was written and validated with fuzzy testing in a week or so and then was pretty much stable. That was just for the parsing and the rest of the web server took a couple more months to stabilize.
    - zzzcpan 13y ago
      > What kind of language are you using ? Go, C, Perl and others.