4 ms·

I'm curious: what made cURL succeed? I remember wget being the hotness back in the day. When I encountered cURL I thought it was a variant of wget.

by wenc 9y ago

jacobush 9y ago
I think it was you could use it a library more easily. Curl is MIT licensed, so less to worry about if integrating in your code base. I also think it was designed as a library from the beginning, while wget was intended solely as a command line utility. Wget is also GPL licensed, so does not mix super well with proprietary programs or if you just don't want to "taint" your codebase with GPL for other reasons.
joering2 9y ago
It is just much easier to use in my opinion. I ran tons of cronjobs to update certain things. Few cool options I have with cURL that I find hard time locating in wget (doesn't mean they don't exist) --max-time (seconds) - for some reason I found wget not respecting always how much time it should be pulling the content. I think its related to fact curl and wget determine the moment of fetching file differently. --retry 0 - i did not find a way to tell wget not to retry pulling page. This was super important to my setup as we ran some pages for 6 hours to make sure the job is done. wget will gave up and retry after 15 minutes: HTTP request sent, awaiting response... Read error (Connection timed out) in headers. Retrying. Curl combined with solo for cronjobs is making pulling scripts on 32 servers very easy task and I hadn't had issues with it for 3 years now.
hemancuso 9y ago
I write ExpanDrive and when we moved from Python->C++ I used Microsoft's https://github.com/Microsoft/cpprestsdk https://github.com/Microsoft/cpprestsdk C++ REST SDK [Casablanca], because it is Microsoft, documented, modern, nice C++ etc. But for the latest iteration we switched everything over to libcurl. Libcurl has a bazillion edge cases handled. There are a million options, and it supports EVERYTHING. It has cross-platform functionality nailed. Daniel has been developing it forever and continues to do so. The code is obvious and procedural. A great project. Not the friendliest API, but it's 100% predictable and it WORKS.
mijoharas 9y ago
One thing to clarify is the difference between cURL and wget. While they both have a command line front-end (probably what you're thinking of), only cURL has a C++ library. This is what is used so widely, every application needs to speak to the internet in some way (read: a crapload of applications) have to use some library. cURL is low-level, efficient, cross-platform, full of features for everything you'd need and free. If you're working in C/C++ then you'll probably need a good reason not to use it, and if you're working in a higher level language then they will probably have their own networking stack (which may or may not be calling libcurl under the hood.

I'm curious: what made cURL succeed? I remember wget being the hotness back in the day. When I encountered cURL I thought it was a variant of wget.

by wenc 9y ago

I'm curious: what made cURL succeed? I remember wget being the hotness back in the day. When I encountered cURL I thought it was a variant of wget.

jacobush 9y ago

I think it was you could use it a library more easily. Curl is MIT licensed, so less to worry about if integrating in your code base. I also think it was designed as a library from the beginning, while wget was intended solely as a command line utility. Wget is also GPL licensed, so does not mix super well with proprietary programs or if you just don't want to "taint" your codebase with GPL for other reasons.

joering2 9y ago

It is just much easier to use in my opinion. I ran tons of cronjobs to update certain things. Few cool options I have with cURL that I find hard time locating in wget (doesn't mean they don't exist) --max-time (seconds) - for some reason I found wget not respecting always how much time it should be pulling the content. I think its related to fact curl and wget determine the moment of fetching file differently. --retry 0 - i did not find a way to tell wget not to retry pulling page. This was super important to my setup as we ran some pages for 6 hours to make sure the job is done. wget will gave up and retry after 15 minutes: HTTP request sent, awaiting response... Read error (Connection timed out) in headers. Retrying. Curl combined with solo for cronjobs is making pulling scripts on 32 servers very easy task and I hadn't had issues with it for 3 years now.

hemancuso 9y ago

I write ExpanDrive and when we moved from Python->C++ I used Microsoft's https://github.com/Microsoft/cpprestsdk https://github.com/Microsoft/cpprestsdk C++ REST SDK [Casablanca], because it is Microsoft, documented, modern, nice C++ etc. But for the latest iteration we switched everything over to libcurl. Libcurl has a bazillion edge cases handled. There are a million options, and it supports EVERYTHING. It has cross-platform functionality nailed. Daniel has been developing it forever and continues to do so. The code is obvious and procedural. A great project. Not the friendliest API, but it's 100% predictable and it WORKS.

mijoharas 9y ago

One thing to clarify is the difference between cURL and wget. While they both have a command line front-end (probably what you're thinking of), only cURL has a C++ library. This is what is used so widely, every application needs to speak to the internet in some way (read: a crapload of applications) have to use some library. cURL is low-level, efficient, cross-platform, full of features for everything you'd need and free. If you're working in C/C++ then you'll probably need a good reason not to use it, and if you're working in a higher level language then they will probably have their own networking stack (which may or may not be calling libcurl under the hood.