10 ms·

Convert Curl Syntax to Python Requests

166 points by kapkapkap 7y ago

empath75 7y ago
I feel like going the other way would be more useful.
- winstonewert 7y ago
  I thought the same thing at first, but the site actually gives an indication of why this would be useful. The developer tools in at least some browsers can export a request as a curl command. So I can thus easily get the code to reproduce any request I see in my browser. That would have been useful a few times in the past, and more useful than the inverse.
kapkapkap 7y ago
I'm not the author of this tool, but I submitted it to HN because I just stumbled upon it after unsuccessfully trying to transpose a complicated cURL from Chrome devtools into python requests for a good 20 min. This worked instantly. Props to the author - https://twitter.com/nickc_dev https://twitter.com/nickc_dev
- Wheaties466 7y ago
  I've been looking for something like this for a while but I never wanted to spend the time actually searching when I could just spend the time converting it manually. Thanks for submitting this.
SEJeff 7y ago
Failed entirely to work on my first attempt with: curl -XHEAD https://google.com It is just flat out wrong on: curl -X HEAD https://google.com In that it does a GET request: import requests response = requests.get('https://google.com/') Clever idea, but my first totally valid example failed (it does OPTIONS, GET, POST, PUT, PATCH successfully)
- OJFord 7y ago
  `curl --head` (short form: `-I`) is converted correctly though, and `curl -X HEAD` (or `-XHEAD`) does generate warnings telling you to use `--head`/`-I`.
- duckmysick 7y ago
  In case the creator of this tool doesn't read this thread, you can report the issues on GitHub: https://github.com/NickCarneiro/curlconverter/issues https://github.com/NickCarneiro/curlconverter/issues
  - greglindahl 7y ago
    Even if the creator of a tool does read the thread, discussions like these are not the place to report bugs.
- kissgyorgy 7y ago
  https://github.com/NickCarneiro/curlconverter/issues/new https://github.com/NickCarneiro/curlconverter/issues/new
scotteh 7y ago
The "uncurl" library[1] does something similar and can be used in the terminal. I've found it extremely useful. [1]: https://github.com/spulec/uncurl https://github.com/spulec/uncurl
symmitchry 7y ago
Paw, which many people love, has a neat and similar feature: you can get extensions that convert the request you build in the app to any format: curl, python, etc. It's very handy. https://paw.cloud/docs/getting-started/code-generator https://paw.cloud/docs/getting-started/code-generator
_wldu 7y ago
Here is a similar site that translates curl to Go: https://mholt.github.io/curl-to-go/ https://mholt.github.io/curl-to-go/
hkchad 7y ago
Postman also has the ability to import a curl command (from chrome dev tool) and then export to many programming languages.
mlevental 7y ago
this is one of the most useful little tools on good's green earth. I used to work for a scraping company and I used it all day every day - load a page with inspector open, find your request, right click copy as curl, paste into trillworks and boom you have the request you want to automate (modulo fiddling with cookies). A+ bang/line of code
runnr_az 7y ago
hey... that's super handy. thx!
deleted 7y ago
[deleted]
daolf 7y ago
For those interested in integrating such a feature on their website. An helper for building API request, for example, there is this great open source project: https://github.com/Kong/apiembed/ https://github.com/Kong/apiembed/ It supports a dozen of language.
keymone 7y ago
reminds me of a little clojure macro i wrote to demonstrate "power" of macros in lisp: https://gist.github.com/keymone/d7725767dc0425a7bcf9 https://gist.github.com/keymone/d7725767dc0425a7bcf9, obviously not feature-complete but sufficed for the demonstration
sikhnerd 7y ago
The capability to convert a network request from one format to another is the main reason I use Postman, which has a great implementation of this and a ton of tools and languages it can do it in. This looks great as well
mycodebreaks 7y ago
Is it all client side? I wonder how many cookies/secret headers will be pasted by unaware and unknowing users.
rgovostes 7y ago
I've also written a tool that converts curl commands to Python Requests or JavaScript XMLHttpRequest code. It is tested with curl commands generated by Safari, Burp Suite, and Charles Proxy. It can also translate raw HTTP requests, so it can be used when you have a packet capture. The design is modular and separates the frontend (curl) from the backend (Python), so more input and output formats can be added. It tries to be smart about generating "clean" code, so it will, for example, remove the Content-Length header when it can be recomputed from the request content. https://ryan.govost.es/http-translator/ https://ryan.govost.es/http-translator/ It is also open source. Feedback or pull requests are welcome. https://github.com/rgov/http-translator https://github.com/rgov/http-translator
agordhandas 7y ago
I independently used this tool a couple days ago. Helped save me a lot of debug time!
jchw 7y ago
Oh wow, not just Python - quire a wide array of languages are supported. Very nice, this will be handy for prototyping things.
ken 7y ago
This is neat, but also seems like a code smell. Do people really need to convert between HTTP request formats/languages much? When I see many independent implementations of an N-to-M mapping, it looks like the perfect use case for a standardized interface, like a DSL. In fact, can't HAR do this? I'd love to be able to just pass a HAR string to any HTTP client library, and have it execute that. Or call "dump_har()" (or "--dump-har") on any client, and have it spit out HAR that I can take it to any other client.
- eat_veggies 7y ago
  When reverse engineering or automating an API my workflow will often look like this: 1. Open browser inspector, watch API requests for one I'm interested in 2. Right click, "Copy as cURL" 3. Paste into a text editor, remove some unnecessary headers, re-running the curl command to verify that the headers I removed aren't important 4. Convert into python requests so that I can play with the parameters
  - ShakataGaNai 7y ago
    100% this. I do a lot of small automations against various API's (or sometimes against systems that don't API's at all) and have used this same workflow many times. Also handy because even well documented API's don't have examples in every language, but most do have examples in curl. So you can easily take those example curl's and dump them out to something programatic in your language.
  - ken 7y ago
    Right, and Firefox and Chrome both have "Copy as HAR". That should save a step, right? Plus, you probably have a text editor that knows about JSON, so it's easy to (say) remove keys from a JSON file, or even just keep the keys on separate lines and add/remove entire lines. I haven't seen a text editor (even Emacs!) that was as good at editing shell command parameters. It's a lot easier to quote special characters for JSON, too, compared to quoting special characters for my shell.
- dmurray 7y ago
  HTTP is the standardized interface. If you wrapped it in a prettier DSL, people would still have to take raw requests and figure out how to make them in the DSL.
tzury 7y ago
Failed for this one Curl -vLkXPOST https://www.google.com
- kissgyorgy 7y ago
  Because that's not a "curl" command. It's a "Curl" command.
  - danielhlockard 7y ago
    Also it works if you split out the flags.
fireattack 7y ago
This is especially handy on Windows, where Chrome failed to even produce proper cURL commands in CMD format [1]. [1]: https://bugs.chromium.org/p/chromium/issues/detail?id=658956 https://bugs.chromium.org/p/chromium/issues/detail?id=658956 [2]: https://bugs.chromium.org/p/chromium/issues/detail?id=798498 https://bugs.chromium.org/p/chromium/issues/detail?id=798498
NickC_dev 7y ago
Author, here. curlconverter gets about 17k unique users per month. The code generated by this tool is probably in production all over the world. But the codebase needs maintenance! I hired a junior developer (cf512) to do about 40 hours of dev work last month, but that contract has ended and I'd appreciate more volunteers. All the code is open source. I see a few bug reports down thread. Please open tickets. A big thanks to the top contributors, csells and jgroom33. Many others have pitched in over the last five years. Also, big thanks to Daniel Stenberg for writing curl in the first place.
- amrrs 7y ago
  Thank you for making this. Wouldn't hacktoberfest tagging help some visibility for contribution?
- 098799 7y ago
  This tool is amazing. I use it daily. Please don't let it disappear.
vesche 7y ago
There's a bit of a hidden feature in Postman that can do this and way more, hit the code button in the right corner and you can convert your web requests to something like a dozen languages: https://i.imgur.com/0qUV8b9.png https://i.imgur.com/0qUV8b9.png
- whitehouse3 7y ago
  Insomnia REST Client converts cURL commands into request sessions too.
  - girvo 7y ago
    Paw, too
danielhlockard 7y ago
oh hey, I wrote the `Go` support for this tool.
- NickC_dev 7y ago
  How could I forget, PR #100. Fyi, Go received about 1200 conversions last month. Thanks!
pramttl 7y ago
Great tool, would be nice to also have a chrome extension for the same that augments dev-tools to reduce one extra step of copy/paste.
m463 7y ago
I wrote a subroutine macos python to do the opposite. On macos, requests is not included (although it can be installed with a little effort), but curl was there. Note that you can easily automate interacting with a web page using the developer menus in safari and firefox. monitor the network requests, then for the request you're interested in, use "copy as cURL'. You can copy/paste that to invoke curl in the same way the web page used it. (sometime I have to use --cookie-jar /tmp/cook --cookie /tmp/cook)
northisup 7y ago
I made a similar library! https://github.com/northisup/curlit https://github.com/northisup/curlit (sorry for the poor documentation!)
40four 7y ago
OK, I'm not going to lie. I didn't even know you could copy requests from the network tab into curl (or fetch/ powershell/ etc.) like that. Embarrassing! :) Seems pretty useful!
- Falling3 7y ago
  One of my favorite parts is it grabs the cookie as well.
faizshah 7y ago
This tool is incredibly useful, I use it at least once a week. Its incredibly useful to explore a site with network in devtools then be able to quickly turn it into a python prototype! Amazing work :)

166 points by kapkapkap 7y ago

empath75 7y ago

I feel like going the other way would be more useful.

winstonewert 7y ago
I thought the same thing at first, but the site actually gives an indication of why this would be useful. The developer tools in at least some browsers can export a request as a curl command. So I can thus easily get the code to reproduce any request I see in my browser. That would have been useful a few times in the past, and more useful than the inverse.

kapkapkap 7y ago

I'm not the author of this tool, but I submitted it to HN because I just stumbled upon it after unsuccessfully trying to transpose a complicated cURL from Chrome devtools into python requests for a good 20 min. This worked instantly. Props to the author - https://twitter.com/nickc_dev https://twitter.com/nickc_dev

Wheaties466 7y ago
I've been looking for something like this for a while but I never wanted to spend the time actually searching when I could just spend the time converting it manually. Thanks for submitting this.

SEJeff 7y ago

Failed entirely to work on my first attempt with: curl -XHEAD https://google.com It is just flat out wrong on: curl -X HEAD https://google.com In that it does a GET request: import requests response = requests.get('https://google.com/') Clever idea, but my first totally valid example failed (it does OPTIONS, GET, POST, PUT, PATCH successfully)

OJFord 7y ago
`curl --head` (short form: `-I`) is converted correctly though, and `curl -X HEAD` (or `-XHEAD`) does generate warnings telling you to use `--head`/`-I`.
duckmysick 7y ago
In case the creator of this tool doesn't read this thread, you can report the issues on GitHub: https://github.com/NickCarneiro/curlconverter/issues https://github.com/NickCarneiro/curlconverter/issues
- greglindahl 7y ago
  Even if the creator of a tool does read the thread, discussions like these are not the place to report bugs.
kissgyorgy 7y ago
https://github.com/NickCarneiro/curlconverter/issues/new https://github.com/NickCarneiro/curlconverter/issues/new

scotteh 7y ago

The "uncurl" library[1] does something similar and can be used in the terminal. I've found it extremely useful. [1]: https://github.com/spulec/uncurl https://github.com/spulec/uncurl

symmitchry 7y ago

Paw, which many people love, has a neat and similar feature: you can get extensions that convert the request you build in the app to any format: curl, python, etc. It's very handy. https://paw.cloud/docs/getting-started/code-generator https://paw.cloud/docs/getting-started/code-generator

_wldu 7y ago

Here is a similar site that translates curl to Go: https://mholt.github.io/curl-to-go/ https://mholt.github.io/curl-to-go/

hkchad 7y ago

Postman also has the ability to import a curl command (from chrome dev tool) and then export to many programming languages.

mlevental 7y ago

this is one of the most useful little tools on good's green earth. I used to work for a scraping company and I used it all day every day - load a page with inspector open, find your request, right click copy as curl, paste into trillworks and boom you have the request you want to automate (modulo fiddling with cookies). A+ bang/line of code

runnr_az 7y ago

hey... that's super handy. thx!

deleted 7y ago

[deleted]

daolf 7y ago

For those interested in integrating such a feature on their website. An helper for building API request, for example, there is this great open source project: https://github.com/Kong/apiembed/ https://github.com/Kong/apiembed/ It supports a dozen of language.

keymone 7y ago

reminds me of a little clojure macro i wrote to demonstrate "power" of macros in lisp: https://gist.github.com/keymone/d7725767dc0425a7bcf9 https://gist.github.com/keymone/d7725767dc0425a7bcf9, obviously not feature-complete but sufficed for the demonstration

sikhnerd 7y ago

The capability to convert a network request from one format to another is the main reason I use Postman, which has a great implementation of this and a ton of tools and languages it can do it in. This looks great as well

mycodebreaks 7y ago

Is it all client side? I wonder how many cookies/secret headers will be pasted by unaware and unknowing users.

rgovostes 7y ago

I've also written a tool that converts curl commands to Python Requests or JavaScript XMLHttpRequest code. It is tested with curl commands generated by Safari, Burp Suite, and Charles Proxy. It can also translate raw HTTP requests, so it can be used when you have a packet capture. The design is modular and separates the frontend (curl) from the backend (Python), so more input and output formats can be added. It tries to be smart about generating "clean" code, so it will, for example, remove the Content-Length header when it can be recomputed from the request content. https://ryan.govost.es/http-translator/ https://ryan.govost.es/http-translator/ It is also open source. Feedback or pull requests are welcome. https://github.com/rgov/http-translator https://github.com/rgov/http-translator

agordhandas 7y ago

I independently used this tool a couple days ago. Helped save me a lot of debug time!

jchw 7y ago

Oh wow, not just Python - quire a wide array of languages are supported. Very nice, this will be handy for prototyping things.

ken 7y ago

This is neat, but also seems like a code smell. Do people really need to convert between HTTP request formats/languages much? When I see many independent implementations of an N-to-M mapping, it looks like the perfect use case for a standardized interface, like a DSL. In fact, can't HAR do this? I'd love to be able to just pass a HAR string to any HTTP client library, and have it execute that. Or call "dump_har()" (or "--dump-har") on any client, and have it spit out HAR that I can take it to any other client.

eat_veggies 7y ago
When reverse engineering or automating an API my workflow will often look like this: 1. Open browser inspector, watch API requests for one I'm interested in 2. Right click, "Copy as cURL" 3. Paste into a text editor, remove some unnecessary headers, re-running the curl command to verify that the headers I removed aren't important 4. Convert into python requests so that I can play with the parameters
- ShakataGaNai 7y ago
  100% this. I do a lot of small automations against various API's (or sometimes against systems that don't API's at all) and have used this same workflow many times. Also handy because even well documented API's don't have examples in every language, but most do have examples in curl. So you can easily take those example curl's and dump them out to something programatic in your language.
- ken 7y ago
  Right, and Firefox and Chrome both have "Copy as HAR". That should save a step, right? Plus, you probably have a text editor that knows about JSON, so it's easy to (say) remove keys from a JSON file, or even just keep the keys on separate lines and add/remove entire lines. I haven't seen a text editor (even Emacs!) that was as good at editing shell command parameters. It's a lot easier to quote special characters for JSON, too, compared to quoting special characters for my shell.
dmurray 7y ago
HTTP is the standardized interface. If you wrapped it in a prettier DSL, people would still have to take raw requests and figure out how to make them in the DSL.

tzury 7y ago

Failed for this one Curl -vLkXPOST https://www.google.com

kissgyorgy 7y ago
Because that's not a "curl" command. It's a "Curl" command.
- danielhlockard 7y ago
  Also it works if you split out the flags.

fireattack 7y ago

This is especially handy on Windows, where Chrome failed to even produce proper cURL commands in CMD format [1]. [1]: https://bugs.chromium.org/p/chromium/issues/detail?id=658956 https://bugs.chromium.org/p/chromium/issues/detail?id=658956 [2]: https://bugs.chromium.org/p/chromium/issues/detail?id=798498 https://bugs.chromium.org/p/chromium/issues/detail?id=798498

NickC_dev 7y ago

Author, here. curlconverter gets about 17k unique users per month. The code generated by this tool is probably in production all over the world. But the codebase needs maintenance! I hired a junior developer (cf512) to do about 40 hours of dev work last month, but that contract has ended and I'd appreciate more volunteers. All the code is open source. I see a few bug reports down thread. Please open tickets. A big thanks to the top contributors, csells and jgroom33. Many others have pitched in over the last five years. Also, big thanks to Daniel Stenberg for writing curl in the first place.

amrrs 7y ago
Thank you for making this. Wouldn't hacktoberfest tagging help some visibility for contribution?
098799 7y ago
This tool is amazing. I use it daily. Please don't let it disappear.

vesche 7y ago

There's a bit of a hidden feature in Postman that can do this and way more, hit the code button in the right corner and you can convert your web requests to something like a dozen languages: https://i.imgur.com/0qUV8b9.png https://i.imgur.com/0qUV8b9.png

whitehouse3 7y ago
Insomnia REST Client converts cURL commands into request sessions too.
- girvo 7y ago
  Paw, too

danielhlockard 7y ago

oh hey, I wrote the `Go` support for this tool.

NickC_dev 7y ago
How could I forget, PR #100. Fyi, Go received about 1200 conversions last month. Thanks!

pramttl 7y ago

Great tool, would be nice to also have a chrome extension for the same that augments dev-tools to reduce one extra step of copy/paste.

m463 7y ago

I wrote a subroutine macos python to do the opposite. On macos, requests is not included (although it can be installed with a little effort), but curl was there. Note that you can easily automate interacting with a web page using the developer menus in safari and firefox. monitor the network requests, then for the request you're interested in, use "copy as cURL'. You can copy/paste that to invoke curl in the same way the web page used it. (sometime I have to use --cookie-jar /tmp/cook --cookie /tmp/cook)

northisup 7y ago

I made a similar library! https://github.com/northisup/curlit https://github.com/northisup/curlit (sorry for the poor documentation!)

40four 7y ago

OK, I'm not going to lie. I didn't even know you could copy requests from the network tab into curl (or fetch/ powershell/ etc.) like that. Embarrassing! :) Seems pretty useful!

Falling3 7y ago
One of my favorite parts is it grabs the cookie as well.

faizshah 7y ago

This tool is incredibly useful, I use it at least once a week. Its incredibly useful to explore a site with network in devtools then be able to quickly turn it into a python prototype! Amazing work :)