9 ms·

Show HN: API Parrot – Automatically Reverse Engineer HTTP APIs

456 points by pvarghav 2y ago

When automating business processes at work, I found it difficult and time-consuming to reverse engineer business systems' APIs. I often had to manually reverse engineer APIs using developer tools or settle for less optimal technologies such as Robotic Process Automation (RPA). Often, the issue is that it can be hard to resolve all the cookies, access tokens, and other elements required to successfully execute the requests. Manually trying to resolve these dependencies using developer tools is especially challenging with multiple requests where data is stored in JavaScript objects or HTML elements. To try to solve this issue, I built a tool called API Parrot that automatically identifies the data correlations between requests and builds a graphical representation of the flow to give users a better understanding. To streamline the process, I also included functionality to record requests, define your own inputs and outputs, and export the entire flow—or parts of it—as JavaScript code. The application is Electron-based and currently compiled for Windows and Linux. Please try it out and give feedback! Online Tutorial: A simple example of reverse engineering the USPS API is available at https://docs.apiparrot.com/docs/category/tutorial---reverse-engineering-the-usps-api https://docs.apiparrot.com/docs/category/tutorial---reverse-...

victor106 2y ago
Looks great, but no Mac app?
TripleChecker 2y ago
[flagged]
- woleium 2y ago
  I am getting bit sick of this triplechecker spam.
vhayda 2y ago
Nice! It needs some refinement and a macOS version.
- tommiegannert 2y ago
  Could you give some examples of what refinement you think it needs?
davide_v 2y ago
Nice, I was looking for something like this. I tried it on Ubuntu but after clicking Capture requests > Launch Chrome, nothing happens.
- ashenke 2y ago
  Yep same problem
  - deleted 2y ago
    [deleted]
- chompin 2y ago
  Same issue, would prefer the option to use any browser also. Chrome is not my cup of tea
- 7357 2y ago
  Looks like it wants to run chrome using `start chrome` which is AFAIK a Windows-only command.
- pvarghav 2y ago
  Thank you for pointing this out. I've addressed the issue, and it should now be fixed in version 0.2.1, which is available for download on the website. Please update to the latest version, and let me know if you encounter any more problems.
  - 4k1l 2y ago
    This problem persist on version 0.2.3
    - pvarghav 2y ago
      On Linux, the application currently uses the command: ```bash google-chrome --proxy-server=<proxy URL> --ignore-certificate-errors-spki-list=<CA fingerprint> ``` to launch Chrome. For some users, this command doesn't seem to work properly. I'm working on adding a feature that will allow you to manually set the command used to launch Chrome if the default one isn't working on your system. In the meantime, you can manually launch Chrome with the arguments shown above. Just replace `<proxy URL>` and `<CA fingerprint>` with the appropriate values for your setup.
brushfoot 2y ago
Impressive project. I was curious how it discovers data relationships and was going to check the repo, but it looks like there's no code, only issues and releases. Is that right? Which leads me to... - Is this closed source? - Does it cost money? - How does it discover data relationships?
- skeptrune 2y ago
  It's entertaining that Github has become such a common place to find information that even closed source projects put something up there
- pvarghav 2y ago
  Thanks for your interest! - Is this closed source? Currently, the code is not open source, but I might open-source parts of it in the future. - Does it cost money? The software is free to use. If there is demand, I might create a "pro" version for businesses in the future. However, I intend to always have a free version available for individuals. - How does it discover data relationships? I've discussed how it discovers data relationships in the documentation here: https://docs.apiparrot.com/docs/tutorial-extras/exchange-model https://docs.apiparrot.com/docs/tutorial-extras/exchange-mod.... In short, the tool breaks down the data in the requests and responses into smaller parts by identifying their formats. For example, `["foo", "bar"]` would be recognized as a JSON array and broken down into the elements `"foo"` and `"bar"`. By applying this method recursively, you build a tree-like structure of the data. If an exact match is found between data in a response from a previous request and data in a subsequent request, a correlation is detected. Please feel free to ask if you have any more questions!
  - urronglol 2y ago
    [flagged]
  - bjt12345 2y ago
    If this can save me time at work, I'd be happy to throw some money at it. My bosses OTOH...let's just say, there's no penalty within companies for pointy haired bosses not making decisions to purchase something like this and ignoring staff. It's a false economy but I'm tired of it and just purchase what I can afford.
gtirloni 2y ago
Is there a ToS/License somewhere?
enricotal 2y ago
Fantastic Tool ... Mac version is paramount
yellow_lead 2y ago
Hi, it seems youve spelled reverse wrong > API Parrot is the tool specifically designed to reverese engineer the HTTP APIs of any website.
- pvarghav 2y ago
  Thanks for pointing this out! It should now be fixed.
yawndex 2y ago
Any current plans for a macOS release?
- pvarghav 2y ago
  Yes, I plan to release a macOS version of API Parrot. Unfortunately, I currently don't own a Mac, and since building macOS applications requires one, this has delayed the release. I'm actively exploring solutions, such as accessing a Mac environment remotely or acquiring the necessary hardware.
  - rrr_oh_man 2y ago
    That sounds like a gofundme situation :)
  - eagleinparadise 2y ago
    +1 for macos!
  - garysahota93 2y ago
    Have you looked into Mac Stadium? Maybe they can be of help! https://www.macstadium.com/ https://www.macstadium.com/
- MK2k 2y ago
  Try a Hackintosh, e.g. as a virtual machine on your Windows or Linux host.
- pvarghav 2y ago
  The macOS version is now available for download at https://apiparrot.com/#download https://apiparrot.com/#download Please note that since the app isn't code-signed yet, you'll need to remove the quarantine attribute to run it. I've updated the documentation with instructions on how to do this: https://docs.apiparrot.com/docs/getting-started/download-and-install-apiparrot https://docs.apiparrot.com/docs/getting-started/download-and... Let me know if you have any questions or run into any issues!
devops000 2y ago
Feedback: add a newsletter form to get notification when you will release the MacOSX version
- toomuchtodo 2y ago
  +1, ready to buy
- pvarghav 2y ago
  Thank you for your suggestion! I've added a newsletter sign-up form at the bottom of the webpage: https://apiparrot.com/#newsletter https://apiparrot.com/#newsletter Feel free to subscribe to receive notifications when we release the MacOSX version.
setheron 2y ago
Very sad half the comments are asking for MacOS app. The rise of development on MacOS for server development when the final target is Linux will cause long term harm to the newer generation of engineers
- yoavmmn 2y ago
  Nowadays everything runs on docker anyway
  - setheron 2y ago
    You'd never see a Windows developer work in MacOS or a iOS developer work in Linux but Linux developers (server side) routinely work in MacOS Unnecessary abstraction
    - quesera 2y ago
      Counter-argument: it could be risky to dev on and deploy to a single monoculture. But empirically, I've been developing on macOS (etc) and Linux (often simultaneously), and deploying to Linux (Debian, RHEL/AL), Solaris (etc), and FreeBSD ... for more than 20 years. Aside from package management tooling differences, package naming, and package content splits (e.g. pkg vs pkg-dev) -- all of which are equally inconsistent between Linux distros -- I cannot recall a single issue caused by this heterogeneity.
    - prophesi 2y ago
      > iOS developer work in Linux I dream of the day Apple releases official docker images. Building for iOS is the only reason I have to touch a Mac.
    - cellwebb 2y ago
      All SNES games should have been developed in Mario Paint or it was an unnecessary abstraction
    - luismedel 2y ago
      Define "unnecessary", please. > [...] or a iOS developer work in Linux In the past I did a lot of successful work on iOS apps from a Windows system, thanks to Xamarin and a mac sitting on a shelf, acting as the remote system. Also, please, remember what "cross compilation" mean.
colesantiago 2y ago
This is the easy part. One of the issues with these tools is that more and more websites now employ multiple aggressive CAPTCHAs, fingerprints, device check, etc, rendering tools like API Parrot almost useless.
m00dy 2y ago
can it reverse websocket-protocols ? If so, how does it do binary decoding etc ?
- pvarghav 2y ago
  Currently only HTTP requests are supported. I might add support for websockets later, however that is a harder problem to solve due to the binary encoding etc.
moon82 2y ago
looks amazing! thanks for sharing, will give it a shot in a short while. Btw, how do you keep yourself motivated on working on free projects? Obviosly it takes a lot of effort and no one is paying for that.
- pvarghav 2y ago
  Thank you! Working on this side project has been both fun and rewarding. I've learned a lot throughout the process, which keeps me motivated even without immediate financial gain. I have plenty of ideas on how to improve the software in various ways. Some of these enhancements could become part of a "pro" version tailored for businesses. My long-term ambition is to turn this into a full-fledged product, which would enable me to dedicate more time to its development.
pkkkzip 2y ago
interesting but not sure what the value add here is, it gives you a graph flow of all the API requests being made? and then the goal is to replay them? aren't there github libraries that do this already?
- sumedh 2y ago
  > aren't there github libraries that do this already? which ones?
rynn 2y ago
How does it compare to mitmmitmproxy2swagger? https://github.com/alufers/mitmproxy2swagger https://github.com/alufers/mitmproxy2swagger
- 1a527dd5 2y ago
  The first and immediate difference for me is the ability to recall the name. I can recall Postman/Insomina fine, and now for API Parrot. I'm never going to be able to recall mitmproxy2swagger. Unfortunately, names matter.
  - itsafarqueue 2y ago
    Thanks 1a527dd5.
    - 1a527dd5 2y ago
      Ha! Nicely played. That was out of purely laziness. I don't like using one handle across sites, so I take the first 8 chars of (New-Guid).ToString() and then dump it in my password manager.
      - rrr_oh_man 2y ago
        https://xkcd.com/1105/ https://xkcd.com/1105/
        swyx 2y ago
        a real pitty that this was not xkcd 1111
  - SparkyMcUnicorn 2y ago
    I often forget the name of things, sometimes even the big ones. GitHub search is one of the primary ways I rediscover them. "reverse-engineer API" returns mitmproxy2swagger as the third result, and this is how I found it last time I needed it. It is a bit frustrating when a project on GitHub doesn't have good tags or searchable keywords, making it harder to find.
sumanyusharma 2y ago
How is this different from Integuru? They posted a few weeks back here: https://news.ycombinator.com/item?id=41983409 https://news.ycombinator.com/item?id=41983409
- martinkostov 2y ago
  I'm curious too
- shawnshivdat 2y ago
  ^
teichman 2y ago
Integru has been really great for us. Curious how you think about differentiation?
sidgarimella 2y ago
Love this. I’ve worked on a few projects in RPA prior and I’m losing faith in selectors. I think either direct data access like this or AI based CV are the automation arms of the future.
remoquete 2y ago
Looks very interesting. Does it produce an OpenAPI file? That'd help immensely in documenting APIs that lack specifications.
1a527dd5 2y ago
This is pretty cool, I ran it against one of a largest customer sites and it was very interesting to see how the page all interconnects. I'm pretty sure it can be used to spot architecture/performance problems.
ozim 2y ago
If only there would be something with schema like XML that people would use for the APIs ;) You could generate diagrams from WSDL and even generate client code from that. There is also bunch of JSON schema stuff nowadays. But yeah for a lot of people schema of API contracts feels like too much work and too much hassle. JSON serialization doesn’t throw errors for new properties quickly added on sending side and receiving side can ignore stuff - well as long as API semantics allow but that’s generally going to be a hassle always even with LLMs somehow autofixing your „schema”.
tveyben 2y ago
I’m not able to read what the product actually does - I keep getting distracted by the ‘snake’ animation surrounding the content .. not sure what the purpose is ;-)
sebmellen 2y ago
This is incredible. We’ve spent ages and ages figuring out the weird internals of certain legacy systems that we’ve ended up having to use bots or RPA to integrate with. If you can polish this into a true product, we would pay for it! Any chance of a Mac version?
- pvarghav 2y ago
  Thanks! I'm glad you like the idea—it sounds like you've had the same struggles I've been through. Good news: the Mac version is now available to download at https://apiparrot.com/#download https://apiparrot.com/#download. Let me know if you have any feedback!
YaBa 2y ago
Interesting but... The first website I've tried it (which I'm currently working on due to a change of platform) couldn't find anything other than the main request, and I know for sure there is a POST reguest to the API to get some data (I had a scrapper working, website changed, had to re-do the scrapper again). I've checked the tutorial, seems that I'm not missing any step, the software simple cannot capture anything if the request is made on the main page, seems to work fine with forms, buttons and "manual" actions. I can DM you the website plus the expected request that is made, visible with any browser internal debugging tools.
speakspokespok 2y ago
I've just gone through the "Docs" section and I appreciate how it covers the intended workflow and use cases. I'm on Debian/Intel and other than the need to install Chrome I only had a few small issues. ++ A self contained appImage is a good way to go, but where do you put it? A default install location should be added for those used to an `apt install`. I went `sudo wget $URL -C /usr/local/bin/` and `chmod +x $appimage`. This worked fine until Collection creation when some internal state change smacked into my root owned file permissions. I `chmod 777` it and restarted the app, no more issue. It's my machine and I can chmod how I want but I think doc clarity would help those unfamiliar with appimage. ++ Renaming projects, collections, etc is cumbersome. For example, when clicking the 'New Project" pencil a change name window opens with several steps needed then to rename the project. That single click could combine opening the window, that window grabbing focus, with the cursor in a blank form window, followed by 'Enter'. ++ Ability to toggle showing the Properties column. On a 14" hi rez laptop, the screen is crowded. And resize Project width. ++ The default flow view size is too small. I hope that's helpful. A small number of UI tweaks and it's already at "Don't F*** With It!" stage. The issues above are small and don't take away from how great and EXCITED I was going through the tutorial. I went through the entire docs and the tutorial and I think it's a fine program. Your layout of the DOM response is also really nice!
- pvarghav 2y ago
  I'm glad to hear that you like the product and found the tutorial helpful. I've updated the documentation to include instructions for installing the AppImage—thanks for bringing that to my attention. I really appreciate your detailed feedback on the UI and usability. I'll definitely take your suggestions into consideration and work on implementing them in future versions. Thanks again for taking the time to share your thoughts!
user3939382 2y ago
This has been built in to postman for years
- sumedh 2y ago
  Can you please share a doc link?
Sytten 2y ago
Lots of interesting ideas, a lot of the same methodology is used by bug bounty hunters/pentesters. It gives me some perspective to build something in my tool.
arshxyz 2y ago
Interesting project, I've often looked for something like this but haven't found anything that does the job. I'm on a mac and can't wait to try this out. Can I ask what you're using adblock-rs for?
- pvarghav 2y ago
  Glad you like the project! I'm working on getting the macOS version built and released as soon as possible. If you'd like to be notified when it's ready, you can sign up for the newsletter here: https://apiparrot.com/#newsletter https://apiparrot.com/#newsletter. As for adblock-rs, I'm using it to detect and automatically disable requests related to ads and other unnecessary stuff. This helps cut down on noise and saves some time for developers.
  - arshxyz 2y ago
    I was able repackage it for macOS. I can help you test it too. Shot you an email at contact@apiparrot.com
    - sumedh 2y ago
      How were you able to do without the source code?
      - arshxyz 2y ago
        I extracted the zip, found the electron build folder, replaced a string in the minified code to launch Chrome on macOS correctly and ran electron-packager with the target being macos. Patching electron apps is fairly common. You can take a look at Spicetify or BetterDiscord to see the process in more detail
        sumedh 2y ago
        Oh wow, didn't know that was possible. Thanks for sharing.
MK2k 2y ago
Nice tool! I tested it against imdb.com with its large graphql responses and after a few requests API Parrot crashed.
255kb 2y ago
Nice project! Just so you know, there is an app called Traffic Parrot (https://trafficparrot.com/ https://trafficparrot.com/). They operate on the same market, so they may not like the name you chose.
pvarghav 2y ago
Update: The macOS version is now available for download at https://apiparrot.com/#download https://apiparrot.com/#download Please note that since the app isn't code-signed yet, you'll need to remove the quarantine attribute to run it. I've updated the documentation with instructions on how to do this: https://docs.apiparrot.com/docs/getting-started/download-and-install-apiparrot https://docs.apiparrot.com/docs/getting-started/download-and... Let me know if you have any questions or run into any issues!
spacecadet404 2y ago
Nice project, I was able to use it to map out some parts of a vendor's API that's been giving me grief today. I'm pretty amateur and this was really intuitive. Happily putting this in my toolbox.
jknutson 2y ago
Love the idea. I’m always finding myself writing little user scripts / browser extensions to extend websites I use all the time, and trying to use an API I found in the devtools network requests page always gets annoying when I have to try and do anything beyond replicating the exact input/output I found in the original request. Haven’t fully looked through the features/docs, so forgive me if my question is answered in there, but what does support look like for: - Exporting to Swagger/OpenAPI Spec - Exporting to generated SDK (I know some tools exist that can generate SDKs from OpenAPI/Swagger, so maybe some of these tools have licenses that are compatible with your product?) - Support for URL path variables (e.g. `/users/{user_id}`) - Support for URL query parameters (and filtering for common “noise” parameters, e.g. Google analytics) - Support for non-JSON input/output (e.g. an endpoint that accepts multipart form data) Awesome idea though. I’m definitely going to try this out. Beautiful UI and website too. I’m stoked to play around with this!
- pvarghav 2y ago
  I'm glad you like the idea! Let me answer your questions one by one: - Exporting to Swagger/OpenAPI Spec: Currently, exporting to Swagger/OpenAPI isn't supported, but it's on my to-do list to look into. Right now, JavaScript code is the only export format. - Exporting to a Generated SDK: Same as above. I'm considering integrating tools that can generate SDKs from OpenAPI/Swagger specs, so this might be included in a future update. - Support for URL Path Variables (e.g., `/users/{user_id}`): Yes, API Parrot supports URL path variables! - Support for URL Query Parameters (and filtering out common "noise" parameters like Google Analytics): Yes, API Parrot supports URL query parameters, and there are measures in place to filter out the noise. - Support for Non-JSON Input/Output (e.g., endpoints that accept multipart form data): There is support for non-JSON input/output formats, but multipart form data isn't supported at this time. You can find all the supported data types on this page: https://docs.apiparrot.com/docs/tutorial-extras/exchange-model https://docs.apiparrot.com/docs/tutorial-extras/exchange-mod... Thanks again for your support! I'm excited for you to try it out, and I'd love to hear your feedback after you've had a chance to play around with it.

456 points by pvarghav 2y ago

victor106 2y ago

Looks great, but no Mac app?

TripleChecker 2y ago

[flagged]

woleium 2y ago
I am getting bit sick of this triplechecker spam.

vhayda 2y ago

Nice! It needs some refinement and a macOS version.

tommiegannert 2y ago
Could you give some examples of what refinement you think it needs?

davide_v 2y ago

Nice, I was looking for something like this. I tried it on Ubuntu but after clicking Capture requests > Launch Chrome, nothing happens.

ashenke 2y ago
Yep same problem
- deleted 2y ago
  [deleted]
chompin 2y ago
Same issue, would prefer the option to use any browser also. Chrome is not my cup of tea
7357 2y ago
Looks like it wants to run chrome using `start chrome` which is AFAIK a Windows-only command.
pvarghav 2y ago
Thank you for pointing this out. I've addressed the issue, and it should now be fixed in version 0.2.1, which is available for download on the website. Please update to the latest version, and let me know if you encounter any more problems.
- 4k1l 2y ago
  This problem persist on version 0.2.3
  - pvarghav 2y ago
    On Linux, the application currently uses the command: ```bash google-chrome --proxy-server=<proxy URL> --ignore-certificate-errors-spki-list=<CA fingerprint> ``` to launch Chrome. For some users, this command doesn't seem to work properly. I'm working on adding a feature that will allow you to manually set the command used to launch Chrome if the default one isn't working on your system. In the meantime, you can manually launch Chrome with the arguments shown above. Just replace `<proxy URL>` and `<CA fingerprint>` with the appropriate values for your setup.

brushfoot 2y ago

Impressive project. I was curious how it discovers data relationships and was going to check the repo, but it looks like there's no code, only issues and releases. Is that right? Which leads me to... - Is this closed source? - Does it cost money? - How does it discover data relationships?

skeptrune 2y ago
It's entertaining that Github has become such a common place to find information that even closed source projects put something up there
pvarghav 2y ago
Thanks for your interest! - Is this closed source? Currently, the code is not open source, but I might open-source parts of it in the future. - Does it cost money? The software is free to use. If there is demand, I might create a "pro" version for businesses in the future. However, I intend to always have a free version available for individuals. - How does it discover data relationships? I've discussed how it discovers data relationships in the documentation here: https://docs.apiparrot.com/docs/tutorial-extras/exchange-model https://docs.apiparrot.com/docs/tutorial-extras/exchange-mod.... In short, the tool breaks down the data in the requests and responses into smaller parts by identifying their formats. For example, `["foo", "bar"]` would be recognized as a JSON array and broken down into the elements `"foo"` and `"bar"`. By applying this method recursively, you build a tree-like structure of the data. If an exact match is found between data in a response from a previous request and data in a subsequent request, a correlation is detected. Please feel free to ask if you have any more questions!
- urronglol 2y ago
  [flagged]
- bjt12345 2y ago
  If this can save me time at work, I'd be happy to throw some money at it. My bosses OTOH...let's just say, there's no penalty within companies for pointy haired bosses not making decisions to purchase something like this and ignoring staff. It's a false economy but I'm tired of it and just purchase what I can afford.

gtirloni 2y ago

Is there a ToS/License somewhere?

enricotal 2y ago

Fantastic Tool ... Mac version is paramount

yellow_lead 2y ago

Hi, it seems youve spelled reverse wrong > API Parrot is the tool specifically designed to reverese engineer the HTTP APIs of any website.

pvarghav 2y ago
Thanks for pointing this out! It should now be fixed.

yawndex 2y ago

Any current plans for a macOS release?

pvarghav 2y ago
Yes, I plan to release a macOS version of API Parrot. Unfortunately, I currently don't own a Mac, and since building macOS applications requires one, this has delayed the release. I'm actively exploring solutions, such as accessing a Mac environment remotely or acquiring the necessary hardware.
- rrr_oh_man 2y ago
  That sounds like a gofundme situation :)
- eagleinparadise 2y ago
  +1 for macos!
- garysahota93 2y ago
  Have you looked into Mac Stadium? Maybe they can be of help! https://www.macstadium.com/ https://www.macstadium.com/
MK2k 2y ago
Try a Hackintosh, e.g. as a virtual machine on your Windows or Linux host.
pvarghav 2y ago
The macOS version is now available for download at https://apiparrot.com/#download https://apiparrot.com/#download Please note that since the app isn't code-signed yet, you'll need to remove the quarantine attribute to run it. I've updated the documentation with instructions on how to do this: https://docs.apiparrot.com/docs/getting-started/download-and-install-apiparrot https://docs.apiparrot.com/docs/getting-started/download-and... Let me know if you have any questions or run into any issues!

devops000 2y ago

Feedback: add a newsletter form to get notification when you will release the MacOSX version

toomuchtodo 2y ago
+1, ready to buy
pvarghav 2y ago
Thank you for your suggestion! I've added a newsletter sign-up form at the bottom of the webpage: https://apiparrot.com/#newsletter https://apiparrot.com/#newsletter Feel free to subscribe to receive notifications when we release the MacOSX version.

setheron 2y ago

Very sad half the comments are asking for MacOS app. The rise of development on MacOS for server development when the final target is Linux will cause long term harm to the newer generation of engineers

yoavmmn 2y ago
Nowadays everything runs on docker anyway
- setheron 2y ago
  You'd never see a Windows developer work in MacOS or a iOS developer work in Linux but Linux developers (server side) routinely work in MacOS Unnecessary abstraction
  - quesera 2y ago
    Counter-argument: it could be risky to dev on and deploy to a single monoculture. But empirically, I've been developing on macOS (etc) and Linux (often simultaneously), and deploying to Linux (Debian, RHEL/AL), Solaris (etc), and FreeBSD ... for more than 20 years. Aside from package management tooling differences, package naming, and package content splits (e.g. pkg vs pkg-dev) -- all of which are equally inconsistent between Linux distros -- I cannot recall a single issue caused by this heterogeneity.
  - prophesi 2y ago
    > iOS developer work in Linux I dream of the day Apple releases official docker images. Building for iOS is the only reason I have to touch a Mac.
  - cellwebb 2y ago
    All SNES games should have been developed in Mario Paint or it was an unnecessary abstraction
  - luismedel 2y ago
    Define "unnecessary", please. > [...] or a iOS developer work in Linux In the past I did a lot of successful work on iOS apps from a Windows system, thanks to Xamarin and a mac sitting on a shelf, acting as the remote system. Also, please, remember what "cross compilation" mean.

colesantiago 2y ago

This is the easy part. One of the issues with these tools is that more and more websites now employ multiple aggressive CAPTCHAs, fingerprints, device check, etc, rendering tools like API Parrot almost useless.

m00dy 2y ago

can it reverse websocket-protocols ? If so, how does it do binary decoding etc ?

pvarghav 2y ago
Currently only HTTP requests are supported. I might add support for websockets later, however that is a harder problem to solve due to the binary encoding etc.

moon82 2y ago

looks amazing! thanks for sharing, will give it a shot in a short while. Btw, how do you keep yourself motivated on working on free projects? Obviosly it takes a lot of effort and no one is paying for that.

pvarghav 2y ago
Thank you! Working on this side project has been both fun and rewarding. I've learned a lot throughout the process, which keeps me motivated even without immediate financial gain. I have plenty of ideas on how to improve the software in various ways. Some of these enhancements could become part of a "pro" version tailored for businesses. My long-term ambition is to turn this into a full-fledged product, which would enable me to dedicate more time to its development.

pkkkzip 2y ago

interesting but not sure what the value add here is, it gives you a graph flow of all the API requests being made? and then the goal is to replay them? aren't there github libraries that do this already?

sumedh 2y ago
> aren't there github libraries that do this already? which ones?

rynn 2y ago

How does it compare to mitmmitmproxy2swagger? https://github.com/alufers/mitmproxy2swagger https://github.com/alufers/mitmproxy2swagger

1a527dd5 2y ago
The first and immediate difference for me is the ability to recall the name. I can recall Postman/Insomina fine, and now for API Parrot. I'm never going to be able to recall mitmproxy2swagger. Unfortunately, names matter.
- itsafarqueue 2y ago
  Thanks 1a527dd5.
  - 1a527dd5 2y ago
    Ha! Nicely played. That was out of purely laziness. I don't like using one handle across sites, so I take the first 8 chars of (New-Guid).ToString() and then dump it in my password manager.
    - rrr_oh_man 2y ago
      https://xkcd.com/1105/ https://xkcd.com/1105/
      - swyx 2y ago
        a real pitty that this was not xkcd 1111
- SparkyMcUnicorn 2y ago
  I often forget the name of things, sometimes even the big ones. GitHub search is one of the primary ways I rediscover them. "reverse-engineer API" returns mitmproxy2swagger as the third result, and this is how I found it last time I needed it. It is a bit frustrating when a project on GitHub doesn't have good tags or searchable keywords, making it harder to find.

sumanyusharma 2y ago

How is this different from Integuru? They posted a few weeks back here: https://news.ycombinator.com/item?id=41983409 https://news.ycombinator.com/item?id=41983409

martinkostov 2y ago
I'm curious too
shawnshivdat 2y ago
^

teichman 2y ago

Integru has been really great for us. Curious how you think about differentiation?

sidgarimella 2y ago

Love this. I’ve worked on a few projects in RPA prior and I’m losing faith in selectors. I think either direct data access like this or AI based CV are the automation arms of the future.

remoquete 2y ago

Looks very interesting. Does it produce an OpenAPI file? That'd help immensely in documenting APIs that lack specifications.

1a527dd5 2y ago

This is pretty cool, I ran it against one of a largest customer sites and it was very interesting to see how the page all interconnects. I'm pretty sure it can be used to spot architecture/performance problems.

ozim 2y ago

If only there would be something with schema like XML that people would use for the APIs ;) You could generate diagrams from WSDL and even generate client code from that. There is also bunch of JSON schema stuff nowadays. But yeah for a lot of people schema of API contracts feels like too much work and too much hassle. JSON serialization doesn’t throw errors for new properties quickly added on sending side and receiving side can ignore stuff - well as long as API semantics allow but that’s generally going to be a hassle always even with LLMs somehow autofixing your „schema”.

tveyben 2y ago

I’m not able to read what the product actually does - I keep getting distracted by the ‘snake’ animation surrounding the content .. not sure what the purpose is ;-)

sebmellen 2y ago

This is incredible. We’ve spent ages and ages figuring out the weird internals of certain legacy systems that we’ve ended up having to use bots or RPA to integrate with. If you can polish this into a true product, we would pay for it! Any chance of a Mac version?

pvarghav 2y ago
Thanks! I'm glad you like the idea—it sounds like you've had the same struggles I've been through. Good news: the Mac version is now available to download at https://apiparrot.com/#download https://apiparrot.com/#download. Let me know if you have any feedback!

YaBa 2y ago

Interesting but... The first website I've tried it (which I'm currently working on due to a change of platform) couldn't find anything other than the main request, and I know for sure there is a POST reguest to the API to get some data (I had a scrapper working, website changed, had to re-do the scrapper again). I've checked the tutorial, seems that I'm not missing any step, the software simple cannot capture anything if the request is made on the main page, seems to work fine with forms, buttons and "manual" actions. I can DM you the website plus the expected request that is made, visible with any browser internal debugging tools.

speakspokespok 2y ago

I've just gone through the "Docs" section and I appreciate how it covers the intended workflow and use cases. I'm on Debian/Intel and other than the need to install Chrome I only had a few small issues. ++ A self contained appImage is a good way to go, but where do you put it? A default install location should be added for those used to an `apt install`. I went `sudo wget $URL -C /usr/local/bin/` and `chmod +x $appimage`. This worked fine until Collection creation when some internal state change smacked into my root owned file permissions. I `chmod 777` it and restarted the app, no more issue. It's my machine and I can chmod how I want but I think doc clarity would help those unfamiliar with appimage. ++ Renaming projects, collections, etc is cumbersome. For example, when clicking the 'New Project" pencil a change name window opens with several steps needed then to rename the project. That single click could combine opening the window, that window grabbing focus, with the cursor in a blank form window, followed by 'Enter'. ++ Ability to toggle showing the Properties column. On a 14" hi rez laptop, the screen is crowded. And resize Project width. ++ The default flow view size is too small. I hope that's helpful. A small number of UI tweaks and it's already at "Don't F*** With It!" stage. The issues above are small and don't take away from how great and EXCITED I was going through the tutorial. I went through the entire docs and the tutorial and I think it's a fine program. Your layout of the DOM response is also really nice!

pvarghav 2y ago
I'm glad to hear that you like the product and found the tutorial helpful. I've updated the documentation to include instructions for installing the AppImage—thanks for bringing that to my attention. I really appreciate your detailed feedback on the UI and usability. I'll definitely take your suggestions into consideration and work on implementing them in future versions. Thanks again for taking the time to share your thoughts!

user3939382 2y ago

This has been built in to postman for years

sumedh 2y ago
Can you please share a doc link?

Sytten 2y ago

Lots of interesting ideas, a lot of the same methodology is used by bug bounty hunters/pentesters. It gives me some perspective to build something in my tool.

arshxyz 2y ago

Interesting project, I've often looked for something like this but haven't found anything that does the job. I'm on a mac and can't wait to try this out. Can I ask what you're using adblock-rs for?

pvarghav 2y ago
Glad you like the project! I'm working on getting the macOS version built and released as soon as possible. If you'd like to be notified when it's ready, you can sign up for the newsletter here: https://apiparrot.com/#newsletter https://apiparrot.com/#newsletter. As for adblock-rs, I'm using it to detect and automatically disable requests related to ads and other unnecessary stuff. This helps cut down on noise and saves some time for developers.
- arshxyz 2y ago
  I was able repackage it for macOS. I can help you test it too. Shot you an email at contact@apiparrot.com
  - sumedh 2y ago
    How were you able to do without the source code?
    - arshxyz 2y ago
      I extracted the zip, found the electron build folder, replaced a string in the minified code to launch Chrome on macOS correctly and ran electron-packager with the target being macos. Patching electron apps is fairly common. You can take a look at Spicetify or BetterDiscord to see the process in more detail
      - sumedh 2y ago
        Oh wow, didn't know that was possible. Thanks for sharing.

MK2k 2y ago

Nice tool! I tested it against imdb.com with its large graphql responses and after a few requests API Parrot crashed.

255kb 2y ago

Nice project! Just so you know, there is an app called Traffic Parrot (https://trafficparrot.com/ https://trafficparrot.com/). They operate on the same market, so they may not like the name you chose.

pvarghav 2y ago

Update: The macOS version is now available for download at https://apiparrot.com/#download https://apiparrot.com/#download Please note that since the app isn't code-signed yet, you'll need to remove the quarantine attribute to run it. I've updated the documentation with instructions on how to do this: https://docs.apiparrot.com/docs/getting-started/download-and-install-apiparrot https://docs.apiparrot.com/docs/getting-started/download-and... Let me know if you have any questions or run into any issues!

spacecadet404 2y ago

Nice project, I was able to use it to map out some parts of a vendor's API that's been giving me grief today. I'm pretty amateur and this was really intuitive. Happily putting this in my toolbox.

jknutson 2y ago

Love the idea. I’m always finding myself writing little user scripts / browser extensions to extend websites I use all the time, and trying to use an API I found in the devtools network requests page always gets annoying when I have to try and do anything beyond replicating the exact input/output I found in the original request. Haven’t fully looked through the features/docs, so forgive me if my question is answered in there, but what does support look like for: - Exporting to Swagger/OpenAPI Spec - Exporting to generated SDK (I know some tools exist that can generate SDKs from OpenAPI/Swagger, so maybe some of these tools have licenses that are compatible with your product?) - Support for URL path variables (e.g. `/users/{user_id}`) - Support for URL query parameters (and filtering for common “noise” parameters, e.g. Google analytics) - Support for non-JSON input/output (e.g. an endpoint that accepts multipart form data) Awesome idea though. I’m definitely going to try this out. Beautiful UI and website too. I’m stoked to play around with this!

pvarghav 2y ago
I'm glad you like the idea! Let me answer your questions one by one: - Exporting to Swagger/OpenAPI Spec: Currently, exporting to Swagger/OpenAPI isn't supported, but it's on my to-do list to look into. Right now, JavaScript code is the only export format. - Exporting to a Generated SDK: Same as above. I'm considering integrating tools that can generate SDKs from OpenAPI/Swagger specs, so this might be included in a future update. - Support for URL Path Variables (e.g., `/users/{user_id}`): Yes, API Parrot supports URL path variables! - Support for URL Query Parameters (and filtering out common "noise" parameters like Google Analytics): Yes, API Parrot supports URL query parameters, and there are measures in place to filter out the noise. - Support for Non-JSON Input/Output (e.g., endpoints that accept multipart form data): There is support for non-JSON input/output formats, but multipart form data isn't supported at this time. You can find all the supported data types on this page: https://docs.apiparrot.com/docs/tutorial-extras/exchange-model https://docs.apiparrot.com/docs/tutorial-extras/exchange-mod... Thanks again for your support! I'm excited for you to try it out, and I'd love to hear your feedback after you've had a chance to play around with it.