9 ms·

Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

341 points by pimterry 7y ago

pimterry 7y ago
Hey HN! I've been building this for a while, but up until now it's been a read-only HTTP debugging tool, and today's my first foray into proper mocking, which opens up a whole new world of fun (and practical use cases). Let me know what you think :-)
- mateuszf 7y ago
  Didn't try it yet but the promise is amazing.
- pault 7y ago
  I hope this works out! Using fiddler us such a frustrating experience. The documentation is practically non existent and the ux is atrocious, and as far as I can tell the closest thing to a manual is a paid ebook by the developer (I was so pissed when I discovered this). I would be very happy to have an open source alternative!
  - pimterry 7y ago
    Me too :-). I built this myself in large part because of frustrations with Fiddler. It was an amazing tool at the time, but it hasn't aged well imo... Hopefully this is your open source alternative - give it a go, let me know if there's something Fiddler does for you that HTTP Toolkit doesn't!
  - altano 7y ago
    If you think Fiddler sucks get a Mac and try using the tools available there like Charles Proxy. Ugh.
    - luckman212 7y ago
      Curious about this comment. I'm a Mac user and a Charles user. I find it to be a great tool. Why the negativity?
      - altano 7y ago
        It's extremely hard to use compared to something like Fiddler, especially when you are doing more than just tracing.
    - jame_bkk 7y ago
      Give Proxyman (https://proxyman.io/ https://proxyman.io/) a try, which is a better alternative for Fiddler and Charles Proxy.
    - stephenr 7y ago
      https://proxie.app/ https://proxie.app/ Works pretty well IMO.
  - jmkni 7y ago
    I don't mind Fiddler, and $15 dollars for a book on a free, incredibly powerful, tool is a bargain IMHO.
    - pault 7y ago
      What bothers me is not the price, it's that the only documentation available is an ebook that one of the developers wrote as a side project. The tool is very powerful, but it's a nightmare to use, and it doesn't matter how powerful it is if you spend more time struggling with the software than fixing the problem you have that requires it in the first place. I'm just happy that someone is attempting to create an alternative with a focus on usability and cross platform support.
      - jmkni 7y ago
        Yeah fair, Fiddler is definitely well...fiddly, and a bit dated as well. Agree that it's good to see new alternatives.
- RMPR 7y ago
  It may be interesting to have a rpm package
mariocesar 7y ago
There are full opensource alternative for this? I like the product, I'm testing it right now. But I'm curious if there is a way to accomplish something like this with existing opensource tools or a similar full opensource project.
- mateuszf 7y ago
  I think you can use for example mitmproxy for this though with more setup
- LongBallLary 7y ago
  It is 100% open source according to their website (in the footer). https://github.com/httptoolkit https://github.com/httptoolkit
- reffaelwallen 7y ago
  I think it is opensource no? https://github.com/httptoolkit https://github.com/httptoolkit
- Nodraak 7y ago
  Would not be able to exactly explain the features and differences, but https://mitmproxy.org/ https://mitmproxy.org/ is a similar project you might be interested in.
  - ORioN63 7y ago
    mitmproxy is indeed a similar project. It offers a dashboard similar to devtools (no modifying in there though). You can, however, write small python scripts that have read-write access to the http flow (request and response objects). I guess it's a more low level tool, because it doesn't provide mocking and other helpful tools out of the box.
    - chrisweekly 7y ago
      mitmproxy (/mitmdump) is awesome and powerful, and is ideal for integrating w/ other cli tools (like lnav) in chained scripts written in your language of choice. highly recommended.
deleted 7y ago
[deleted]
sebazzz 7y ago
How does this compare Fiddler?
- pimterry 7y ago
  It has all the same core functionality, plus: - It's open-source - It's cross-platform - It can do all the interception setup for you, for a single specific target. Fiddler et al set the system proxy, so there's a lot of other traffic going on, and you then need to make applications trust the certificate manually too. Notably some apps (like Node) ignore the system proxy settings entirely, which means you need code changes. HTTP Toolkit does some clever injection, and skips all of that. - Nice & modern UI - Built in docs for all standard HTTP status codes/headers/methods/etc - The Pro features: validation & inline docs from 1400+ API specifications, testing & comparison of compression options, explanations for how your response will be cached & why (and warnings). I keep meaning to do a proper comparison page (any day now), but that's the gist.
  - nachi 7y ago
    What about intercepting requests from other devices on the same network? Fiddler and Charles both let you set a proxy on Android/iOS devices pointing to your instance.
    - pimterry 7y ago
      HTTP Toolkit can definitely intercept Android & iOS or other network devices, it just can't do it for you automatically (yet. Watch this space: https://github.com/httptoolkit/feedback/issues/10 https://github.com/httptoolkit/feedback/issues/10). If you want to do intercept something manually, there's general instructions under the 'Anything' interception option. In short: - Set the proxy on the device to point at your machine's local IP, usually on port 8000 (active port is shown under the 'Anything' option) - Trust the generated MITM certificate on your device, so you can intercept HTTPS too (path to the certificate also shown under 'Anything'). Fundamentally, HTTP Toolkit, Charles & Fiddler are all HTTP & HTTPS proxies, so they all allow intercepting the same things, it's just a matter of how easy the setup process is.
barberousse 7y ago
Am getting too many redirects opening that link
- pimterry 7y ago
  Huh, that shouldn't happen! What browser & OS are you using?
  - kevinday 7y ago
    I'm getting the same on Chrome/Mac. Requesting /mock/ just 301's me back to /mock/ https://pastebin.com/vmgSU6nj https://pastebin.com/vmgSU6nj
    - pimterry 7y ago
      Hmm, no idea. I'm using Chrome on a Mac, and it works fine, and my analytics suggest at least 400 other Chrome+Mac users have been getting through, so it's not a general problem... Any chance you can export a HAR from your browser dev tools and send it to help@httptoolkit.tech? There's something very odd happening there, could be a Netlify bug.
      - kevinday 7y ago
        It's working now - nothing changed as far as I know on my end, but if it happens again I'll do that.
        pimterry 7y ago
        How bizarre, might be a Netlify hiccup somewhere? No idea, but I'm glad it's working now!
swiley 7y ago
THIS would have been nice to have at the beginning of the summer. The tests I wrote tested the boundary between my code and the HTTP library but the API I was writing against wasn't particularly well behaved and so that wasn't quite enough.
meritt 7y ago
How is this achieving https interception with zero setup? Does that not require a system-level root CA to be installed and trusted?
- pimterry 7y ago
  > Does that not require a system-level root CA to be installed and trusted? No, happily :-) The trick is that it starts the application to be intercepted for you, so it can control it a little. It then does some magic to get that specific instance of the application to trust the certificate. There's a lot going on there, but as an example: Chrome has a --ignore-certificate-errors-spki-list to inject the hashes of extra CAs that can be trusted in this specific Chrome instance. When HTTP Toolkit starts a Chrome process, it adds that command line option, with the hash of your locally generated CA. There's nothing here with a lasting effect, and other running apps on your machine won't trust HTTP Toolkit unless you specifically configure them to. Only the processes spawned by HTTP Toolkit trust your CA, which avoids a lot of the downside of other similar tools like this.
  - debatem1 7y ago
    So, I'm confused. Does this work on a set of programs where you know eg the right command line parameters to give them or on all programs? For example, let's say I have a program which uses the openssl API to set a custom trust store. Do you mess with openssl to make sure your cert is in there, or will this break?
    - pimterry 7y ago
      It works on a set of programs, plus terminal interception that works on anything (more or less) that you spawn from the given terminal. For that terminal interception, HTTP Toolkit injects a lot of environment variables, including adding some wrappers to your path, which allow it to inject into a _lot_ of places. It sets `SSL_CERT_FILE` for example, which reconfigures the default CA for any processes started from that terminal that use OpenSSL. Again, all scoped to just this one terminal window. In the case where you specifically manage the trust store as you describe I suspect that'll break (the requests will fail, but they'll still appear in HTTP Toolkit as just 'Certificate rejected for ABC.com', without the full details). The vast majority of applications don't do that though; they rely on the environment to tell them who to trust, and HTTP Toolkit creates the environment. For specific cases that don't, I also include a bunch of extra overrides and injections, so if there's anything that comes up that doesn't fit it's fairly easy to work around it. It's all open-source, so you can look through the specifics if you like. The core terminal interception setup logic is https://github.com/httptoolkit/httptoolkit-server/blob/master/src/interceptors/terminal/fresh-terminal-interceptor.ts https://github.com/httptoolkit/httptoolkit-server/blob/maste.... That spawns a terminal and injects the environment variables from https://github.com/httptoolkit/httptoolkit-server/blob/master/src/interceptors/terminal/terminal-env-overrides.ts https://github.com/httptoolkit/httptoolkit-server/blob/maste..., which notably add a bunch of things from https://github.com/httptoolkit/httptoolkit-server/tree/master/overrides https://github.com/httptoolkit/httptoolkit-server/tree/maste... to your PATH/PYTHONPATH/RUBYLIB/etc.
hartator 7y ago
Love it. We definitely need a fiddler in macosx.
- gmanley 7y ago
  Charles is pretty good, though it’s paid. It’s definitely nice to have an opensource tool, though.
  - hartator 7y ago
    Charles is okay. But when you have been used to Fiddler is hard to go back.
chrismatheson 7y ago
Is this something that could be used in a CI pipeline as well?
- pimterry 7y ago
  As a UI desktop app, not really, although it depends what you're trying to do. However, the internals are all open-source too, so you can use them standalone and automate HTTP & HTTPS with that. Take a look at https://github.com/httptoolkit/mockttp https://github.com/httptoolkit/mockttp
  - chrismatheson 7y ago
    I think (maybe postman) had a nice gui, and then the same mocks could be loaded up as part of a test. I was thinking more along the lines of how the application actually inspects network traffic, is that something that could be setup & torn down within a unprivileged docker container for example?
    - pimterry 7y ago
      Yes, that's exactly what you'd want Mockttp for. It's a Node.js library, so you'd need a Node container and a little script to configure it. Mockttp can spin up an HTTP/HTTPS proxy configured to pass through traffic in a couple of lines, and from there you can add any other mocking rules you want (the Mock tab here is really just a UI over this) and you can subscribe to requests/responses/whatever and report that data elsewhere in any form you'd like (that's where all the data shown in the View tab here comes from). The details get more complicated of course, but it's definitely possible.
- cdeutsch 7y ago
  You could try a cloud based solution like https://curlhub.io/ https://curlhub.io/
dessant 7y ago
Is intercepting Android traffic planned?
- pimterry 7y ago
  Yes, definitely! You can follow progress by subscribing to this GH issue: https://github.com/httptoolkit/feedback/issues/10 https://github.com/httptoolkit/feedback/issues/10 (or signing up to the general HTTP Toolkit mailing list ofc). In the meantime, it's quite possible to intercept Android traffic, it's just that you'll have to do the proxy & certificate setup yourself.
  - dessant 7y ago
    That's great, thanks for the link! In recent Android versions it has become difficult to inspect the traffic precisely because certificates installed by users are no longer trusted, so one has to either root the phone or modify the apk. Do you plan to automate app repackaging with the needed changes, or is there a better method for apps to trust local certificates?
    - pimterry 7y ago
      > Do you plan to automate app repackaging with the needed changes, or is there a better method for apps to trust local certificates? Eventually I'm aiming for the former, I'm fairly confident there is no better method. To start with, it'll be an Android app that configures your proxy settings (acting a VPN) and walks you through adding the certificate to the user store, and for your own apps I believe that's useful already: you just need to enable user stores in your app config (https://developer.android.com/training/articles/security-config#certificates https://developer.android.com/training/articles/security-con...), and everything will Just Work. For other apps though, on new-ish Android apps it's more difficult, as you say. My plan is to try and rewrite them, _probably_ as a Pro feature, but TBC. It should be a matter of: - Get hold of the app APK (I think you can pull it with some adb tricks, slightly unclear) - Edit the XML config to enable the user cert store, change the app id too (so it doesn't replace or have signature conflicts with the real app) - Repackage & reinstall the resulting APK over adb I haven't tested this yet though, so who knows!
mcx 7y ago
Would be cool to have something like this for debugging outgoing container traffic in Kubernetes.
- pimterry 7y ago
  Docker is on the list: https://github.com/httptoolkit/feedback/issues/1 https://github.com/httptoolkit/feedback/issues/1 (you can subscribe to that for updates). Would that work for your setup?
guardiangod 7y ago
Thanks for the tool. Is it possible for HTTP Mock to dump both the entire SSL session traffic, and the decrypted HTTP traffic (with fake tcp headers), to pcaps for future analysis?
jordache 7y ago
why do I need to create an account for this? gawd... I just need a Fiddler in macOS... one can't really improve what Fiddler provides.
- pimterry 7y ago
  You don't need to create an account for this! Click download, use the app, it's fine. You can provide your email after download if you want updates from the mailing list (or you can not), and you can click 'Get Pro' somewhere and provide your email if you want a paid Pro account. The core app will give you pretty much everything Fiddler does though, on Mac, and doesn't require your email or anything else.
bdcravens 7y ago
I understand why you're doing it, but the download link redirecting me to an email capture page is quite jarring. If your download link is at the top of your homepage, and there's more product details below that link, it's pretty typical to hit download and assume it's a normal download link while you read the rest of the page. An easily dismissible modal would work better (or put an email capture in your app on first-run)
- pimterry 7y ago
  Hmm, interesting. I really just did it without much thought, I didn't think it was particularly unusual behaviour. That said, this completely makes sense, I'm definitely going to look at that. I could fairly easily swap the 'sign up for updates' in directly in place of the download button when that's clicked I think, which would avoid the extra page and the modal all together, make everything much nicer, and still give keen users a clear opportunity to properly follow the project. Thanks! Great point.
  - gatherhunterer 7y ago
    I don't see too much of a problem, just that if the download link does not come up right away (this could be caused by a user's browser settings or connection speed, which are beyond your control) it looks like they have been taken to a "Here, have some spam instead" page. I would just switch the position and styling of "Sign up for updates" and "You are now downloading..." so that confirming the download request is the primary message on the screen. You could also ask a user to wait three seconds and make them look at your (now less prominent) appeal to sign-up for emails while they do so. The project looks very nice and I do a lot of HTTP work so I will try it out today. Good work!
- ElijahLynn 7y ago
  +1 for easy access download without forcing an email submission. update: K, I had gotten side tracked and forgot that it already downloaded and left me on the email submission page. User error. Downloads fine without email.
viana007 7y ago
For this type of HTTP intercept I like to use Fiddler and Charles.
jame_bkk 7y ago
How does HTTP Mock compare with Proxymam.io? Your tool is promising with zero-setup, but it works with different macOS app, doesn't it? I would try it tmr.
- nghiatran_feels 7y ago
  The first noticeable feature is that HTTP Mock is cross-platform and Proxyman is exclusively built on macOS. I'm not sure how zero-setup works, but it's cool feature indeed. I was frustrated to setup those certificate on Fiddler. I suppose that Fiddler tends to expert users, not me.
ElijahLynn 7y ago
Arch AUR package here everyone > https://aur.archlinux.org/packages/httptoolkit/ https://aur.archlinux.org/packages/httptoolkit/ `yay -S httptoolkit` AUR is 0.1.13 and latest is 0.1.14 fwiw.
- ElijahLynn 7y ago
  WOW. Just WOW. Easiest intercepter I have EVER used. (Charles, Fiddler, MITMProxy). After install I was able to edit requests and responses with the MOCK rules in no time. The language was confusing but I got through it and WOW. Nice job on the UI too, Linux UI's are hit or miss and HTTP Toolkit's is pleasant!
  - pimterry 7y ago
    Haha, thanks! Glad you like it. When you say 'the language was confusing' though, what do you mean? I should fix that.
    - ElijahLynn 7y ago
      It has to do around intercepting the response body and resume vs edit wording. I'll try to submit an issue soon with more details.
  - ElijahLynn 7y ago
    Alrighty, after tinkering yesterday an opportunity came up to use this for real work! And helped us mock testing for a bad JSON response we are seeing on a remote server, but mocked it locally for better error logging. (gave shoutout to HTTP Toolkit here > https://github.com/department-of-veterans-affairs/vets-website/pull/10827 https://github.com/department-of-veterans-affairs/vets-websi...) This just earned a subscription, will get it processed soon! Thanks again, a breeze to use!
ElijahLynn 7y ago
Totally nice side effect. Since this launches "fresh Chrome/Firefox". I can use it to read all these paid news sites that track how many "free articles" I have and don't have to worry about resetting all my cookies. Just launch HTTP Toolkit then Fresh Firefox/Chrome, paste link and done.
- meowface 7y ago
  You can also open the dev console and click "Clear site data". It won't evade complex tracking/fingerprinting techniques, but I don't think paywall sites use those to enforce the paywall (yet...).
kaushikt 7y ago
Is netflix's PollyJS also trying to solve this? https://netflix.github.io/pollyjs/#/ https://netflix.github.io/pollyjs/#/

341 points by pimterry 7y ago

Hey HN! I've been building this for a while, but up until now it's been a read-only HTTP debugging tool, and today's my first foray into proper mocking, which opens up a whole new world of fun (and practical use cases). Let me know what you think :-)

mateuszf 7y ago
Didn't try it yet but the promise is amazing.
pault 7y ago
I hope this works out! Using fiddler us such a frustrating experience. The documentation is practically non existent and the ux is atrocious, and as far as I can tell the closest thing to a manual is a paid ebook by the developer (I was so pissed when I discovered this). I would be very happy to have an open source alternative!
- pimterry 7y ago
  Me too :-). I built this myself in large part because of frustrations with Fiddler. It was an amazing tool at the time, but it hasn't aged well imo... Hopefully this is your open source alternative - give it a go, let me know if there's something Fiddler does for you that HTTP Toolkit doesn't!
- altano 7y ago
  If you think Fiddler sucks get a Mac and try using the tools available there like Charles Proxy. Ugh.
  - luckman212 7y ago
    Curious about this comment. I'm a Mac user and a Charles user. I find it to be a great tool. Why the negativity?
    - altano 7y ago
      It's extremely hard to use compared to something like Fiddler, especially when you are doing more than just tracing.
  - jame_bkk 7y ago
    Give Proxyman (https://proxyman.io/ https://proxyman.io/) a try, which is a better alternative for Fiddler and Charles Proxy.
  - stephenr 7y ago
    https://proxie.app/ https://proxie.app/ Works pretty well IMO.
- jmkni 7y ago
  I don't mind Fiddler, and $15 dollars for a book on a free, incredibly powerful, tool is a bargain IMHO.
  - pault 7y ago
    What bothers me is not the price, it's that the only documentation available is an ebook that one of the developers wrote as a side project. The tool is very powerful, but it's a nightmare to use, and it doesn't matter how powerful it is if you spend more time struggling with the software than fixing the problem you have that requires it in the first place. I'm just happy that someone is attempting to create an alternative with a focus on usability and cross platform support.
    - jmkni 7y ago
      Yeah fair, Fiddler is definitely well...fiddly, and a bit dated as well. Agree that it's good to see new alternatives.
RMPR 7y ago
It may be interesting to have a rpm package

mariocesar 7y ago

There are full opensource alternative for this? I like the product, I'm testing it right now. But I'm curious if there is a way to accomplish something like this with existing opensource tools or a similar full opensource project.

mateuszf 7y ago
I think you can use for example mitmproxy for this though with more setup
LongBallLary 7y ago
It is 100% open source according to their website (in the footer). https://github.com/httptoolkit https://github.com/httptoolkit
reffaelwallen 7y ago
I think it is opensource no? https://github.com/httptoolkit https://github.com/httptoolkit
Nodraak 7y ago
Would not be able to exactly explain the features and differences, but https://mitmproxy.org/ https://mitmproxy.org/ is a similar project you might be interested in.
- ORioN63 7y ago
  mitmproxy is indeed a similar project. It offers a dashboard similar to devtools (no modifying in there though). You can, however, write small python scripts that have read-write access to the http flow (request and response objects). I guess it's a more low level tool, because it doesn't provide mocking and other helpful tools out of the box.
  - chrisweekly 7y ago
    mitmproxy (/mitmdump) is awesome and powerful, and is ideal for integrating w/ other cli tools (like lnav) in chained scripts written in your language of choice. highly recommended.

deleted 7y ago

[deleted]

sebazzz 7y ago

How does this compare Fiddler?

pimterry 7y ago
It has all the same core functionality, plus: - It's open-source - It's cross-platform - It can do all the interception setup for you, for a single specific target. Fiddler et al set the system proxy, so there's a lot of other traffic going on, and you then need to make applications trust the certificate manually too. Notably some apps (like Node) ignore the system proxy settings entirely, which means you need code changes. HTTP Toolkit does some clever injection, and skips all of that. - Nice & modern UI - Built in docs for all standard HTTP status codes/headers/methods/etc - The Pro features: validation & inline docs from 1400+ API specifications, testing & comparison of compression options, explanations for how your response will be cached & why (and warnings). I keep meaning to do a proper comparison page (any day now), but that's the gist.
- nachi 7y ago
  What about intercepting requests from other devices on the same network? Fiddler and Charles both let you set a proxy on Android/iOS devices pointing to your instance.
  - pimterry 7y ago
    HTTP Toolkit can definitely intercept Android & iOS or other network devices, it just can't do it for you automatically (yet. Watch this space: https://github.com/httptoolkit/feedback/issues/10 https://github.com/httptoolkit/feedback/issues/10). If you want to do intercept something manually, there's general instructions under the 'Anything' interception option. In short: - Set the proxy on the device to point at your machine's local IP, usually on port 8000 (active port is shown under the 'Anything' option) - Trust the generated MITM certificate on your device, so you can intercept HTTPS too (path to the certificate also shown under 'Anything'). Fundamentally, HTTP Toolkit, Charles & Fiddler are all HTTP & HTTPS proxies, so they all allow intercepting the same things, it's just a matter of how easy the setup process is.

barberousse 7y ago

Am getting too many redirects opening that link

pimterry 7y ago
Huh, that shouldn't happen! What browser & OS are you using?
- kevinday 7y ago
  I'm getting the same on Chrome/Mac. Requesting /mock/ just 301's me back to /mock/ https://pastebin.com/vmgSU6nj https://pastebin.com/vmgSU6nj
  - pimterry 7y ago
    Hmm, no idea. I'm using Chrome on a Mac, and it works fine, and my analytics suggest at least 400 other Chrome+Mac users have been getting through, so it's not a general problem... Any chance you can export a HAR from your browser dev tools and send it to help@httptoolkit.tech? There's something very odd happening there, could be a Netlify bug.
    - kevinday 7y ago
      It's working now - nothing changed as far as I know on my end, but if it happens again I'll do that.
      - pimterry 7y ago
        How bizarre, might be a Netlify hiccup somewhere? No idea, but I'm glad it's working now!

swiley 7y ago

THIS would have been nice to have at the beginning of the summer. The tests I wrote tested the boundary between my code and the HTTP library but the API I was writing against wasn't particularly well behaved and so that wasn't quite enough.

meritt 7y ago

How is this achieving https interception with zero setup? Does that not require a system-level root CA to be installed and trusted?

pimterry 7y ago
> Does that not require a system-level root CA to be installed and trusted? No, happily :-) The trick is that it starts the application to be intercepted for you, so it can control it a little. It then does some magic to get that specific instance of the application to trust the certificate. There's a lot going on there, but as an example: Chrome has a --ignore-certificate-errors-spki-list to inject the hashes of extra CAs that can be trusted in this specific Chrome instance. When HTTP Toolkit starts a Chrome process, it adds that command line option, with the hash of your locally generated CA. There's nothing here with a lasting effect, and other running apps on your machine won't trust HTTP Toolkit unless you specifically configure them to. Only the processes spawned by HTTP Toolkit trust your CA, which avoids a lot of the downside of other similar tools like this.
- debatem1 7y ago
  So, I'm confused. Does this work on a set of programs where you know eg the right command line parameters to give them or on all programs? For example, let's say I have a program which uses the openssl API to set a custom trust store. Do you mess with openssl to make sure your cert is in there, or will this break?
  - pimterry 7y ago
    It works on a set of programs, plus terminal interception that works on anything (more or less) that you spawn from the given terminal. For that terminal interception, HTTP Toolkit injects a lot of environment variables, including adding some wrappers to your path, which allow it to inject into a _lot_ of places. It sets `SSL_CERT_FILE` for example, which reconfigures the default CA for any processes started from that terminal that use OpenSSL. Again, all scoped to just this one terminal window. In the case where you specifically manage the trust store as you describe I suspect that'll break (the requests will fail, but they'll still appear in HTTP Toolkit as just 'Certificate rejected for ABC.com', without the full details). The vast majority of applications don't do that though; they rely on the environment to tell them who to trust, and HTTP Toolkit creates the environment. For specific cases that don't, I also include a bunch of extra overrides and injections, so if there's anything that comes up that doesn't fit it's fairly easy to work around it. It's all open-source, so you can look through the specifics if you like. The core terminal interception setup logic is https://github.com/httptoolkit/httptoolkit-server/blob/master/src/interceptors/terminal/fresh-terminal-interceptor.ts https://github.com/httptoolkit/httptoolkit-server/blob/maste.... That spawns a terminal and injects the environment variables from https://github.com/httptoolkit/httptoolkit-server/blob/master/src/interceptors/terminal/terminal-env-overrides.ts https://github.com/httptoolkit/httptoolkit-server/blob/maste..., which notably add a bunch of things from https://github.com/httptoolkit/httptoolkit-server/tree/master/overrides https://github.com/httptoolkit/httptoolkit-server/tree/maste... to your PATH/PYTHONPATH/RUBYLIB/etc.

hartator 7y ago

Love it. We definitely need a fiddler in macosx.

gmanley 7y ago
Charles is pretty good, though it’s paid. It’s definitely nice to have an opensource tool, though.
- hartator 7y ago
  Charles is okay. But when you have been used to Fiddler is hard to go back.

chrismatheson 7y ago

Is this something that could be used in a CI pipeline as well?

pimterry 7y ago
As a UI desktop app, not really, although it depends what you're trying to do. However, the internals are all open-source too, so you can use them standalone and automate HTTP & HTTPS with that. Take a look at https://github.com/httptoolkit/mockttp https://github.com/httptoolkit/mockttp
- chrismatheson 7y ago
  I think (maybe postman) had a nice gui, and then the same mocks could be loaded up as part of a test. I was thinking more along the lines of how the application actually inspects network traffic, is that something that could be setup & torn down within a unprivileged docker container for example?
  - pimterry 7y ago
    Yes, that's exactly what you'd want Mockttp for. It's a Node.js library, so you'd need a Node container and a little script to configure it. Mockttp can spin up an HTTP/HTTPS proxy configured to pass through traffic in a couple of lines, and from there you can add any other mocking rules you want (the Mock tab here is really just a UI over this) and you can subscribe to requests/responses/whatever and report that data elsewhere in any form you'd like (that's where all the data shown in the View tab here comes from). The details get more complicated of course, but it's definitely possible.
cdeutsch 7y ago
You could try a cloud based solution like https://curlhub.io/ https://curlhub.io/

dessant 7y ago

Is intercepting Android traffic planned?

pimterry 7y ago
Yes, definitely! You can follow progress by subscribing to this GH issue: https://github.com/httptoolkit/feedback/issues/10 https://github.com/httptoolkit/feedback/issues/10 (or signing up to the general HTTP Toolkit mailing list ofc). In the meantime, it's quite possible to intercept Android traffic, it's just that you'll have to do the proxy & certificate setup yourself.
- dessant 7y ago
  That's great, thanks for the link! In recent Android versions it has become difficult to inspect the traffic precisely because certificates installed by users are no longer trusted, so one has to either root the phone or modify the apk. Do you plan to automate app repackaging with the needed changes, or is there a better method for apps to trust local certificates?
  - pimterry 7y ago
    > Do you plan to automate app repackaging with the needed changes, or is there a better method for apps to trust local certificates? Eventually I'm aiming for the former, I'm fairly confident there is no better method. To start with, it'll be an Android app that configures your proxy settings (acting a VPN) and walks you through adding the certificate to the user store, and for your own apps I believe that's useful already: you just need to enable user stores in your app config (https://developer.android.com/training/articles/security-config#certificates https://developer.android.com/training/articles/security-con...), and everything will Just Work. For other apps though, on new-ish Android apps it's more difficult, as you say. My plan is to try and rewrite them, _probably_ as a Pro feature, but TBC. It should be a matter of: - Get hold of the app APK (I think you can pull it with some adb tricks, slightly unclear) - Edit the XML config to enable the user cert store, change the app id too (so it doesn't replace or have signature conflicts with the real app) - Repackage & reinstall the resulting APK over adb I haven't tested this yet though, so who knows!

mcx 7y ago

Would be cool to have something like this for debugging outgoing container traffic in Kubernetes.

pimterry 7y ago
Docker is on the list: https://github.com/httptoolkit/feedback/issues/1 https://github.com/httptoolkit/feedback/issues/1 (you can subscribe to that for updates). Would that work for your setup?

guardiangod 7y ago

Thanks for the tool. Is it possible for HTTP Mock to dump both the entire SSL session traffic, and the decrypted HTTP traffic (with fake tcp headers), to pcaps for future analysis?

jordache 7y ago

why do I need to create an account for this? gawd... I just need a Fiddler in macOS... one can't really improve what Fiddler provides.

pimterry 7y ago
You don't need to create an account for this! Click download, use the app, it's fine. You can provide your email after download if you want updates from the mailing list (or you can not), and you can click 'Get Pro' somewhere and provide your email if you want a paid Pro account. The core app will give you pretty much everything Fiddler does though, on Mac, and doesn't require your email or anything else.

bdcravens 7y ago

I understand why you're doing it, but the download link redirecting me to an email capture page is quite jarring. If your download link is at the top of your homepage, and there's more product details below that link, it's pretty typical to hit download and assume it's a normal download link while you read the rest of the page. An easily dismissible modal would work better (or put an email capture in your app on first-run)

pimterry 7y ago
Hmm, interesting. I really just did it without much thought, I didn't think it was particularly unusual behaviour. That said, this completely makes sense, I'm definitely going to look at that. I could fairly easily swap the 'sign up for updates' in directly in place of the download button when that's clicked I think, which would avoid the extra page and the modal all together, make everything much nicer, and still give keen users a clear opportunity to properly follow the project. Thanks! Great point.
- gatherhunterer 7y ago
  I don't see too much of a problem, just that if the download link does not come up right away (this could be caused by a user's browser settings or connection speed, which are beyond your control) it looks like they have been taken to a "Here, have some spam instead" page. I would just switch the position and styling of "Sign up for updates" and "You are now downloading..." so that confirming the download request is the primary message on the screen. You could also ask a user to wait three seconds and make them look at your (now less prominent) appeal to sign-up for emails while they do so. The project looks very nice and I do a lot of HTTP work so I will try it out today. Good work!
ElijahLynn 7y ago
+1 for easy access download without forcing an email submission. update: K, I had gotten side tracked and forgot that it already downloaded and left me on the email submission page. User error. Downloads fine without email.

viana007 7y ago

For this type of HTTP intercept I like to use Fiddler and Charles.

jame_bkk 7y ago

How does HTTP Mock compare with Proxymam.io? Your tool is promising with zero-setup, but it works with different macOS app, doesn't it? I would try it tmr.

nghiatran_feels 7y ago
The first noticeable feature is that HTTP Mock is cross-platform and Proxyman is exclusively built on macOS. I'm not sure how zero-setup works, but it's cool feature indeed. I was frustrated to setup those certificate on Fiddler. I suppose that Fiddler tends to expert users, not me.

ElijahLynn 7y ago

Arch AUR package here everyone > https://aur.archlinux.org/packages/httptoolkit/ https://aur.archlinux.org/packages/httptoolkit/ `yay -S httptoolkit` AUR is 0.1.13 and latest is 0.1.14 fwiw.

ElijahLynn 7y ago
WOW. Just WOW. Easiest intercepter I have EVER used. (Charles, Fiddler, MITMProxy). After install I was able to edit requests and responses with the MOCK rules in no time. The language was confusing but I got through it and WOW. Nice job on the UI too, Linux UI's are hit or miss and HTTP Toolkit's is pleasant!
- pimterry 7y ago
  Haha, thanks! Glad you like it. When you say 'the language was confusing' though, what do you mean? I should fix that.
  - ElijahLynn 7y ago
    It has to do around intercepting the response body and resume vs edit wording. I'll try to submit an issue soon with more details.
- ElijahLynn 7y ago
  Alrighty, after tinkering yesterday an opportunity came up to use this for real work! And helped us mock testing for a bad JSON response we are seeing on a remote server, but mocked it locally for better error logging. (gave shoutout to HTTP Toolkit here > https://github.com/department-of-veterans-affairs/vets-website/pull/10827 https://github.com/department-of-veterans-affairs/vets-websi...) This just earned a subscription, will get it processed soon! Thanks again, a breeze to use!

ElijahLynn 7y ago

Totally nice side effect. Since this launches "fresh Chrome/Firefox". I can use it to read all these paid news sites that track how many "free articles" I have and don't have to worry about resetting all my cookies. Just launch HTTP Toolkit then Fresh Firefox/Chrome, paste link and done.

meowface 7y ago
You can also open the dev console and click "Clear site data". It won't evade complex tracking/fingerprinting techniques, but I don't think paywall sites use those to enforce the paywall (yet...).

kaushikt 7y ago

Is netflix's PollyJS also trying to solve this? https://netflix.github.io/pollyjs/#/ https://netflix.github.io/pollyjs/#/