4 ms·

How is this achieving https interception with zero setup? Does that not require a system-level root CA to be installed and trusted?

by meritt 7y ago

pimterry 7y ago
> Does that not require a system-level root CA to be installed and trusted? No, happily :-) The trick is that it starts the application to be intercepted for you, so it can control it a little. It then does some magic to get that specific instance of the application to trust the certificate. There's a lot going on there, but as an example: Chrome has a --ignore-certificate-errors-spki-list to inject the hashes of extra CAs that can be trusted in this specific Chrome instance. When HTTP Toolkit starts a Chrome process, it adds that command line option, with the hash of your locally generated CA. There's nothing here with a lasting effect, and other running apps on your machine won't trust HTTP Toolkit unless you specifically configure them to. Only the processes spawned by HTTP Toolkit trust your CA, which avoids a lot of the downside of other similar tools like this.
- debatem1 7y ago
  So, I'm confused. Does this work on a set of programs where you know eg the right command line parameters to give them or on all programs? For example, let's say I have a program which uses the openssl API to set a custom trust store. Do you mess with openssl to make sure your cert is in there, or will this break?
  - pimterry 7y ago
    It works on a set of programs, plus terminal interception that works on anything (more or less) that you spawn from the given terminal. For that terminal interception, HTTP Toolkit injects a lot of environment variables, including adding some wrappers to your path, which allow it to inject into a _lot_ of places. It sets `SSL_CERT_FILE` for example, which reconfigures the default CA for any processes started from that terminal that use OpenSSL. Again, all scoped to just this one terminal window. In the case where you specifically manage the trust store as you describe I suspect that'll break (the requests will fail, but they'll still appear in HTTP Toolkit as just 'Certificate rejected for ABC.com', without the full details). The vast majority of applications don't do that though; they rely on the environment to tell them who to trust, and HTTP Toolkit creates the environment. For specific cases that don't, I also include a bunch of extra overrides and injections, so if there's anything that comes up that doesn't fit it's fairly easy to work around it. It's all open-source, so you can look through the specifics if you like. The core terminal interception setup logic is https://github.com/httptoolkit/httptoolkit-server/blob/master/src/interceptors/terminal/fresh-terminal-interceptor.ts https://github.com/httptoolkit/httptoolkit-server/blob/maste.... That spawns a terminal and injects the environment variables from https://github.com/httptoolkit/httptoolkit-server/blob/master/src/interceptors/terminal/terminal-env-overrides.ts https://github.com/httptoolkit/httptoolkit-server/blob/maste..., which notably add a bunch of things from https://github.com/httptoolkit/httptoolkit-server/tree/master/overrides https://github.com/httptoolkit/httptoolkit-server/tree/maste... to your PATH/PYTHONPATH/RUBYLIB/etc.
    - debatem1 7y ago
      At that point why not just LD_PRELOAD and hook the creation of all cert stores?
      - pimterry 7y ago
        I did look at it! It's still an option in my back pocket if I find things that don't work nicely, but right now this is much easier. I'd need to use LD_PRELOAD to hook lots of different libraries and tools (OpenSSL is just the start, and I also need to hook proxy settings), plus handle different versions of the hooked libs, and make it work reliably on Windows & Linux & Mac. AFAICT, it's hard to do everywhere reliably for the general case, though as I say I might use it for specific niche cases later. At the moment there's zero native code required in this terminal hooking, which is very nice, and it's very nearly identical logic for all platforms too. It's not so bad, right now it's less than 1000 lines of code in total, and it works out of the box for Python/Node/Ruby/PHP/curl/npm/apt-get, etc etc etc.
        atleta 7y ago
        You could maybe catch file open and read operations and return your CA cert if/when the proxied app tries to read the cert files.
        kingosticks 7y ago
        An LD_PRELOAD approach would be useful for working around that flavour of certificate 'pinning' where the cert files are embedded in the application rather read from the filesystem. I don't know how common that is compared to the other pinning methods (public key pinning, hashing) but maybe you could make it work for those also. I guess you'd have to implement it for each individual library but does any other (free) mitm proxy offer this feature?
miles 7y ago
Netripper[0] can also intercept HTTPS with zero setup via DLL injection. I cobbled together a simple guide[1] on its use, though for Chrome, signature changes have made it a bit more challenging[2]. [0] https://github.com/NytroRST/NetRipper https://github.com/NytroRST/NetRipper [1] https://tinyapps.org/blog/201508150700_sniffing_encrypted_traffic.html https://tinyapps.org/blog/201508150700_sniffing_encrypted_tr... [2] https://github.com/NytroRST/NetRipper/issues/21 https://github.com/NytroRST/NetRipper/issues/21

How is this achieving https interception with zero setup? Does that not require a system-level root CA to be installed and trusted?

by meritt 7y ago

How is this achieving https interception with zero setup? Does that not require a system-level root CA to be installed and trusted?

pimterry 7y ago

> Does that not require a system-level root CA to be installed and trusted? No, happily :-) The trick is that it starts the application to be intercepted for you, so it can control it a little. It then does some magic to get that specific instance of the application to trust the certificate. There's a lot going on there, but as an example: Chrome has a --ignore-certificate-errors-spki-list to inject the hashes of extra CAs that can be trusted in this specific Chrome instance. When HTTP Toolkit starts a Chrome process, it adds that command line option, with the hash of your locally generated CA. There's nothing here with a lasting effect, and other running apps on your machine won't trust HTTP Toolkit unless you specifically configure them to. Only the processes spawned by HTTP Toolkit trust your CA, which avoids a lot of the downside of other similar tools like this.

debatem1 7y ago
So, I'm confused. Does this work on a set of programs where you know eg the right command line parameters to give them or on all programs? For example, let's say I have a program which uses the openssl API to set a custom trust store. Do you mess with openssl to make sure your cert is in there, or will this break?
- pimterry 7y ago
  It works on a set of programs, plus terminal interception that works on anything (more or less) that you spawn from the given terminal. For that terminal interception, HTTP Toolkit injects a lot of environment variables, including adding some wrappers to your path, which allow it to inject into a _lot_ of places. It sets `SSL_CERT_FILE` for example, which reconfigures the default CA for any processes started from that terminal that use OpenSSL. Again, all scoped to just this one terminal window. In the case where you specifically manage the trust store as you describe I suspect that'll break (the requests will fail, but they'll still appear in HTTP Toolkit as just 'Certificate rejected for ABC.com', without the full details). The vast majority of applications don't do that though; they rely on the environment to tell them who to trust, and HTTP Toolkit creates the environment. For specific cases that don't, I also include a bunch of extra overrides and injections, so if there's anything that comes up that doesn't fit it's fairly easy to work around it. It's all open-source, so you can look through the specifics if you like. The core terminal interception setup logic is https://github.com/httptoolkit/httptoolkit-server/blob/master/src/interceptors/terminal/fresh-terminal-interceptor.ts https://github.com/httptoolkit/httptoolkit-server/blob/maste.... That spawns a terminal and injects the environment variables from https://github.com/httptoolkit/httptoolkit-server/blob/master/src/interceptors/terminal/terminal-env-overrides.ts https://github.com/httptoolkit/httptoolkit-server/blob/maste..., which notably add a bunch of things from https://github.com/httptoolkit/httptoolkit-server/tree/master/overrides https://github.com/httptoolkit/httptoolkit-server/tree/maste... to your PATH/PYTHONPATH/RUBYLIB/etc.
  - debatem1 7y ago
    At that point why not just LD_PRELOAD and hook the creation of all cert stores?
    - pimterry 7y ago
      I did look at it! It's still an option in my back pocket if I find things that don't work nicely, but right now this is much easier. I'd need to use LD_PRELOAD to hook lots of different libraries and tools (OpenSSL is just the start, and I also need to hook proxy settings), plus handle different versions of the hooked libs, and make it work reliably on Windows & Linux & Mac. AFAICT, it's hard to do everywhere reliably for the general case, though as I say I might use it for specific niche cases later. At the moment there's zero native code required in this terminal hooking, which is very nice, and it's very nearly identical logic for all platforms too. It's not so bad, right now it's less than 1000 lines of code in total, and it works out of the box for Python/Node/Ruby/PHP/curl/npm/apt-get, etc etc etc.
      - atleta 7y ago
        You could maybe catch file open and read operations and return your CA cert if/when the proxied app tries to read the cert files.
      - kingosticks 7y ago
        An LD_PRELOAD approach would be useful for working around that flavour of certificate 'pinning' where the cert files are embedded in the application rather read from the filesystem. I don't know how common that is compared to the other pinning methods (public key pinning, hashing) but maybe you could make it work for those also. I guess you'd have to implement it for each individual library but does any other (free) mitm proxy offer this feature?

miles 7y ago

Netripper[0] can also intercept HTTPS with zero setup via DLL injection. I cobbled together a simple guide[1] on its use, though for Chrome, signature changes have made it a bit more challenging[2]. [0] https://github.com/NytroRST/NetRipper https://github.com/NytroRST/NetRipper [1] https://tinyapps.org/blog/201508150700_sniffing_encrypted_traffic.html https://tinyapps.org/blog/201508150700_sniffing_encrypted_tr... [2] https://github.com/NytroRST/NetRipper/issues/21 https://github.com/NytroRST/NetRipper/issues/21