4 ms·

It would be cool if something like this would be possible with standard Linux tools on the command line. How hard would it be to implement a network rule "if h

It would be cool if something like this would be possible with standard Linux tools on the command line. How hard would it be to implement a network rule "if http://abc.com/def http://abc.com/def is requested reply with this data: ..."? Or is it possible to inject something like this on the fly into Apache? Would be very nice to mock end-2-end tests.

marcosdumay 5y ago
I'm not sure what you are expecting that isn't a main feature of Apache. I imagine you expect something like a CGI script with mod-rewrite, but your comment only actually requires plain Apache. A network rule of "if URL is requested reply with this data" is implemented by putting a file at the expected place.
- dotancohen 5y ago
  > only actually requires plain Apache ... and a hosts file entry. Still, trivial for any machine that the operator administers.
cedricvanrompay 5y ago
Note that HTTPS, unlike says VPNs (IPSec, Wireguard...), provides authenticity protection up to the application itself. If the kernel sends a modified packet to the application, it will be rejected by the application. As a result, to implement HTTPS interception / rewrite / injection you need some degree of modification of the application itself. The "minimal" way is to add a new TLS certificate to the certificate trust store the application uses that is marked as "allowed for every domain" (that's what Burp suite does). It seems that HTTP toolkit does it differently for the browsers it supports, probably a plugin/extension added to the browser that alters the traffic after the TLS block (HTTPS is HTTP over TLS)
- cjcampbell 5y ago
  No plug-in or extension. I dug in when I first learned about it because I wanted to make sure I understood how it would impact browser security when it was not in use. The strategy differs per browser, but essentially they launch the supported browsers with a self-signed certificate and a custom profile. No configuration changes needed for the browser, which is really nice.
pimterry 5y ago
I'm the author of HTTP Toolkit, I actually built the internals much earlier as an open-source library (Mockttp: https://github.com/httptoolkit/mockttp https://github.com/httptoolkit/mockttp) designed for exactly the end-to-end testing mocking use case you're talking about. It's MIT-licensed, and you can build an automated HTTP/HTTPS rewriting proxy using that in a handful of lines of JS, and script any kind of transformations or inject any responses you like. There's a general guide to getting started here: https://httptoolkit.tech/blog/javascript-mitm-proxy-mockttp/ https://httptoolkit.tech/blog/javascript-mitm-proxy-mockttp/. For the more general interactive testing/debugging case, you can also use HTTP Toolkit itself (it has a rules builder for this kind of thing) but if you're building automation you should just use the internals directly, they have exactly the same capabilities. HTTP Toolkit just provides a UI and convenient interception setup tools over the top.
- intpx 5y ago
  Is there an internet law for when you spend weeks looking for a framework or library to solve a problem and it only reveals itself after your organization makes a really bad choice to do something else? I have been looking for exactly this kind of local proxy to dynamically hack headers and dynamically spoof responses for an internal app. This would have been the perfect starting place...
  - kingcharles 5y ago
    Oh lord, this should be a Law. You only find the exact framework or app you've been looking for exactly one day after you've finished implementing your own solution from whole cloth.
- inglor 5y ago
  This is great! I've been looking for this sort of thing today. It's very helpful and also nice to see some projects I maintain or contribute to in the dependencies list :) You might want to consider migrating from node-abort-controller to native AbortController by the way.

marcosdumay 5y ago

I'm not sure what you are expecting that isn't a main feature of Apache. I imagine you expect something like a CGI script with mod-rewrite, but your comment only actually requires plain Apache. A network rule of "if URL is requested reply with this data" is implemented by putting a file at the expected place.

dotancohen 5y ago
> only actually requires plain Apache ... and a hosts file entry. Still, trivial for any machine that the operator administers.

cedricvanrompay 5y ago

Note that HTTPS, unlike says VPNs (IPSec, Wireguard...), provides authenticity protection up to the application itself. If the kernel sends a modified packet to the application, it will be rejected by the application. As a result, to implement HTTPS interception / rewrite / injection you need some degree of modification of the application itself. The "minimal" way is to add a new TLS certificate to the certificate trust store the application uses that is marked as "allowed for every domain" (that's what Burp suite does). It seems that HTTP toolkit does it differently for the browsers it supports, probably a plugin/extension added to the browser that alters the traffic after the TLS block (HTTPS is HTTP over TLS)

cjcampbell 5y ago
No plug-in or extension. I dug in when I first learned about it because I wanted to make sure I understood how it would impact browser security when it was not in use. The strategy differs per browser, but essentially they launch the supported browsers with a self-signed certificate and a custom profile. No configuration changes needed for the browser, which is really nice.

pimterry 5y ago

I'm the author of HTTP Toolkit, I actually built the internals much earlier as an open-source library (Mockttp: https://github.com/httptoolkit/mockttp https://github.com/httptoolkit/mockttp) designed for exactly the end-to-end testing mocking use case you're talking about. It's MIT-licensed, and you can build an automated HTTP/HTTPS rewriting proxy using that in a handful of lines of JS, and script any kind of transformations or inject any responses you like. There's a general guide to getting started here: https://httptoolkit.tech/blog/javascript-mitm-proxy-mockttp/ https://httptoolkit.tech/blog/javascript-mitm-proxy-mockttp/. For the more general interactive testing/debugging case, you can also use HTTP Toolkit itself (it has a rules builder for this kind of thing) but if you're building automation you should just use the internals directly, they have exactly the same capabilities. HTTP Toolkit just provides a UI and convenient interception setup tools over the top.

intpx 5y ago
Is there an internet law for when you spend weeks looking for a framework or library to solve a problem and it only reveals itself after your organization makes a really bad choice to do something else? I have been looking for exactly this kind of local proxy to dynamically hack headers and dynamically spoof responses for an internal app. This would have been the perfect starting place...
- kingcharles 5y ago
  Oh lord, this should be a Law. You only find the exact framework or app you've been looking for exactly one day after you've finished implementing your own solution from whole cloth.
inglor 5y ago
This is great! I've been looking for this sort of thing today. It's very helpful and also nice to see some projects I maintain or contribute to in the dependencies list :) You might want to consider migrating from node-abort-controller to native AbortController by the way.