4 ms·

How mitmproxy works

120 points by veb 13y ago

tptacek 13y ago
This is a great, informative post; I'd just point out that generating on-the-fly certs is a basic functionality of all the web app testing tools, Burp Suite being the most famous. If you haven't looked at Burp Suite and, unlike Aldo Cortesi, can stomach running a Java program (I don't like Java either, but...), Burp is both the industry standard tool used by appsec testing for web work and pretty inexpensive for what it does.
- cortesi 13y ago
  I run plenty of Java every day without breaking out in hives. I don't particularly enjoy _writing_ Java, though, and the most natural way to extend the Java interceptors is to use Java (Jython notwithstanding). I could have been clearer about this in my post. I should also add - Burp is a fine tool, but I would hope there's room for more than a single "industry standard" in this area. Mitmproxy has some comparative strengths, is completely open, and is improving rapidly. I'd hate for people to be discouraged from trying it out and helping to make it better.
- kanzure 13y ago
  Well, mitmproxy isn't really just a Burp replacement. I've used both Burp and libmproxy for app sec reasons, but when I had strange reasons to write my own caching server for my crawler infrastructure, I dropped libmproxy in, wrote out some controller logic on top, and it just worked. I've been laughing every step to the bank ever since. (bias: submitted a few patches to mitmproxy.git)
- hohumdevdrum 13y ago
  I can't even stomach running a Python "program" (I call it a script), if I don't have to. But consider you can't even compile OpenSSL without have Perl 5. SSL is such a huge waste of time. I prefer crypto solutions that are easy to use; that focus on a single thing instead of trying to do try to do many things; that are designed and implemented by one or a select few cryptographers who are know to be competent, not designed ad-hoc by a committee. But this is only my opinion.
WatchDog 13y ago
From the documentation, the mitmdump utility is likened to tcpdump. Can mitmdump work without being configured as a proxy/router, ie: using libpcap/tcpdump? I guess it wouldn't really be mitm in this regard, but manipulating the traffic is not always needed. Currently I use httpry to watch http traffic without interfering with its flow, but httpry is fairly limited in its functionality. By the way, very impressive piece of software. Its nice to have more quality open tools like this.
- cortesi 13y ago
  [edit: I actually misread your question. We don't at the moment have reassembly of HTTP flows from pcaps in base, but there's an external project that does this. As another comment noted, transparent mode is probably still your best bet. https://github.com/cjneasbi/mitmextract https://github.com/cjneasbi/mitmextract] Yes, it's very similar to tcpdump in this respect. You can write flows to disk like this: mitmdump -w outfile And then read them back: mitmdump -r outfile All the standard options for modifying, replaying and filtering are available when you read saved flows. So you could do this: mitmdump -r outfile -s foo.py -w newfile Which will read flows from file, run the script foo.py over each flow as its read (which can then modify them arbitrarily), and then write the result to newfile.
  - WatchDog 13y ago
    Transparent mode is great but invasive, and mitmextract is fine for non real-time analysis. Sometimes you might want to analyze traffic on production systems and changing routes is not an option. What I would love to see is a way of doing a non-invasive, realtime processing of http traffic, eg when a specific post request is made, trigger some job.
    - cortesi 13y ago
      That would be nice indeed. Mitmextract uses libnids, which can do stream reassembly on the fly. It's not such a big step going from reassembling a pcap file to doing the same thing on network traffic.
- 13y ago
SkippyZA 13y ago
mitmproxy = Malcolm in the Middle Proxy?
- bvdbijl 13y ago
  Mitm=man in the middle
mogui 13y ago
Great article really! I develop mobile application and you don't know how your software is indispensable for me :) i use it quite everyday really appreciated
mogui 13y ago
Great article really! I develop mobile application and you don't know how your software is indispensable for me :) i use it quite everyday really appreciated

120 points by veb 13y ago

This is a great, informative post; I'd just point out that generating on-the-fly certs is a basic functionality of all the web app testing tools, Burp Suite being the most famous. If you haven't looked at Burp Suite and, unlike Aldo Cortesi, can stomach running a Java program (I don't like Java either, but...), Burp is both the industry standard tool used by appsec testing for web work and pretty inexpensive for what it does.

cortesi 13y ago
I run plenty of Java every day without breaking out in hives. I don't particularly enjoy _writing_ Java, though, and the most natural way to extend the Java interceptors is to use Java (Jython notwithstanding). I could have been clearer about this in my post. I should also add - Burp is a fine tool, but I would hope there's room for more than a single "industry standard" in this area. Mitmproxy has some comparative strengths, is completely open, and is improving rapidly. I'd hate for people to be discouraged from trying it out and helping to make it better.
kanzure 13y ago
Well, mitmproxy isn't really just a Burp replacement. I've used both Burp and libmproxy for app sec reasons, but when I had strange reasons to write my own caching server for my crawler infrastructure, I dropped libmproxy in, wrote out some controller logic on top, and it just worked. I've been laughing every step to the bank ever since. (bias: submitted a few patches to mitmproxy.git)
hohumdevdrum 13y ago
I can't even stomach running a Python "program" (I call it a script), if I don't have to. But consider you can't even compile OpenSSL without have Perl 5. SSL is such a huge waste of time. I prefer crypto solutions that are easy to use; that focus on a single thing instead of trying to do try to do many things; that are designed and implemented by one or a select few cryptographers who are know to be competent, not designed ad-hoc by a committee. But this is only my opinion.

WatchDog 13y ago

From the documentation, the mitmdump utility is likened to tcpdump. Can mitmdump work without being configured as a proxy/router, ie: using libpcap/tcpdump? I guess it wouldn't really be mitm in this regard, but manipulating the traffic is not always needed. Currently I use httpry to watch http traffic without interfering with its flow, but httpry is fairly limited in its functionality. By the way, very impressive piece of software. Its nice to have more quality open tools like this.

cortesi 13y ago
[edit: I actually misread your question. We don't at the moment have reassembly of HTTP flows from pcaps in base, but there's an external project that does this. As another comment noted, transparent mode is probably still your best bet. https://github.com/cjneasbi/mitmextract https://github.com/cjneasbi/mitmextract] Yes, it's very similar to tcpdump in this respect. You can write flows to disk like this: mitmdump -w outfile And then read them back: mitmdump -r outfile All the standard options for modifying, replaying and filtering are available when you read saved flows. So you could do this: mitmdump -r outfile -s foo.py -w newfile Which will read flows from file, run the script foo.py over each flow as its read (which can then modify them arbitrarily), and then write the result to newfile.
- WatchDog 13y ago
  Transparent mode is great but invasive, and mitmextract is fine for non real-time analysis. Sometimes you might want to analyze traffic on production systems and changing routes is not an option. What I would love to see is a way of doing a non-invasive, realtime processing of http traffic, eg when a specific post request is made, trigger some job.
  - cortesi 13y ago
    That would be nice indeed. Mitmextract uses libnids, which can do stream reassembly on the fly. It's not such a big step going from reassembling a pcap file to doing the same thing on network traffic.
13y ago

SkippyZA 13y ago

mitmproxy = Malcolm in the Middle Proxy?

bvdbijl 13y ago
Mitm=man in the middle

mogui 13y ago

Great article really! I develop mobile application and you don't know how your software is indispensable for me :) i use it quite everyday really appreciated

mogui 13y ago

Great article really! I develop mobile application and you don't know how your software is indispensable for me :) i use it quite everyday really appreciated