3 ms·

SwiperProxy: Fast open-source web proxy

41 points by Argure 12y ago

dspillett 12y ago
Might be of interest to me. A detailed bullet-point list of features (available an obvious click or two from the front page - I appreciate you might not want that sort of detail on the front page in order to keep it clean) would be useful for assessing it though, i.e. does it support SNI (so it could proxy for several HTTPS end-points using a single IPv4 address), IPv6, and so forth. Also, benchmarks against using nginx and other common options as a proxy would be handy, though I get the impression from the blurb on the front page that ease of installation/configuration/maintenance is more of a priority than speed (not that this is a problem - getting that side of things right is a most laudable goal)
- Argure 12y ago
  SNI is supported in the sense that, it will work fine when using it to browse to a server that has multiple https vhosts with different x.509 certificates. The proxy itself does not do anything with SNI. I'm assuming that if your use case is to point multiple domains at a single proxy address, you should probably be using a more advanced httpd than python's BaseHTTPServer. ipv6 is supported, and you can go to any AAAA-only hostname just fine. Writing up a feature list and running benchmarks are some good ideas, thanks. :) Edit: I should probably mention, the live demo is heavily ratelimited as an easy way of conserving bandwidth. This way, it is more than fast enough for browsing pretty much anything, but makes it a less interesting for mass downloading large files and streaming video. It can still do that if you don't rate limit, obviously. https://swiperproxy.github.io/documentation/sys-shorewall.html https://swiperproxy.github.io/documentation/sys-shorewall.ht...
- sciurus 12y ago
  > benchmarks against using nginx and other common options as a proxy would be handy, Unless I'm missing something that's not a possible comparison. Nginx is a reverse proxy, but this is a forward proxy.
  - dspillett 12y ago
    In that case I misunderstood in my quick glance. I'll look again later...
ponytech 12y ago
Buy a short domain name and here you have a very easy tool for phishing.
- ponytech 12y ago
  Downvoted ? Go to http://proxy.donttrustrobots.nl/www.facebook.com/ http://proxy.donttrustrobots.nl/www.facebook.com/ and tell me if your grandma would see the difference
  - gnrlbzik 12y ago
    Good point though, not sure it applies well to this post though.
  - Argure 12y ago
    Having it show the exact same page to the end user is the entire point. There are plenty of proxies that hijack your session to show you their own stuff (usually ads), and what I wanted was explicitly not that. The only things the proxy will change in terms of content are URL rewrites so that you won't accidentally leave the proxy. Of course it's entirely possible for whoever runs a proxy to sniff all of your data; that's unavoidable. There's a big warning by default on the index page for this exact reason. I'm not overly concerned about phishing though. If it happens, it happens, whether or not I publish proxy software. And really, even if I did have the proxy modify the DOM to generate huge warnings, what's the point? Anyone who wants to use this as a phishing tool can just as easily remove it. That, and the average phishing victim usually doesn't look at warnings, but will click past them. There are already better phishing methods in place., and phishing is going to be a problem regardless of this software's existence and regardless of what it does.
SwellJoe 12y ago
So, what's the use case? There are a lot of proxies, and some of them are fast, and they tend to have a niche where they are strong. Why did you make it? (That's not an existential question, I'm assuming you had a use case that wasn't well served by existing tools, and knowing that would help me understand what I might use it for.)
- Argure 12y ago
  There's a lot of proxies, yes, and most of them either sell whatever data they can to advertisers and then hijack your browsing to show those same advertisements. Those that don't, will usually try to sell you some premium service or VPN. Which I'm totally fine with, running these things for a lot of people doesn't come cheap and if that's your business model then hey, go for it. I don't particularly want to use them though, and the only way that's avoidable is going self-hosted. With self-hosted proxies your list of available options shortens dramatically. Sure, there's all kinds of VPN's, SOCKS proxies, hell, even Tor, but all of those require you to have people configure things on their machine. The first test it needed to pass, was sending it to my grandparents and see if they could use it without any explanation whatsoever. And then, why stop there. Lets make it as easy for the sysadmin to deploy as well. Hence why it immediately runs as a fully functional webserver. There's a good few advanced things you can do with it, yes, and all of them are either already documented or being documented right now. But at the end of the day, any novice sysadmin should be able to clone and have a functional proxy within five minutes out of the box. These two things are something I haven't seen any other proxy do so far and that's the use case: a very simple web proxy for both the end user and the person running it. At this point I feel it strikes a very nice balance between simplicity and having plenty of features and customization.
  - SwellJoe 12y ago
    I don't think you've really answered my question (I still don't feel like I know why I would want to use this project, anyway). What do you use it for? You've told me why it's better than other proxies that you would use in the same circumstance that would lead to you using your proxy (which I'm getting the feeling means, "proxies that allow me to access data unavailable directly from where I am, or that allow me to access data more anonymously than I can currently access directly", but nothing like that is mentioned on the front page). So, I believe you that it's easy to setup. Why would I set it up? I'm not trying to be ornery, and I'm not proxy illiterate, I worked on Squid for a decade, and have built my own toy proxies to test ideas. I'm trying to convince you to make your website more clear about why someone would use your software. You can't just call it a "proxy" and expect understanding. Even when talking to someone who has deployed 1000+ proxies, you're talking past me. There are a lot of proxies that do many things. What is yours for? What do you want people to do with it? There are caching proxies to make slow links seem faster and use less bandwidth, there are security-oriented proxies that anonymize to one degree or another, there are proxies to allow users to safely tunnel through a firewall in a locked down corporate environment, there are reverse proxies for scaling and accelerating websites, there are proxies for content filtering and access logging used in schools and libraries and such, there are proxies that simulate different network conditions to aid developers, there are proxies for sniffing user data illicitly, there are proxies to allow people in other countries to watch YouTube or Netflix. There are all sorts of proxies. What is yours for?
_mikz 12y ago
I made something similar many years ago: https://github.com/mikz/rProxy https://github.com/mikz/rProxy I went with this approach because I wanted to use it with Opera Mini and it had no support for normal proxy protocol. Why I wanted to send all my traffic through yet different proxy? Simply, I wanted to change the web app I was using to find bus connections so it had bookmarks and could remember where is my home, so I don't have to type it over and over again. So I made this XML transformation language that behaved quite like jQuery so I could move elements on the site and have some state between requests. https://github.com/mikz/rProxy/blob/master/app/plugins/isis_schedule/actions.xml https://github.com/mikz/rProxy/blob/master/app/plugins/isis_... https://github.com/mikz/rProxy/blob/master/app/plugins/dpp/actions.xml https://github.com/mikz/rProxy/blob/master/app/plugins/dpp/a...
mdaniel 12y ago
Just as a friendly heads up to those who use the 8.8.8.8 family of DNS resolvers, it is currently not resolving any query for donttrustrobots.nl (no "A", no "NS" and definitely no "proxy.donttrustrobots.nl"). Using the DNS on AWS returns the correct answer.
- Argure 12y ago
  I've been dealing with LeaseWeb's incompetence on adding my DNSSEC key to the nl. zone. After days of explaining that I run my own nameservers and that theirs are irrelevant, they finally managed to add... the bloody wrong key. http://dnsviz.net/d/donttrustrobots.nl/dnssec/ http://dnsviz.net/d/donttrustrobots.nl/dnssec/ Edit: I've removed dnssec from that domain in the mean time. Should be reachable on any dnssec validating resolver as well when it has propagated.

41 points by Argure 12y ago

Might be of interest to me. A detailed bullet-point list of features (available an obvious click or two from the front page - I appreciate you might not want that sort of detail on the front page in order to keep it clean) would be useful for assessing it though, i.e. does it support SNI (so it could proxy for several HTTPS end-points using a single IPv4 address), IPv6, and so forth. Also, benchmarks against using nginx and other common options as a proxy would be handy, though I get the impression from the blurb on the front page that ease of installation/configuration/maintenance is more of a priority than speed (not that this is a problem - getting that side of things right is a most laudable goal)

Argure 12y ago
SNI is supported in the sense that, it will work fine when using it to browse to a server that has multiple https vhosts with different x.509 certificates. The proxy itself does not do anything with SNI. I'm assuming that if your use case is to point multiple domains at a single proxy address, you should probably be using a more advanced httpd than python's BaseHTTPServer. ipv6 is supported, and you can go to any AAAA-only hostname just fine. Writing up a feature list and running benchmarks are some good ideas, thanks. :) Edit: I should probably mention, the live demo is heavily ratelimited as an easy way of conserving bandwidth. This way, it is more than fast enough for browsing pretty much anything, but makes it a less interesting for mass downloading large files and streaming video. It can still do that if you don't rate limit, obviously. https://swiperproxy.github.io/documentation/sys-shorewall.html https://swiperproxy.github.io/documentation/sys-shorewall.ht...
sciurus 12y ago
> benchmarks against using nginx and other common options as a proxy would be handy, Unless I'm missing something that's not a possible comparison. Nginx is a reverse proxy, but this is a forward proxy.
- dspillett 12y ago
  In that case I misunderstood in my quick glance. I'll look again later...

ponytech 12y ago

Buy a short domain name and here you have a very easy tool for phishing.

ponytech 12y ago
Downvoted ? Go to http://proxy.donttrustrobots.nl/www.facebook.com/ http://proxy.donttrustrobots.nl/www.facebook.com/ and tell me if your grandma would see the difference
- gnrlbzik 12y ago
  Good point though, not sure it applies well to this post though.
- Argure 12y ago
  Having it show the exact same page to the end user is the entire point. There are plenty of proxies that hijack your session to show you their own stuff (usually ads), and what I wanted was explicitly not that. The only things the proxy will change in terms of content are URL rewrites so that you won't accidentally leave the proxy. Of course it's entirely possible for whoever runs a proxy to sniff all of your data; that's unavoidable. There's a big warning by default on the index page for this exact reason. I'm not overly concerned about phishing though. If it happens, it happens, whether or not I publish proxy software. And really, even if I did have the proxy modify the DOM to generate huge warnings, what's the point? Anyone who wants to use this as a phishing tool can just as easily remove it. That, and the average phishing victim usually doesn't look at warnings, but will click past them. There are already better phishing methods in place., and phishing is going to be a problem regardless of this software's existence and regardless of what it does.

SwellJoe 12y ago

So, what's the use case? There are a lot of proxies, and some of them are fast, and they tend to have a niche where they are strong. Why did you make it? (That's not an existential question, I'm assuming you had a use case that wasn't well served by existing tools, and knowing that would help me understand what I might use it for.)

Argure 12y ago
There's a lot of proxies, yes, and most of them either sell whatever data they can to advertisers and then hijack your browsing to show those same advertisements. Those that don't, will usually try to sell you some premium service or VPN. Which I'm totally fine with, running these things for a lot of people doesn't come cheap and if that's your business model then hey, go for it. I don't particularly want to use them though, and the only way that's avoidable is going self-hosted. With self-hosted proxies your list of available options shortens dramatically. Sure, there's all kinds of VPN's, SOCKS proxies, hell, even Tor, but all of those require you to have people configure things on their machine. The first test it needed to pass, was sending it to my grandparents and see if they could use it without any explanation whatsoever. And then, why stop there. Lets make it as easy for the sysadmin to deploy as well. Hence why it immediately runs as a fully functional webserver. There's a good few advanced things you can do with it, yes, and all of them are either already documented or being documented right now. But at the end of the day, any novice sysadmin should be able to clone and have a functional proxy within five minutes out of the box. These two things are something I haven't seen any other proxy do so far and that's the use case: a very simple web proxy for both the end user and the person running it. At this point I feel it strikes a very nice balance between simplicity and having plenty of features and customization.
- SwellJoe 12y ago
  I don't think you've really answered my question (I still don't feel like I know why I would want to use this project, anyway). What do you use it for? You've told me why it's better than other proxies that you would use in the same circumstance that would lead to you using your proxy (which I'm getting the feeling means, "proxies that allow me to access data unavailable directly from where I am, or that allow me to access data more anonymously than I can currently access directly", but nothing like that is mentioned on the front page). So, I believe you that it's easy to setup. Why would I set it up? I'm not trying to be ornery, and I'm not proxy illiterate, I worked on Squid for a decade, and have built my own toy proxies to test ideas. I'm trying to convince you to make your website more clear about why someone would use your software. You can't just call it a "proxy" and expect understanding. Even when talking to someone who has deployed 1000+ proxies, you're talking past me. There are a lot of proxies that do many things. What is yours for? What do you want people to do with it? There are caching proxies to make slow links seem faster and use less bandwidth, there are security-oriented proxies that anonymize to one degree or another, there are proxies to allow users to safely tunnel through a firewall in a locked down corporate environment, there are reverse proxies for scaling and accelerating websites, there are proxies for content filtering and access logging used in schools and libraries and such, there are proxies that simulate different network conditions to aid developers, there are proxies for sniffing user data illicitly, there are proxies to allow people in other countries to watch YouTube or Netflix. There are all sorts of proxies. What is yours for?

_mikz 12y ago

I made something similar many years ago: https://github.com/mikz/rProxy https://github.com/mikz/rProxy I went with this approach because I wanted to use it with Opera Mini and it had no support for normal proxy protocol. Why I wanted to send all my traffic through yet different proxy? Simply, I wanted to change the web app I was using to find bus connections so it had bookmarks and could remember where is my home, so I don't have to type it over and over again. So I made this XML transformation language that behaved quite like jQuery so I could move elements on the site and have some state between requests. https://github.com/mikz/rProxy/blob/master/app/plugins/isis_schedule/actions.xml https://github.com/mikz/rProxy/blob/master/app/plugins/isis_... https://github.com/mikz/rProxy/blob/master/app/plugins/dpp/actions.xml https://github.com/mikz/rProxy/blob/master/app/plugins/dpp/a...

mdaniel 12y ago

Just as a friendly heads up to those who use the 8.8.8.8 family of DNS resolvers, it is currently not resolving any query for donttrustrobots.nl (no "A", no "NS" and definitely no "proxy.donttrustrobots.nl"). Using the DNS on AWS returns the correct answer.

Argure 12y ago
I've been dealing with LeaseWeb's incompetence on adding my DNSSEC key to the nl. zone. After days of explaining that I run my own nameservers and that theirs are irrelevant, they finally managed to add... the bloody wrong key. http://dnsviz.net/d/donttrustrobots.nl/dnssec/ http://dnsviz.net/d/donttrustrobots.nl/dnssec/ Edit: I've removed dnssec from that domain in the mean time. Should be reachable on any dnssec validating resolver as well when it has propagated.