4 ms·

This question is a bit naive, but outside of Facebook, can you think of what kind of application this is well suited for?

swartkrans 12y ago
Any kind of internal service that listens and accepts an API. Like you could put a bunch of monitoring agents on all your machines and those agents get information about your machines and report back to the service that makes use of Proxygen. Your service would use Proxygen instead of hiding behind an nginx or apache server and running via some kind of fcgi setup. The agents themselves could be using the Proxygen client to talk to the service using Proxygen server pieces.
dcsommer 12y ago
Well besides the fun of hacking around and building little HTTP servers, I could see this being useful if you want to save money by running fewer instances of your HTTP service. For instance, if you have a widely deployed python webservice that isn't scaling well, you could rewrite it in C++ with very little boilerplate using proxygen's httpserver. It's early stages for the proxygen open source project. Maybe further down the road we'll provide off-the-shelf binaries, but we think the library is already interesting enough to warrant a release.
- johnbellone 12y ago
  Is that the inception of the project? I am more curious about the lineage. How was the decision made to go this route instead of throwing more instances at it?
  - dcsommer 12y ago
    The blog post goes into more detail, but Proxygen started with an effort to write a L7 reverse proxy that could deeply integrate into FB internal services. We pulled out a lot of the non-FB specific stuff into this open source release. Before that, we used hardware load balancers for this role, which was expensive.
    - nbm 12y ago
      By expensive, not just capital costs, but costs around operating them - they weren't as reliably configurable, health-checkable, and instrumentable as we'd want, and Proxygen (and a later L4 load balancer) were. Also the previous load balancers had constraints we weren't willing to accept - they required special connectivity to our networks, we could not use particular combinations of options, and we had to rely on vendors to solve problems that most of their customers were not encountering and/or able to detect.
      - johnbellone 12y ago
        Yeah, we are hitting the same wall right now and we've been going down the same route using HAProxy/Chef. Our plan is to put an API in front of it and treat it similar to an ELB. Are you still using hardware load balancers for SSL termination?
        nbm 12y ago
        The machines running Proxygen do the TLS termination.
    - cmbaus 12y ago
      That's interesting. I had been working something similar years ago. I eventually open sourced it, but discontinued work on it. Http://github.com/baus/swithflow I'm surprised more systems don't take this approach of doing more work in L7 proxies.
    - e12e 12y ago
      I'll have a look at the blog for more -- but this was something that couldn't be solved with haproxy and/or trafficserver?
      - nbm 12y ago
        haproxy only got TLS support in 2012, whereas work on proxygen began quite before that. In addition, haproxy does not support SPDY natively (although it can forward it to something that does). Apache Traffic Server only got SPDY support in a release about 45 days ago according to their release notes. There are many other relatively basic features besides those, and similar considerations exist for the other projects that existed back then (and even now).
deleted 12y ago
[deleted]

swartkrans 12y ago

Any kind of internal service that listens and accepts an API. Like you could put a bunch of monitoring agents on all your machines and those agents get information about your machines and report back to the service that makes use of Proxygen. Your service would use Proxygen instead of hiding behind an nginx or apache server and running via some kind of fcgi setup. The agents themselves could be using the Proxygen client to talk to the service using Proxygen server pieces.

dcsommer 12y ago

Well besides the fun of hacking around and building little HTTP servers, I could see this being useful if you want to save money by running fewer instances of your HTTP service. For instance, if you have a widely deployed python webservice that isn't scaling well, you could rewrite it in C++ with very little boilerplate using proxygen's httpserver. It's early stages for the proxygen open source project. Maybe further down the road we'll provide off-the-shelf binaries, but we think the library is already interesting enough to warrant a release.

johnbellone 12y ago
Is that the inception of the project? I am more curious about the lineage. How was the decision made to go this route instead of throwing more instances at it?
- dcsommer 12y ago
  The blog post goes into more detail, but Proxygen started with an effort to write a L7 reverse proxy that could deeply integrate into FB internal services. We pulled out a lot of the non-FB specific stuff into this open source release. Before that, we used hardware load balancers for this role, which was expensive.
  - nbm 12y ago
    By expensive, not just capital costs, but costs around operating them - they weren't as reliably configurable, health-checkable, and instrumentable as we'd want, and Proxygen (and a later L4 load balancer) were. Also the previous load balancers had constraints we weren't willing to accept - they required special connectivity to our networks, we could not use particular combinations of options, and we had to rely on vendors to solve problems that most of their customers were not encountering and/or able to detect.
    - johnbellone 12y ago
      Yeah, we are hitting the same wall right now and we've been going down the same route using HAProxy/Chef. Our plan is to put an API in front of it and treat it similar to an ELB. Are you still using hardware load balancers for SSL termination?
      - nbm 12y ago
        The machines running Proxygen do the TLS termination.
  - cmbaus 12y ago
    That's interesting. I had been working something similar years ago. I eventually open sourced it, but discontinued work on it. Http://github.com/baus/swithflow I'm surprised more systems don't take this approach of doing more work in L7 proxies.
  - e12e 12y ago
    I'll have a look at the blog for more -- but this was something that couldn't be solved with haproxy and/or trafficserver?
    - nbm 12y ago
      haproxy only got TLS support in 2012, whereas work on proxygen began quite before that. In addition, haproxy does not support SPDY natively (although it can forward it to something that does). Apache Traffic Server only got SPDY support in a release about 45 days ago according to their release notes. There are many other relatively basic features besides those, and similar considerations exist for the other projects that existed back then (and even now).

deleted 12y ago

[deleted]