6 ms·

Pingora: build fast, reliable and programmable networked systems

245 points by KajMagnus 3y ago

deleted 3y ago
[deleted]
lewisl9029 3y ago
Have been looking forward to this release for quite a while! Huge props to the Cloudflare team for putting this out there! I've been operating a cluster of NGINX nodes on Fly.io and using njs (NGINX's custom JS scripting engine) for all of my custom routing logic, and have been really feeling the limitations (had to spin up a separate companion app in nodejs to work around some of these). Having access to the entirety of the Rust language and ecosystem to customize routing behavior sounds incredibly compelling! I did a quick scan over the codebase and couldn't see anything around disk caching like in NGINX, only memory caching. Curious if Cloudflare is operating all their production nodes with memory caching as opposed to disk caching at the moment? I'd love to see an option for disk caching for use cases that are a bit more cost sensitive.
- jpgvm 3y ago
  Their cache is likely an entirely different per-PoP service that operates as a network service.
samin5489 3y ago
[dead]
Sytten 3y ago
I had been waiting on that for a couple years since they first talked about it in their blog. Congrats!
- KajMagnus 3y ago
  Me too! Nginx (written in C) sometimes have can't-happen-in-Rust major severity vulnerabilities, one just recently: https://nginx.org/en/security_advisories.html https://nginx.org/en/security_advisories.html (related to HTTP3)
  - ramp6 3y ago
    Bad take, Those where in preview features (aka should not be used in prod and behind a feature gate)
ttrrooppeerr 3y ago
Their, at the time, blog post announcement about Pingora: https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/ https://blog.cloudflare.com/how-we-built-pingora-the-proxy-t...
- dukeoptamus 3y ago
  The current announcement: https://blog.cloudflare.com/pingora-open-source https://blog.cloudflare.com/pingora-open-source
- mcpherrinm 3y ago
  And a current blog post announcing the Open-sourcing: https://blog.cloudflare.com/pingora-open-source https://blog.cloudflare.com/pingora-open-source
eptcyka 3y ago
Would be cool of it was possible to use this with rustls. But I assume they didn't use it since i rustls o ly focuses on TLS 1.3 and does not support anything lower.
- mcpherrinm 3y ago
  Being able to use rustls as a drop-in replacement for openssl is on their roadmap: https://github.com/rustls/rustls/blob/main/ROADMAP.md#future-priorities https://github.com/rustls/rustls/blob/main/ROADMAP.md#future... So that'll certainly one option in the future.
  - heinrich5991 3y ago
    Permalink (press 'y' anywhere on GitHub): https://github.com/rustls/rustls/blob/425b5272901d60a4e3d18d4f17d5d7291c331272/ROADMAP.md#future-priorities https://github.com/rustls/rustls/blob/425b5272901d60a4e3d18d....
- phicoh 3y ago
  Rustls claims to support TLS 1.2 as well (https://github.com/rustls/rustls https://github.com/rustls/rustls)
diarrhea 3y ago
Very cool. Would play around with this in a heartbeat in the home lab, but Caddy's automatic HTTPS (ACME HTTP-01 and DNS-01 challenges) would be sorely missed. I suppose one can supplement certbot in this setup? Or use the "highly programmable" APIs Pingora provides directly?
- hermanradtke 3y ago
  cert it via cron that uses the graceful upgrade functionality should work https://github.com/cloudflare/pingora/blob/main/docs/quick_start.md#gracefully-upgrade-the-service https://github.com/cloudflare/pingora/blob/main/docs/quick_s...
sanxiyn 3y ago
I heard that when Cloudbleed happened, Cloudflare CTO declared that there will be no more new projects in C or C++, and that's how it happened. Needless to say, lots of time and money were wasted.
- pornel 3y ago
  Before Rust, most features had to be built on top of nginx, and had to carefully balance performance overhead of Lua (openresty) vs risks and maintenance costs of C modules or patching nginx itself. The switch to Rust has been very positive overall, because it allowed Cloudflare to tackle much more ambitious projects, and own its entire stack.
- pas 3y ago
  > Needless to say, lots of time and money were wasted. it might be needless, but for those of us not in the know, could you elaborate on that please? what ended up as waste? why? altogether was it worth it?
  - diarrhea 3y ago
    This person seems to be a Rust contributor, so if this isn't a jab against Rust, I'm not sure what their point is.
  - sanxiyn 3y ago
    These kinds of top down directives are almost always wasteful, because they ignore local situations. It could have been case by case decisions, considering cost and benefit. Now I understand why this is politically difficult, but it still resulted in suboptimal technical decisions, and ill will against Rust, which is unfortunate.
    - deleted 3y ago
      [deleted]
    - rapsey 3y ago
      I would imagine cloudbleed shook the company severely. I doubt the decision to abandon C/C++ caused a lot of ill will within the company. Considering the level of exposure Cloudflare software has, it is I think a very level headed decision.
inapis 3y ago
so in a quick glance, this does not look like nginx/caddy. It is not a binary you download, install and just configure to talk to your upstream servers. Rather a set of packages which you assemble to fulfill a particular use case. You basically end up writing a "new" reverse proxy just for your use case. Not sure how useful this would be for anyone except very large businesses or someone like cloudflare itself.
- mcpherrinm 3y ago
  River is going to be a fully-featured reverse proxy product built on top of Pingora: https://www.memorysafety.org/blog/introducing-river/ https://www.memorysafety.org/blog/introducing-river/ It's mentioned briefly in the Cloudflare blog post. I'd expect that if you're a current nginx or other proxy, River is what you'd want as an alternative. One of the big things I'm excited about in the River plan is the ability to use wasm for scripting, which should make it relatively easy to extend.
  - derekperkins 3y ago
    I'm surprised Envoy isn't mentioned anywhere
  - mkeedlinger 3y ago
    Can't wait! I've currently settled on Caddy as my reverse proxy, but something rust based and extended with wasm would be great
- wongarsu 3y ago
  From their quickstart and user guide it doesn't look too verbose. For anyone whose config has reached the point where you are wondering whether it would be easier to express in code, this might be a good product to try. Or if you run shared hosting or some other scenario where you are effectively maintaining your server configuration twice: once for the actual server and once in the database for your dashboard and other services. This would allow you to just read the config from the canonical source, instead of risking bugs when the two configs diverge.
- 3y ago
dang 3y ago
Related: Cloudflare has replaced Nginx with in-house, Rust-written Pingora - https://news.ycombinator.com/item?id=32864119 https://news.ycombinator.com/item?id=32864119 - Sept 2022 (141 comments) Pingora, the proxy that connects Cloudflare to the Internet - https://news.ycombinator.com/item?id=32836661 https://news.ycombinator.com/item?id=32836661 - Sept 2022 (123 comments)
jamesear 3y ago
Cloudflare work on some cool stuff! It's a shame they don't hire for software engineer positions in Australia.
- adamch 3y ago
  There are some CF SWEs in Australia. But few teams are willing to work across 3 timezones. The SRE teams might be more open to hiring, as there's a big SRE presence in Singapore.
bogwog 3y ago
https://www.spanishdict.com/translate/pinga https://www.spanishdict.com/translate/pinga
- petsfed 3y ago
  https://en.wikipedia.org/wiki/Pingora_Peak https://en.wikipedia.org/wiki/Pingora_Peak Its an indigenous word, but probably not the indigenous name for the mountain. I've heard it means "high, rocky and inaccessible peak" in Shoshoni. Popo Agie (pronounced puh-poe-zha) Tower was proposed, but there are dozens of Popo Agie things already (the Popo Agie river runs from the Wind River range, through Sinks Canyon into nearby Lander, Wyoming), so the USGS insisted on a more distinct name.
throwaway63467 3y ago
Nice! It seems HTTP/3 isn’t mentioned or supported, which is weird as they offer HTTP/3 support in their cloud service and have their own QUIC implementation. Haven’t looked in detail so maybe it’s just missing from the readme or they didn’t open-source it (yet).
- gideontong 3y ago
  HTTP/3 is on the roadmap!
- pornel 3y ago
  Cloudflare uses QUIC on the browser<->cdn side, but Pingora sits on the cdn<->server side. That side of the connection usually isn't going over slow and lossy mobile networks, so QUIC isn't that useful there.
  - deleted 3y ago
    [deleted]

245 points by KajMagnus 3y ago

deleted 3y ago

[deleted]

Have been looking forward to this release for quite a while! Huge props to the Cloudflare team for putting this out there! I've been operating a cluster of NGINX nodes on Fly.io and using njs (NGINX's custom JS scripting engine) for all of my custom routing logic, and have been really feeling the limitations (had to spin up a separate companion app in nodejs to work around some of these). Having access to the entirety of the Rust language and ecosystem to customize routing behavior sounds incredibly compelling! I did a quick scan over the codebase and couldn't see anything around disk caching like in NGINX, only memory caching. Curious if Cloudflare is operating all their production nodes with memory caching as opposed to disk caching at the moment? I'd love to see an option for disk caching for use cases that are a bit more cost sensitive.

jpgvm 3y ago
Their cache is likely an entirely different per-PoP service that operates as a network service.

samin5489 3y ago

[dead]

Sytten 3y ago

I had been waiting on that for a couple years since they first talked about it in their blog. Congrats!

KajMagnus 3y ago
Me too! Nginx (written in C) sometimes have can't-happen-in-Rust major severity vulnerabilities, one just recently: https://nginx.org/en/security_advisories.html https://nginx.org/en/security_advisories.html (related to HTTP3)
- ramp6 3y ago
  Bad take, Those where in preview features (aka should not be used in prod and behind a feature gate)

ttrrooppeerr 3y ago

Their, at the time, blog post announcement about Pingora: https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/ https://blog.cloudflare.com/how-we-built-pingora-the-proxy-t...

dukeoptamus 3y ago
The current announcement: https://blog.cloudflare.com/pingora-open-source https://blog.cloudflare.com/pingora-open-source
mcpherrinm 3y ago
And a current blog post announcing the Open-sourcing: https://blog.cloudflare.com/pingora-open-source https://blog.cloudflare.com/pingora-open-source

eptcyka 3y ago

Would be cool of it was possible to use this with rustls. But I assume they didn't use it since i rustls o ly focuses on TLS 1.3 and does not support anything lower.

mcpherrinm 3y ago
Being able to use rustls as a drop-in replacement for openssl is on their roadmap: https://github.com/rustls/rustls/blob/main/ROADMAP.md#future-priorities https://github.com/rustls/rustls/blob/main/ROADMAP.md#future... So that'll certainly one option in the future.
- heinrich5991 3y ago
  Permalink (press 'y' anywhere on GitHub): https://github.com/rustls/rustls/blob/425b5272901d60a4e3d18d4f17d5d7291c331272/ROADMAP.md#future-priorities https://github.com/rustls/rustls/blob/425b5272901d60a4e3d18d....
phicoh 3y ago
Rustls claims to support TLS 1.2 as well (https://github.com/rustls/rustls https://github.com/rustls/rustls)

diarrhea 3y ago

Very cool. Would play around with this in a heartbeat in the home lab, but Caddy's automatic HTTPS (ACME HTTP-01 and DNS-01 challenges) would be sorely missed. I suppose one can supplement certbot in this setup? Or use the "highly programmable" APIs Pingora provides directly?

hermanradtke 3y ago
cert it via cron that uses the graceful upgrade functionality should work https://github.com/cloudflare/pingora/blob/main/docs/quick_start.md#gracefully-upgrade-the-service https://github.com/cloudflare/pingora/blob/main/docs/quick_s...

sanxiyn 3y ago

I heard that when Cloudbleed happened, Cloudflare CTO declared that there will be no more new projects in C or C++, and that's how it happened. Needless to say, lots of time and money were wasted.

pornel 3y ago
Before Rust, most features had to be built on top of nginx, and had to carefully balance performance overhead of Lua (openresty) vs risks and maintenance costs of C modules or patching nginx itself. The switch to Rust has been very positive overall, because it allowed Cloudflare to tackle much more ambitious projects, and own its entire stack.
pas 3y ago
> Needless to say, lots of time and money were wasted. it might be needless, but for those of us not in the know, could you elaborate on that please? what ended up as waste? why? altogether was it worth it?
- diarrhea 3y ago
  This person seems to be a Rust contributor, so if this isn't a jab against Rust, I'm not sure what their point is.
- sanxiyn 3y ago
  These kinds of top down directives are almost always wasteful, because they ignore local situations. It could have been case by case decisions, considering cost and benefit. Now I understand why this is politically difficult, but it still resulted in suboptimal technical decisions, and ill will against Rust, which is unfortunate.
  - deleted 3y ago
    [deleted]
  - rapsey 3y ago
    I would imagine cloudbleed shook the company severely. I doubt the decision to abandon C/C++ caused a lot of ill will within the company. Considering the level of exposure Cloudflare software has, it is I think a very level headed decision.

inapis 3y ago

so in a quick glance, this does not look like nginx/caddy. It is not a binary you download, install and just configure to talk to your upstream servers. Rather a set of packages which you assemble to fulfill a particular use case. You basically end up writing a "new" reverse proxy just for your use case. Not sure how useful this would be for anyone except very large businesses or someone like cloudflare itself.

mcpherrinm 3y ago
River is going to be a fully-featured reverse proxy product built on top of Pingora: https://www.memorysafety.org/blog/introducing-river/ https://www.memorysafety.org/blog/introducing-river/ It's mentioned briefly in the Cloudflare blog post. I'd expect that if you're a current nginx or other proxy, River is what you'd want as an alternative. One of the big things I'm excited about in the River plan is the ability to use wasm for scripting, which should make it relatively easy to extend.
- derekperkins 3y ago
  I'm surprised Envoy isn't mentioned anywhere
- mkeedlinger 3y ago
  Can't wait! I've currently settled on Caddy as my reverse proxy, but something rust based and extended with wasm would be great
wongarsu 3y ago
From their quickstart and user guide it doesn't look too verbose. For anyone whose config has reached the point where you are wondering whether it would be easier to express in code, this might be a good product to try. Or if you run shared hosting or some other scenario where you are effectively maintaining your server configuration twice: once for the actual server and once in the database for your dashboard and other services. This would allow you to just read the config from the canonical source, instead of risking bugs when the two configs diverge.
3y ago

dang 3y ago

Related: Cloudflare has replaced Nginx with in-house, Rust-written Pingora - https://news.ycombinator.com/item?id=32864119 https://news.ycombinator.com/item?id=32864119 - Sept 2022 (141 comments) Pingora, the proxy that connects Cloudflare to the Internet - https://news.ycombinator.com/item?id=32836661 https://news.ycombinator.com/item?id=32836661 - Sept 2022 (123 comments)

jamesear 3y ago

Cloudflare work on some cool stuff! It's a shame they don't hire for software engineer positions in Australia.

adamch 3y ago
There are some CF SWEs in Australia. But few teams are willing to work across 3 timezones. The SRE teams might be more open to hiring, as there's a big SRE presence in Singapore.

bogwog 3y ago

https://www.spanishdict.com/translate/pinga https://www.spanishdict.com/translate/pinga

petsfed 3y ago
https://en.wikipedia.org/wiki/Pingora_Peak https://en.wikipedia.org/wiki/Pingora_Peak Its an indigenous word, but probably not the indigenous name for the mountain. I've heard it means "high, rocky and inaccessible peak" in Shoshoni. Popo Agie (pronounced puh-poe-zha) Tower was proposed, but there are dozens of Popo Agie things already (the Popo Agie river runs from the Wind River range, through Sinks Canyon into nearby Lander, Wyoming), so the USGS insisted on a more distinct name.

throwaway63467 3y ago

Nice! It seems HTTP/3 isn’t mentioned or supported, which is weird as they offer HTTP/3 support in their cloud service and have their own QUIC implementation. Haven’t looked in detail so maybe it’s just missing from the readme or they didn’t open-source it (yet).

gideontong 3y ago
HTTP/3 is on the roadmap!
pornel 3y ago
Cloudflare uses QUIC on the browser<->cdn side, but Pingora sits on the cdn<->server side. That side of the connection usually isn't going over slow and lossy mobile networks, so QUIC isn't that useful there.
- deleted 3y ago
  [deleted]