20 ms·
Cloudflare has replaced Nginx with in-house, Rust-written Pingora
- m00dy 4y agoso anyone knows Tokio is production ready ?
- m00dy 4y agoI think io_uring support [0] still is not production ready. [0]: https://github.com/tokio-rs/tokio-uring#project-status https://github.com/tokio-rs/tokio-uring#project-status
- TotoHorner 4y agoOriginal post on Cloudflare blog -> https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/ https://blog.cloudflare.com/how-we-built-pingora-the-proxy-t...
- asicsp 4y agoDiscussion (106 comments): https://news.ycombinator.com/item?id=32836661 https://news.ycombinator.com/item?id=32836661
- dredmorbius 4y agoTwo days ago. (I was going to share that link as well...)
- deleted 4y ago[deleted]
- GRBLDeveloped 4y agoRust in the title? Straight to the front page. Jokes aside that's some serious performance boost. Wonder what it'll perform like in everyone else's prod environment.
- ugjka 4y agoIt is not just the performance boost, but the fact they can support any non-rfc compliant bullshit there is on the web they feel like is necessary.
- silentsea90 4y agoI wonder if as part of the rollout, they selectively routed non-rfc compliant connections to nginx and get 90% of the way there with their changes while they added more support
- kosolam 4y agosource. very interesting
- jgrahamc 4y agoWe are going to open source it.
- anonymoushn 4y agoThis is an exciting prospect. Some financial exchanges are currently proxying their order entry and market data through Cloudflare, so it's useful for their customers to understand how the proxy works.
- blibble 4y agocare to elaborate? I've worked with essentially every exchange in the world and I've never seen http anywhere near order entry or market data
- anonymoushn 4y agoBinance does a modest 50 billion dollars of daily volume.
- kosolam 4y agoGood :)
- kosolam 4y agoBtw, I’d like cf to have a feature that will prevent anyone (including cf) from viewing my traffic. Is this in plans? I want the protection of cf but don’t want to sacrifice privacy of my customers data..
- jgrahamc 4y agohttps://www.cloudflare.com/products/cloudflare-spectrum/ https://www.cloudflare.com/products/cloudflare-spectrum/
- londons_explore 4y agoAt cloudflare-scale, I think I would want a proxy that sits at least partially in the network hardware. Ie. so that the bytes of a large image or video that is coming from an origin server and being sent to a client never has to pass through the system RAM or CPU. I'd probably implement this as mods to the sendfile() API so that a certain number of bytes can be copied from one socket to another, with that request going all the way to the firmware of the network card which will do the actual work. It probably needs to work with HTTP/3 UDP and encryption too - so decrypt the data from this socket and send it to that socket reencrypted with this other key and packetized to this HTTP/3 stream. The firmware would need some way to do aborts/timeouts and kick a partially complete request back to software too. Is it complex...? Yes. But will the compute savings be worth it...? At cloudflare scale, I think the answer is yes.
- jgrahamc 4y agoWe constantly think about the possibilities of using exotic hardware for acceleration. So far we've got a very, very long way with "commodity" hardware and the Linux kernel. One day, we'll probably do something exotic. https://blog.cloudflare.com/cloudflares-gen-x-servers-for-an-accelerated-future/ https://blog.cloudflare.com/cloudflares-gen-x-servers-for-an... https://blog.cloudflare.com/tubular-fixing-the-socket-api-with-ebpf/ https://blog.cloudflare.com/tubular-fixing-the-socket-api-wi...
- junon 4y agoJust curious, is there a high cost overhead (aside from the cost of actual units) of e.g. custom silicon?
- jgrahamc 4y agoIt’s just a question of “is it worth it?”. How much does this cool accelerator thing cost? How much does it save?
- Thaxll 4y agoWhy not using the same kind of solution as Netflix does where they offload TLS and the like in hardware? https://www.nvidia.com/en-us/networking/ethernet/connectx-6-dx/ https://www.nvidia.com/en-us/networking/ethernet/connectx-6-...
- lbriner 4y ago"Also considered safer"? The article is interesting and the solution great but saying it is safer because Rust is misleading. I wrote a Rust milter for Postfix incorrectly and although it didn't crash as such, it completely didn't work because data was coming into it from outside. i.e. it is still possible to write broken code in Rust. I've personally never considered nginx unsafe either but tbh, I am not exactly an nginx guru on the cutting edge of performance or load.
- bad416f1f5a2 4y ago> The article is interesting and the solution great but saying it is safer because Rust is misleading. Using Rust (without unsafe) precludes memory management bugs being turned into exploits. That closes an entire category of attack. Sure, “safety” can be defined along different axes, but Rust handles one entirely. How does that not make it safer than using a memory unsafe language?
- allendoerfer 4y ago> How does that not make it safer than using a memory unsafe language? It was never compared to using a memory unsafe language. It was compared to a wildly used and battle-tested open source software.
- sophacles 4y agoPeople die wearing seatbelts. They are still safer than not wearing seatbelts.
- npalli 4y agoWas curious if the Cloudfare team evaluated other languages for this effort (I thought they used Golang quite a bit at the company).
- jgrahamc 4y agoWe use Go and Rust and C++ and all sorts of things. I wasn't involved in selecting Rust for this project but in general we've seen Rust be a very good fit for things that might long ago have been written in C. https://blog.cloudflare.com/boringtun-userspace-wireguard-rust/ https://blog.cloudflare.com/boringtun-userspace-wireguard-ru... https://blog.cloudflare.com/building-cloudflare-images-in-rust-and-cloudflare-workers/ https://blog.cloudflare.com/building-cloudflare-images-in-ru... https://blog.cloudflare.com/enjoy-a-slice-of-quic-and-rust/ https://blog.cloudflare.com/enjoy-a-slice-of-quic-and-rust/
- xiphias2 4y agoThe most important part if the blog entry is that they are building on top of async Tokio but not Hyper, so other companies can probably consider Tokio as production ready for large scale workloads.
- perrohunter 4y agoDo you think there’s problems with Hyper?
- onei 4y agoI believe the lack of using Hyper was so they could be more flexible in what nonstandard parts of HTTP they could support, e.g. status codes into the 900s.
- IceWreck 4y agoHyper conforms to HTTP but a bit too much. Cloudflare has to proxy users who may send invalid HTTP like status code 666 or something.
- db48x 4y agoAnd conforming to HTTP is a good thing when you are writing a server, where you want all your responses to be perfectly formed. But for a proxy that is mostly passing through other people's traffic unaltered, it is not so important.
- iforgotpassword 4y agoYou could almost argue it's counter productive to an extend. One of these invalid response codes might become valid in the future and then suddenly your proxy breaks valid communication attempts. One of the reasons TLS1.3 has to look like TLS1.2 in the handshake.
- tialaramex 4y agoI don't know about HTTP, but TLS had explicit invariants. You could write a working TLS 1.2 proxy in say, 2010, and do it correctly, having of course no idea how TLS 1.3 will work, and it would have worked with TLS 1.3 as conceived over a year before publication, back when it admitted it was TLS 1.3 (0x03 0x04 ~= SSL 3.4) But real world systems, most prominently "middleboxes" often sold as security devices, did not obey the invariants, they were actually bad implementations of TLS 1.2, but they worked and so they'd been able to proliferate. TLS 1.3 as eventually standardised needed to cope with that nonsense. So, if there are HTTP invariants, then a proxy merely needs to get those correct and would in principle interoperate despite newer HTTP versions. With TLS 1.2 invariants talking to a TLS 1.3 client that meant a proper proxy would shrug and say "I can't speak TLS 1.3, you need to talk TLS 1.2" and that works fine. Whereas the middleboxes tended to go "OMG. A byte I didn't understand! We're under attack! Light the beacons, all men to the battlements!"
- xani_ 4y agoWeird that they didn't use HAProxy in the first place if they are (apparently) not even using it to serve files from filesystem
- jgrahamc 4y agoWe had a great deal of experience with NGINX and the original architecture for Cloudflare was based on NGINX. When we came to move away from it we wanted to fully own our destiny (as we had done years before when writing our own DNS server) and so moving to HAProxy would not have made sense.
- aukaost 4y agoThey were wondering why the original architecture was using NGINX over HAProxy, not why HAProxy wasn't chosen as an NGINX replacement.
- jgrahamc 4y agoI think because Lee Holloway (the technical founder of Cloudflare) was used to use NGINX and so he used it for the original architecture. It's also the case that Pingora replaced part of something that handled both connections to origin servers and reading from cache. So the comment about "not serving files" isn't 100% correct as the NGINX instance was serving files as part of its work.
- xani_ 4y agoWell, it does offer more varied stuff from the get go so I can see why someone wouldn't want to limit their options, HAProxy was and is purely a proxy while NGINX is a bit of everything. SPOE/SPOA added a bit of programmability to HAProxy but it is still basically only messing around with headers and acting upon that, nothing to do with content.
- neomantra 4y agoBack in the day, Cloudflare's WAF was based on OpenResty, so the high-performance Lua-programmability at the edge (which is noted in this blog entry) was probably a factor. Quick research shows HAProxy added Lua support in 2015, which is a bit later than their use of OpenResty.
- ducktective 4y agoWouldn't "switches to" have been a better word instead of "ditches" which implies they are getting rid of it? Considering Nginx is an old and reliable FOSS software... What precedent would it set for developers when 10 years down the line, a new shiny tool would just "ditch" the result of their hard work?
- dig1 4y ago
- runevault 4y agoYou can argue writing a replacement isn't hard, but writing one that had (at time of writing) served north of 100 trillion requests without ANY errors in the proxy server itself is impressive to me even with Rust's promises.
- sidewndr46 4y agoDoesn't a server author get to decide what an error is? You could always just decide the fault was in the client, not the server. For example, I worked with one CDN company where the engineers decided that injecting random NULL bytes into the TCP stream of an HTTP request was perfectly valid. They obviously are in fact idiots. This is similar to how AWS can always claim 5-9s of uptime. They decide what counts as downtime. Even when large portions of us-east-1 are broken, they have 5-9s of uptime.
- manfre 4y agoThe way too common green i..."some customers are experiencing elevated error rates..."
- sidewndr46 4y agoI imagine a dystopian future where the error reads "some customers were experiencing elevated error rates. Their accounts have been terminated in line with Amazon's zero tolerance policy for errors"
- sophacles 4y agoIn the blog post, the claim is "without crashes due to server code". That is different than protocol errors or logic errors (both of which are recoverable in a well written server). There's a lot less weasel room in such a claim.
- 4y ago
- numlock86 4y agoMain takeaway: Unless you are operating at Cloudflare scale nginx is probably fine.
- flumpcakes 4y agoAnd considering there was no real need to move away apart from the difficulty getting new features I would say even if you are operating at Cloudflare scale Nginx is still fine! (Although they did save hardware resources by moving to their own proxy and increasing cache hit rate - but seemingly none of these were business killing issues.)
- marcosdumay 4y agoDid anybody have any doubt that nginx is fine? What is newsworthy is that there's a new server that's validated as fine.
- deleted 4y ago[deleted]