9 ms·
How We Designed for Performance and Scale
- rdtsc 11y agoBtw there is a new feature in the kernel to help avoid using the accept shared memory mutex -- EPOLLEXCLUSIVE and EPOLLROUNDROBIN This should round robin accept in the kernel, and not wake up all the epoll listeners. https://lwn.net/Articles/632590/ https://lwn.net/Articles/632590/
- 15155 11y agoIsn't this exactly what EPOLLET (edge triggering) does?
- _wmd 11y agoEdge vs level triggered refers to how a change in an fd's state is reflected in userspace: level triggering will cause repeated wakeups while the condition remains true, whereas edge triggering will cause exactly one, until the monitored state flips from true to false and back again. The new flags relate to what happens when multiple threads are waiting on the same set of file descriptors (i.e. sleeping on the same epoll FD passed as the first parameter to epoll_wait(), or having the same client FD in multiple epoll sets -- sorry I'm not sure which way around it is). Previously the kernel had no support for waking exactly one thread to handle one event, so if there was a single set shared among a bunch of sleeping tasks, all tasks would be scheduled, causing (presumably) synchronization contention on the kernel's internal structures. The new mode ensures only a single task is woken up for an event on a single FD, even when multiple tasks are waiting on it.
- nodesocket 11y ago"NGINX’s binary upgrade process achieves the holy grail of high-availability; you can upgrade the software on the fly, without any dropped connections, downtime or interruption in service." Is this really true? I remember seeing an article[1] recently on using an iptables hack to prevent dropping connections when reloading haproxy. Does nginx actually provide zero-downtime configuration reloads? [1] https://medium.com/@Drew_Stokes/actual-zero-downtime-with-haproxy-18318578fde6 https://medium.com/@Drew_Stokes/actual-zero-downtime-with-ha...
- MCRed 11y agoIf it's the holy grail, then erlang has had it for a couple decades... and whether it actually works in nginx I don't know, but it does work in erlang. The way it's handled is that the process where the previous socket was connected remains in place until it terminates but all new sockets connect to the new code.
- nodesocket 11y ago"The way it's handled is that the process where the previous socket was connected remains in place until it terminates but all new sockets connect to the new code." That would imply that two separate processes are bound to the same port right? I thought that was not possible.
- SEJeff 11y agoThings have changed :) https://lwn.net/Articles/542629/ https://lwn.net/Articles/542629/
- ericmoritz 11y agoNo, usually there's an accept process spawning working processes and passing the connection off to it. Erlang can have two versions of a module present in memory at a time. At the moment of a code upgrade, the processes working with existing connections continue to use the old code while newly accepted connections are passed to the version of the module that was loaded in between connection accepts.
- simi_ 11y agoI had a feeling I've read about nginx before: http://aosabook.org/en/nginx.html http://aosabook.org/en/nginx.html The whole book is worth a read, although I found some sections painfully boring (perhaps my limited attention span is to blame).
- the_why_of_y 11y agoThere's a 3rd volume "The Performance of Open Source Applications" now, and it has a chapter on another high performance HTTP server, Warp: http://www.aosabook.org/en/posa/warp.html http://www.aosabook.org/en/posa/warp.html Interesting what kind of performance one can get out of GHC nowadays. Article says the authors of Warp had to implement a new parallel IO manager for GHC to get there, but that was merged into GHC 7.8.
- cshimmin 11y ago> You can reload configuration multiple times per second (and many NGINX users do exactly that) I thought this was an interesting remark. Can anyone clue me in to what these "many users" might be doing, that requires them to reload configuration so frequently?
- Igglyboo 11y agoDynamically generating urls, like tumblr or github pages? (just a guess)
- fweespeech 11y agoI know of at least one VPS company that build its load balancer SaaS offering on NGINX and it automatically reloads whenever a node is added/removed. So I would assume it was that function that would cause such rapid reloads. [e.g. If you had a 50 node pool, you might go +/-5 over a span of a second and change the config 5 times in 1s]
- earless1 11y agoI front my docker fleet with Nginx and I need to reload config for new hosts
- threeseed 11y agoService Discovery. Dynamically generating routing rules so microservices can have pretty URLs. For example with us when you deploy a new version of a microservice it starts on a random port, registers with Consul on startup and then dynamically regenerates and reloads Nginx.
- saurabhtandon 11y agoInteresting overview. I wish they had some data comparison which could explain the significance and efficiency of this approach vs other/old approaches.
- nailer 11y agoThis is a lovely article, but: > The fundamental basis of any Unix application is the thread or process. (From the Linux OS perspective, threads and processes are mostly identical; the major difference is the degree to which they share memory.) It's better to be specific in performance discussions, rather than use 'thread' and 'process' interchangeably. As well as the article mentioned about memory sharing, threads (which are called Lightweight Processes, or LWPs, in Linux 'ps') are granular. ps -eLf NWLP in the command above is 'number of lightweight processes', ie number of threads. Processes are not granular: they're one or many threads. IIRC it can be beneficial to assign threads of the same process to the same physical core or same die for cache affinity. There's all kind of performance stuff where 'threads' and 'processes' do not mean the same thing. Being specific is rad.
- alexchamberlain 11y agoIn Linux, they are both just entries in the process table. They are created by the `clone` syscall, and the "normal" way of creating them share different amounts of resources by _default_. You're right to say that treating them differently can be beneficial in some situations, but it really depends.
- nailer 11y agoYep. I guess the most basic case is: "is this single process multithreaded, as I have a multicore machine and there's only one PID"
- afarrell 11y agoAside: you really nailed the "I think they're saying something significantly incorrect but don't want to be a jerk about it" tone. Kudos.
- amelius 11y agoOne thread per CPU, and non-blocking I/O, that's sounds like the usual way to approach the problem. I'm surprised it uses state machines to handle the non-blocking I/O, because modern software engineering provides much more pleasant approaches such as using coroutines.
- nly 11y agoCoroutines have the distinct disadvantage of needing a stack, much like threads. So-called 'stackless' coroutines aren't really so different to computed gotos in a state machine
- amelius 11y agoFor parsing html, you wouldn't need that large a stack. Further, you could make the stack grow dynamically at a very small cost. Also, I bet that even if stackless coroutines are close to FSMs, they are much easier to program.
- exacube 11y agoYou need green/user-space threads if you want to use co-routines effectively, because if a coroutine blocks, it would block off the entire OS thread from doing anything else.
- exacube 11y agoWhat are stackless coroutines? What do they look like?
- acidx 11y agoRead Adam Dunkels' paper on Protothreads.
- exacube 11y agoThanks for your comment. I don't know how to feel about Protothreads. It's just syntatic sugar for using a state-machine to provide continuations. You're also not allowed to carry state over between continuations (although I can see how you can extend it to carry-over some struct of data or something). This greatly diminishes the usefulness of Protothreads.. so it feels more like a fun proof-of-concept. Are there any popular/real-world use cases of Protothreads?
- McElroy 11y agoI did as they said at the bottom and gave them my e-mail and other personal details so I could download the eBook that they were giving free preview copies of - "Building Microservices". Unfortunately, they sent link to PDF only so it's not usable to me. Just a heads up to others so you save yourself the time of discovering that. (I'll just wait for when the book is finished and then I'll buy it so I get ePub. I like O'Reilly and have bought many books there before.)
- justincormack 11y agoThe book is out now from O-Reilly - its a good book.
- andorov 11y agoIf you have a kindle you can send a pdf to your kindle email with the subject 'convert' - http://www.amazon.com/gp/sendtokindle/email http://www.amazon.com/gp/sendtokindle/email
- dschiptsov 11y agoThey forgot to mention pool-allocated buffers, zero-copy strings, and very clean, layered codebase - every syscall was counted. The original nginx is a rare example of what is the best in software engineering - deep understanding of principles and almost Asperger's attention to details (which is obviously good). Its success is justified.