5 ms·

The post should be marked as [2006]. A lot has changed since then. For example it doesn't talk about NUMA but it should, if it were written today. I would love

The post should be marked as [2006]. A lot has changed since then. For example it doesn't talk about NUMA but it should, if it were written today. I would love to hear some opinions from the authors of more contemporary web servers like one of H2O or Envoy. (The latter may have slightly different use cases though.)

markdog12 5y ago
Forgive my ignorance, but doesn't http://varnish-cache.org/docs/trunk/phk/notes.html#more-caches http://varnish-cache.org/docs/trunk/phk/notes.html#more-cach... talk about that, with an example?
- flakiness 5y ago
  It's about cache vs memory. NUMA is about local memory vs remote memory. So similar but different IMO.
tyingq 5y ago
What's different about NUMA today vs 2006?
- wayoutthere 5y ago
  NUMA was not widely available / understood in 2006. I don’t recall seeing my first NUMA servers until probably 2008ish.
  - tyingq 5y ago
    That makes sense to me for the entire computing landscape, like if we're including windows, various 32-bit x86 unix like OS's, etc. But varnish was used on Solaris, IRIX, and HP/UX. All of which had cc-NUMA support fairly well established by 2006. I suppose, though, it was still new-ish.
  - anonymfus 5y ago
    First Opterons were released in 2003
- phkamp 5y ago
  I would say nothing, except maybe for how many mentions it gets in HW-review articles. Unless you are in numerical HPC or kernel programming, you do not need to think about NUMA, but it is not black/white, it is a slope. Varnish tries to do certain things "NUMA-friendly", without actually being "NUMA-aware" as such, and the kernels seem to do a really good job with that.
wpietri 5y ago
Even for 2006, what would make this argument for me is actual numbers. Is it plausible that some programs worked well like that then? Sure. But I can promise not everything did. The kernel's approach to caching is optimized for the general case, but each program is specific. It's plausible to me that a caching service like Varnish is a very close match. But that's an empirical question, not a theoretical one.
jart 5y ago
Well if by contemporary you mean written by Google builds with Bazel and runs in Docker, stuff like Envoy is pretty good and it uses Joyent's HTTP parser which is more readable than the pigeon C parsing Varnish likes to do. But Varnish understands UNIX much better than Envoy does, as evidenced by its portability and the things you've read in the blog post. Compared to Squid then hands down Varnish is the best. I'm not sure if it's better than open source GFE but it's pretty good. None of them go as fast as redbean of course. The blog post talks about how they only need 18 system calls to serve a local asset. redbean only needs one system call: writev(). That's because redbean cheats by having a novel design. The zip central directory means we don't need file system system calls. The invariant RDTSC instruction means we can also have timestamps without context switching.
phkamp 5y ago
Yes, 2006 would be a more proper tag. NUMA is interesting, I've had a lot of fun with it on a system to control the adaptive mirrors in ESO's ELT telescope, and it is /amazing/ what you can do with a modern server, even on a vanilla kernel. However, it is a lot harder to exploit NUMA in an application like Varnish, because the requests arrive as a randomized stream, and the effort to figure out which NUMA pool is best situated to handle the request, is not significantly different from just handling the request to begin with. We used to serve a cache hit in seven system calls, an accept(2), a read(2), a write(2) and four timestamps, and as a result, Varnish servers ran with CPU's >70% idle. If we had added NUMA-awareness on top of that, we would on average Nnuma-1/Nnuma of the time add at two system-calls to migrate the request to another NUMA domain, and that made little sense. Since then NICs have become NUMA aware and since we use scatter/gather I/O, that matters a lot more than which CPU core did the overhead processing. Where NUMA may become mandatory is 100G and 400G networking, but very few people seem interested in running that much traffic through a single server, for reasons of reliability, but I'm keeping an eye on it.
- drewg123 5y ago
  Checkout TCP_REUSPORT_LB_NUMA on FreeBSD. It will filter incoming TCP connections to listen sockets owned by threads bound to the same NUMA domain as the NIC that received the packets. This is part of the Netflix FreeBSD NUMA work described here https://papers.freebsd.org/2019/eurobsdcon/gallatin-numa_optimizations_network_stack/ https://papers.freebsd.org/2019/eurobsdcon/gallatin-numa_opt... The idea is that you can keep connections local to a NUMA domain. We (Netflix) have a local patch to nginx to support this (basically just setting the socket ioctl on the listen socket after the nginx worker is bound)
  - phkamp 5y ago
    Yeah, so there are a lot of things we could optimize for on different kernels, but we prefer to not do so, until we have a really good reason. As I said above, it is not obvious to me that we would gain much over what we already do, but I am keeping an eye on it.
- eqvinox 5y ago
pmlnr 5y ago
Envoy is not a web server.
- Matthias247 5y ago
  Why not? And what’s the definition of a „web server“ anyway? In my opinion it’s about whether a software can process http requests on a configurable port - and envoy can for sure do that.
  - pmlnr 5y ago
    A webserver, as bare minimum, should be able to serve static files from the disk - but you are correct, there is no such definition, so this is an opinion. Regardless, calling envoy a webserver is misleading: it's both more and less, than a "traditional" webserver. It's a proxy; call it a proxy, please.
    - jrockway 5y ago
      > A webserver, as bare minimum, should be able to serve static files from the disk. You can configure routes with a `direct_response` handler from envoy.yaml that is read from disk ;)
  - galoisgirl 5y ago
    It's a function from request to response.

markdog12 5y ago

Forgive my ignorance, but doesn't http://varnish-cache.org/docs/trunk/phk/notes.html#more-caches http://varnish-cache.org/docs/trunk/phk/notes.html#more-cach... talk about that, with an example?

flakiness 5y ago
It's about cache vs memory. NUMA is about local memory vs remote memory. So similar but different IMO.

tyingq 5y ago

What's different about NUMA today vs 2006?

wayoutthere 5y ago
NUMA was not widely available / understood in 2006. I don’t recall seeing my first NUMA servers until probably 2008ish.
- tyingq 5y ago
  That makes sense to me for the entire computing landscape, like if we're including windows, various 32-bit x86 unix like OS's, etc. But varnish was used on Solaris, IRIX, and HP/UX. All of which had cc-NUMA support fairly well established by 2006. I suppose, though, it was still new-ish.
- anonymfus 5y ago
  First Opterons were released in 2003
phkamp 5y ago
I would say nothing, except maybe for how many mentions it gets in HW-review articles. Unless you are in numerical HPC or kernel programming, you do not need to think about NUMA, but it is not black/white, it is a slope. Varnish tries to do certain things "NUMA-friendly", without actually being "NUMA-aware" as such, and the kernels seem to do a really good job with that.

wpietri 5y ago

Even for 2006, what would make this argument for me is actual numbers. Is it plausible that some programs worked well like that then? Sure. But I can promise not everything did. The kernel's approach to caching is optimized for the general case, but each program is specific. It's plausible to me that a caching service like Varnish is a very close match. But that's an empirical question, not a theoretical one.

jart 5y ago

Well if by contemporary you mean written by Google builds with Bazel and runs in Docker, stuff like Envoy is pretty good and it uses Joyent's HTTP parser which is more readable than the pigeon C parsing Varnish likes to do. But Varnish understands UNIX much better than Envoy does, as evidenced by its portability and the things you've read in the blog post. Compared to Squid then hands down Varnish is the best. I'm not sure if it's better than open source GFE but it's pretty good. None of them go as fast as redbean of course. The blog post talks about how they only need 18 system calls to serve a local asset. redbean only needs one system call: writev(). That's because redbean cheats by having a novel design. The zip central directory means we don't need file system system calls. The invariant RDTSC instruction means we can also have timestamps without context switching.

phkamp 5y ago

Yes, 2006 would be a more proper tag. NUMA is interesting, I've had a lot of fun with it on a system to control the adaptive mirrors in ESO's ELT telescope, and it is /amazing/ what you can do with a modern server, even on a vanilla kernel. However, it is a lot harder to exploit NUMA in an application like Varnish, because the requests arrive as a randomized stream, and the effort to figure out which NUMA pool is best situated to handle the request, is not significantly different from just handling the request to begin with. We used to serve a cache hit in seven system calls, an accept(2), a read(2), a write(2) and four timestamps, and as a result, Varnish servers ran with CPU's >70% idle. If we had added NUMA-awareness on top of that, we would on average Nnuma-1/Nnuma of the time add at two system-calls to migrate the request to another NUMA domain, and that made little sense. Since then NICs have become NUMA aware and since we use scatter/gather I/O, that matters a lot more than which CPU core did the overhead processing. Where NUMA may become mandatory is 100G and 400G networking, but very few people seem interested in running that much traffic through a single server, for reasons of reliability, but I'm keeping an eye on it.

drewg123 5y ago
Checkout TCP_REUSPORT_LB_NUMA on FreeBSD. It will filter incoming TCP connections to listen sockets owned by threads bound to the same NUMA domain as the NIC that received the packets. This is part of the Netflix FreeBSD NUMA work described here https://papers.freebsd.org/2019/eurobsdcon/gallatin-numa_optimizations_network_stack/ https://papers.freebsd.org/2019/eurobsdcon/gallatin-numa_opt... The idea is that you can keep connections local to a NUMA domain. We (Netflix) have a local patch to nginx to support this (basically just setting the socket ioctl on the listen socket after the nginx worker is bound)
- phkamp 5y ago
  Yeah, so there are a lot of things we could optimize for on different kernels, but we prefer to not do so, until we have a really good reason. As I said above, it is not obvious to me that we would gain much over what we already do, but I am keeping an eye on it.
eqvinox 5y ago

pmlnr 5y ago

Envoy is not a web server.

Matthias247 5y ago
Why not? And what’s the definition of a „web server“ anyway? In my opinion it’s about whether a software can process http requests on a configurable port - and envoy can for sure do that.
- pmlnr 5y ago
  A webserver, as bare minimum, should be able to serve static files from the disk - but you are correct, there is no such definition, so this is an opinion. Regardless, calling envoy a webserver is misleading: it's both more and less, than a "traditional" webserver. It's a proxy; call it a proxy, please.
  - jrockway 5y ago
    > A webserver, as bare minimum, should be able to serve static files from the disk. You can configure routes with a `direct_response` handler from envoy.yaml that is read from disk ;)
- galoisgirl 5y ago
  It's a function from request to response.