8 ms·

"Varnish allocate[s] some virtual memory, it tells the operating system to back this memory with space from a disk file. When it needs to send the object to a c

"Varnish allocate[s] some virtual memory, it tells the operating system to back this memory with space from a disk file. When it needs to send the object to a client, it simply refers to that piece of virtual memory and leaves the rest to the kernel. If/when the kernel decides it needs to use RAM for something else, the page will get written to the backing file and the RAM page reused elsewhere. When Varnish next time refers to the virtual memory, the operating system will find a RAM page, possibly freeing one, and read the contents in from the backing file. And that's it. " https://www.varnish-cache.org/trac/wiki/ArchitectNotes https://www.varnish-cache.org/trac/wiki/ArchitectNotes I'll try and hold back the snark but I find it interesting that after attacking '1975 programming' and Squid's deficiencies, here we are 8 years later and maybe the kernel doesn't know best.

xxxyy 12y ago
I'm not sure whether it was using write() over mmap() that boosted the performance. This part however looks very interesting: > Taking advantage of the fact that Varnish is a cache and letting it actually eliminate objects that are blocking new allocations simplifies the allocation process.
- cbsmith 12y ago
  Which is exactly the thinking as to why squid cache used write()...
- perbu 12y ago
  mmap a file into memory. As you touch a page the CPU generates a page fault and invokes the kernel. The kernel merges the partially written page with the underlying data. As there is no way to avoid the merge you end up doing write() instead. The page fault is a synchronous read. Horrible for performance.
  - xxxyy 12y ago
    What do you mean by "The kernel merges the partially written page". A pagefault traps the execution just before any CPU write succeeds, then the kernel optionally loads the page, maps it into process' space, flushes TLBs and resumes process execution. mmap() style disk writes happen on page-level granularity, usually not before the kernel runs out of free memory frames. I agree on the fact that using mmap() reads vs read() reads leaves the kernel with no possibility to reorder requests, as any thread trapped in a pagefault is obviously unable to generate more I/O requests. Reordering can lead to better performance. This is not the case with asynchronous read().
    - perbu 12y ago
      I might have expressed myself somewhat unclear. Let me try again. varnishd tries to write to a page not in memory and gets a page fault while doing so. As the kernel has no idea of your intentions to overwrite the whole page it will read the page into memory and then return control to varnish who completes the memory write, merging the write request just made onto the preexisting page. varnish does not utilize any asynchronous reads. All IO is blocking.
      - xxxyy 12y ago
        I understand now. Not enough that the kernel unnecessarily loads a page that is about to be fully rewritten, it also blocks the thread that is trapped in a pagefault, while it could potentially go and serve another connection. I can see how write() can be faster than mmap(), thanks.
nkurz 12y ago
You have a strong point. I was more struck by the incongruity of the following statement: "We started out using a memory mapped file to store objects in. It had some problems associated with it and was replaced with a storage engine that relied on malloc to store content. While it usually performed better than the memory mapped files performance suffered as the content grew past the limitations imposed by physical memory." So it looks like they gave up on the "kernel knows best" approach quite a while ago. But then they show a graph, where the older mmap() approach is more than twice as fast as the malloc() approach across the entire range of the graph. They explain this in a sentence below the graph saying "Malloc suffers quite a bit here as the swap performance on Linux is rather abysmally bad. " Well, yes. But presumably that is the case we're interested in, as a cache that has room for everything is a much easier problem. So what are we to make of the earlier contention that malloc() trumps mmap()?
- wmf 12y ago
  Varnish was originally designed to run on FreeBSD (as PHK works on both projects). Have they changed to primarily target Linux, perhaps in response to market demand?
  - perbu 12y ago
    ~ 99% of all varnish instances run on Linux. FreeBSD is still important. Varnish Software doesn't currently have any customers who are planning on staying on FreeBSD. They are all migrating to Linux.
    - vorador 12y ago
      I'm a linux user but I've always been interested in running my projects on FreeBSD. Could you give a couple reasons why those people are migrating?
      - perbu 12y ago
        FreeBSD admins are hard to come by. Same with software; try getting something hip running on FreeBSD these days. Even better, try getting a support contract on it, it will not be trivial. However, the quality of the sysadmins that prefer FreeBSD is pretty high, though. And FreeBSD is pretty neat.
imanaccount247 12y ago
This is exactly what I came to point out. Varnish was pushed in a very arrogant manner insulting other software that was just plain better. While they were trash talking squid and pretending the kernel could just magically know how to do everything perfectly for any arbitrary userland workload, we were consistently getting better performance from squid.
- xxxyy 12y ago
  Squid was faster? Varnish is an engineering masterpiece, do you have any data to prove this? I would be very glad to see it.
  - imanaccount247 12y ago
    >Varnish is an engineering masterpiece No, it isn't. That's very much the point. They have spent all this time realizing that doing it wrong is bad and rewriting it to actually work. >do you have any data to prove this? No, that was 2 jobs ago. I'm sure you can get a hold of the source code of old varnish and test it out yourself.
    - xxxyy 12y ago
      I am not talking about mmap() vs write() here. That is just ordinary propaganda. There is a lot of powerful ideas at work there. The two that I know about: 1) Compiling config file into an .so object. 2) Using innovative priority queue (heap) implementation: http://queue.acm.org/detail.cfm?id=1814327 http://queue.acm.org/detail.cfm?id=1814327 Varnish userbase is the best proof for its quality though.
      - imanaccount247 12y ago
        >Compiling config file into an .so object. That is neither new, nor interesting "engineering". Lots of software that needs significant customization uses the host programming language for configuration. >Using innovative priority queue (heap) implementation How is "we used an existing data structure and deliberately misrepresent this as some amazing discovery" an amazing feat of engineering? This supports the notion that varnish's biggest achievement is an amazing feat of marketing.
perbu 12y ago
You missed the point of the note. The point was to have the kernel balance what is on disk and what is in memory. Varnish (w/MSE) still relies on the kernel for this. However, as you cannot atomically write a whole page through a mmap (thereby avoiding the page fault) we have to use write(). It still is read through mmap and the kernel still drop pages whenever it feels like it.
- otterley 12y ago
  But if you're reading via mmap, aren't you still relying on the kernel's naive LRU page eviction algorithm?
  - perbu 12y ago
    The kernel switched to LFU in 2.6 if I'm not mistaken. But, yes, we are. There are quite a few caches with separate eviction algorithms if you follow the rabbit hole all the way down.

xxxyy 12y ago

I'm not sure whether it was using write() over mmap() that boosted the performance. This part however looks very interesting: > Taking advantage of the fact that Varnish is a cache and letting it actually eliminate objects that are blocking new allocations simplifies the allocation process.

cbsmith 12y ago
Which is exactly the thinking as to why squid cache used write()...
perbu 12y ago
mmap a file into memory. As you touch a page the CPU generates a page fault and invokes the kernel. The kernel merges the partially written page with the underlying data. As there is no way to avoid the merge you end up doing write() instead. The page fault is a synchronous read. Horrible for performance.
- xxxyy 12y ago
  What do you mean by "The kernel merges the partially written page". A pagefault traps the execution just before any CPU write succeeds, then the kernel optionally loads the page, maps it into process' space, flushes TLBs and resumes process execution. mmap() style disk writes happen on page-level granularity, usually not before the kernel runs out of free memory frames. I agree on the fact that using mmap() reads vs read() reads leaves the kernel with no possibility to reorder requests, as any thread trapped in a pagefault is obviously unable to generate more I/O requests. Reordering can lead to better performance. This is not the case with asynchronous read().
  - perbu 12y ago
    I might have expressed myself somewhat unclear. Let me try again. varnishd tries to write to a page not in memory and gets a page fault while doing so. As the kernel has no idea of your intentions to overwrite the whole page it will read the page into memory and then return control to varnish who completes the memory write, merging the write request just made onto the preexisting page. varnish does not utilize any asynchronous reads. All IO is blocking.
    - xxxyy 12y ago
      I understand now. Not enough that the kernel unnecessarily loads a page that is about to be fully rewritten, it also blocks the thread that is trapped in a pagefault, while it could potentially go and serve another connection. I can see how write() can be faster than mmap(), thanks.

nkurz 12y ago

You have a strong point. I was more struck by the incongruity of the following statement: "We started out using a memory mapped file to store objects in. It had some problems associated with it and was replaced with a storage engine that relied on malloc to store content. While it usually performed better than the memory mapped files performance suffered as the content grew past the limitations imposed by physical memory." So it looks like they gave up on the "kernel knows best" approach quite a while ago. But then they show a graph, where the older mmap() approach is more than twice as fast as the malloc() approach across the entire range of the graph. They explain this in a sentence below the graph saying "Malloc suffers quite a bit here as the swap performance on Linux is rather abysmally bad. " Well, yes. But presumably that is the case we're interested in, as a cache that has room for everything is a much easier problem. So what are we to make of the earlier contention that malloc() trumps mmap()?

wmf 12y ago
Varnish was originally designed to run on FreeBSD (as PHK works on both projects). Have they changed to primarily target Linux, perhaps in response to market demand?
- perbu 12y ago
  ~ 99% of all varnish instances run on Linux. FreeBSD is still important. Varnish Software doesn't currently have any customers who are planning on staying on FreeBSD. They are all migrating to Linux.
  - vorador 12y ago
    I'm a linux user but I've always been interested in running my projects on FreeBSD. Could you give a couple reasons why those people are migrating?
    - perbu 12y ago
      FreeBSD admins are hard to come by. Same with software; try getting something hip running on FreeBSD these days. Even better, try getting a support contract on it, it will not be trivial. However, the quality of the sysadmins that prefer FreeBSD is pretty high, though. And FreeBSD is pretty neat.

imanaccount247 12y ago

This is exactly what I came to point out. Varnish was pushed in a very arrogant manner insulting other software that was just plain better. While they were trash talking squid and pretending the kernel could just magically know how to do everything perfectly for any arbitrary userland workload, we were consistently getting better performance from squid.

xxxyy 12y ago
Squid was faster? Varnish is an engineering masterpiece, do you have any data to prove this? I would be very glad to see it.
- imanaccount247 12y ago
  >Varnish is an engineering masterpiece No, it isn't. That's very much the point. They have spent all this time realizing that doing it wrong is bad and rewriting it to actually work. >do you have any data to prove this? No, that was 2 jobs ago. I'm sure you can get a hold of the source code of old varnish and test it out yourself.
  - xxxyy 12y ago
    I am not talking about mmap() vs write() here. That is just ordinary propaganda. There is a lot of powerful ideas at work there. The two that I know about: 1) Compiling config file into an .so object. 2) Using innovative priority queue (heap) implementation: http://queue.acm.org/detail.cfm?id=1814327 http://queue.acm.org/detail.cfm?id=1814327 Varnish userbase is the best proof for its quality though.
    - imanaccount247 12y ago
      >Compiling config file into an .so object. That is neither new, nor interesting "engineering". Lots of software that needs significant customization uses the host programming language for configuration. >Using innovative priority queue (heap) implementation How is "we used an existing data structure and deliberately misrepresent this as some amazing discovery" an amazing feat of engineering? This supports the notion that varnish's biggest achievement is an amazing feat of marketing.

perbu 12y ago

You missed the point of the note. The point was to have the kernel balance what is on disk and what is in memory. Varnish (w/MSE) still relies on the kernel for this. However, as you cannot atomically write a whole page through a mmap (thereby avoiding the page fault) we have to use write(). It still is read through mmap and the kernel still drop pages whenever it feels like it.

otterley 12y ago
But if you're reading via mmap, aren't you still relying on the kernel's naive LRU page eviction algorithm?
- perbu 12y ago
  The kernel switched to LFU in 2.6 if I'm not mistaken. But, yes, we are. There are quite a few caches with separate eviction algorithms if you follow the rabbit hole all the way down.