evan_miller

3 ms·

1.
by evan_miller 11y ago
You're missing something very significant: Compilation jobs involve a lot of forking. Forking, memory accesses, and TLB differences across various VM architectures can have an enormous impact on performance. For example, we found two o
2.
by evan_miller 12y ago
The semantics for write(2) and mmap(2) differ. Neither can be used to implement the other in full. It's possible to have access patterns which don't demonstrate the efficiencies of mmap but it would be incorrect to suggest they do
3.
by evan_miller 12y ago
Additionally they may also legally earn money. How else would children appear in movies? The particulars vary by state, but here's a quick glimpse as to the federal perspective: http://www.dol.gov/elaws/faq/es
4.
by evan_miller 12y ago
It's actually -ne, perhaps with -a if one wants awk-style field splitting of lines. -l modifies the output record separator (makes print append \n)
5.
by evan_miller 12y ago
Have you tried disabling only defrag? The performance issues I'm familiar with are generally related to defrag running at allocation time. I would be curious to see your results while enabling THP but: echo never > /sys/ke
6.
by evan_miller 12y ago
Ah, thanks for your feedback. I'm never sure how much detail I should dive into. I will add more next time. I'll answer them here just for practice :) Recall, this is a problem in the 99th percentile. That is, on average our reque
7.
by evan_miller 12y ago
The fsync() is implicit behavior of the glog framework for Go. We didn't actually want it.
8.
by evan_miller 12y ago
Yes. I haven't done anything with a log-structured FS. I have only played around with conventional filesystems.
9.
by evan_miller 12y ago
Sort of. The catch is that even a very small write, say just a few megabytes, can drastically change the cost of an fsync(). On my test aws VM even writing just 4 megabytes one time is enough to trigger the problem. Even on an otherwise ful
10.
by evan_miller 12y ago
Not recently, no. I do know that a journaled filesystem can exacerbate this sort of problem as it can make extra work. For example: http://lwn.net/Articles/328363/ In a few cases in the past when dealing with unim
11.
by evan_miller 12y ago
With fsync() taken out of the equation this is essentially how a naive logger operates. The linux page cache plays the role of shared memory and only under rather heavy contention will a write() incur latency. The trouble outlined in my blo
12.
by evan_miller 12y ago
Hi, post author here. The slight difference in i/o (specifically: writes) was the trigger. I talked a little more about that here: http://corner.squareup.com/2014/09/logging-can-be-tricky.htm... And here: ht
13.
by evan_miller 12y ago
Hi, post author here. I didn't make it as clear as I could have, but the difference is that the problematic system had an unrelated process creating slightly more writes. I sort-of glossed over this with the 5%-20% difference in i&#x2F