3 ms·
right I suspect you have way fewer files than we do and everything is in the dentry cache. Pretty sure that most of your files are bigger than 60KB too :-) (whi
by khc 8y ago
right I suspect you have way fewer files than we do and everything is in the dentry cache. Pretty sure that most of your files are bigger than 60KB too :-) (which is our p90)
- drewg123 8y agoWow, it is a different world :)
- khc 8y agoour machines also run many different services (the CDN product is just one of them) and isolating I/O from different products is difficult. I'd also love to have NVMe.
- solarengineer 8y agoAny chance you could use dtrace to isolate process specific I/O?
- khc 8y ago"isolate" here I meant prevent one process's IO from affecting another one
- the8472 8y agoAt my job we have to open many small files from NFS. The latency of open() absolutely murders sequential performance (>80 seconds just to open a scene descriptions). Prewawrming the fairly shortlived NFS access caches in parallel evaporates most of the performance penalty.
- pram 8y agoTIL what a 'dentry' is!
- pg314 8y agoHave you looked into using something like SQLite instead of the filesystem? [1] [1] https://www.sqlite.org/fasterthanfs.html https://www.sqlite.org/fasterthanfs.html
- Kalium 8y agoSQLite makes a ton of sense for systems that don't need to worry about concurrent writes. It's possible that a CDN's cache system might need to concern itself with concurrent writes.
- khc 8y agoOther than the concurrent write that another comment mentioned, looks like this test is done with a data set that fits entirely in RAM. I wish we have enough RAM for the entire internet :-(