3 ms·
Would reading without the cache (e.g. O_DIRECT) can be used to achieve a similar effect?
by ithkuil 4y ago
Would reading without the cache (e.g. O_DIRECT) can be used to achieve a similar effect?
- jart 4y agoWell I think that would guarantee your always hit disk for sure. The goal here is to be able to run the llama.cpp command repeatedly, possibly in shell scripts, and rely on the file caches being available so that the command loads instantly (we're talking like 1 second rather than 60 seconds to run, because disk is 100x slower).
- dekhn 4y agoO_DIRECT is absurdly slow in my my experience.
- the8472 4y agoO_DIRECT is basically telling the OS to load a block of data straight from disk into your process memory, skipping all the caching, sharing, scheduling, coalescing and readaheads the OS normally does for you. Which means you now either must do all that yourself in userspace or you can use io_uring to have it happen in the background and get notified when it's done. But if the intent is to have multiple processes share the samedata then mmap likely is the better choice.