3 ms·

This seems more interesting as demonstration of the amortized performance increase you'd expect from using io_uring, or as a tutorial for using it. I don't unde

This seems more interesting as demonstration of the amortized performance increase you'd expect from using io_uring, or as a tutorial for using it. I don't understand why I'd switch from using something like eza. If I'm listing 10,000 files the difference is between 40ms and 20ms. I absolutely would not notice that for a single invocation of the command.

0x000xca0xfe 1y ago
Well I have a directory with a couple million JSON files and ls/du take minutes. Most of the coreutils are not fast enough to actually utilize modern SSDs.
- otterley 1y ago
  What’s the filesystem type? Ext4 suffers terrible lookup performance with large directories, while xfs absolutely flies.
  - 0x000xca0xfe 1y ago
    Yup, default ext4 and most files are <4KB, so it's extra bad. Thanks for the comment, didn't know that!
rockorager 1y ago
Yeah, I wrote this as a fun little experiment to learn more io_uring usage. The practical savings of using this are tiny, maybe 5 seconds over your entire life. That wasn't the point haha
- JuettnerDistrib 1y ago
  I'd be curious to know if this helps on supercomputers, which are notorious for frequently hanging for a few seconds on an ls -l.
  - mrlongroots 1y ago
    It could, but important to keep in mind that the filesystem architecture there is also very different with a parallel filesystem with disaggregated data and metadata. When you run `ls -l` you could potentially be enumerating a directory with one file per rank, or worse, one file per particle or something. You could try making the read fast, but I also think that it makes no sense to have that many files: you can do things to reduce the number of files on disk. Also many are trying to push for distributed object stores instead of parallel filesystems... fun space.
- maplant 1y ago
  It's a very cool experiment. Just wanted to perhaps steer the conversation towards those things rather than whether or not this was a good ls replacement because like you say that feels like it was missing the point

0x000xca0xfe 1y ago

Well I have a directory with a couple million JSON files and ls/du take minutes. Most of the coreutils are not fast enough to actually utilize modern SSDs.

otterley 1y ago
What’s the filesystem type? Ext4 suffers terrible lookup performance with large directories, while xfs absolutely flies.
- 0x000xca0xfe 1y ago
  Yup, default ext4 and most files are <4KB, so it's extra bad. Thanks for the comment, didn't know that!

rockorager 1y ago

Yeah, I wrote this as a fun little experiment to learn more io_uring usage. The practical savings of using this are tiny, maybe 5 seconds over your entire life. That wasn't the point haha

JuettnerDistrib 1y ago
I'd be curious to know if this helps on supercomputers, which are notorious for frequently hanging for a few seconds on an ls -l.
- mrlongroots 1y ago
  It could, but important to keep in mind that the filesystem architecture there is also very different with a parallel filesystem with disaggregated data and metadata. When you run `ls -l` you could potentially be enumerating a directory with one file per rank, or worse, one file per particle or something. You could try making the read fast, but I also think that it makes no sense to have that many files: you can do things to reduce the number of files on disk. Also many are trying to push for distributed object stores instead of parallel filesystems... fun space.
maplant 1y ago
It's a very cool experiment. Just wanted to perhaps steer the conversation towards those things rather than whether or not this was a good ls replacement because like you say that feels like it was missing the point