5 ms·

If we hit a lot of random sub folder URLs (that should all 404), would that trigger the webserver to check the drive to see if the file exists and skip the cach

If we hit a lot of random sub folder URLs (that should all 404), would that trigger the webserver to check the drive to see if the file exists and skip the cache?

jewel 4y ago
No. The cache is at the block layer, not the file/directory layer. So the filesystem will look up the directory structure, which will be cached.
- asveikau 4y ago
  On top of the block layer cache there's also the namei cache for filename to inode lookups, I'm not sure if that covers a file not found case or just a success path, but it may apply here too.
  - Vogtinator 4y ago
    Yep, that's called a "negative dentry".
    - asveikau 4y ago
      But if a negative dentry does not yet exist, and a filename is requested for the first time, does it read from the fs? The DDoS scenario would be doing GET requests of random nonexistent filenames. You could change the name at every step so that the check for a negative dentry is never a cache hit.
      - vocram 4y ago
        If requesting http paths result in accessing the equivalent fs paths, then yes, you can dos it. But nowadays it’s uncommon for a webserver.
        asveikau 4y ago
        Well the point under discussion is that i/o from a floppy disk is slow. Most servers have much faster disks and it'd be absurd to consider this. Also the page/buffer cache would probably cache the FAT direntries so probably not a huge issue here.
  - bch 4y ago
    Still backed by block layer cache for (hopefully) quick response regardless of outcome
    - danachow 4y ago
      What block layer cache?
      - bch 4y ago
        I think I was wrong, typed too quickly. There’s (potentially) a file buffer block cache available, but that’s not for caching raw disk blocks, but filesystem blocks of content.
- koverstreet 4y ago
  Actually, no. There is no general block layer cache - the closest thing is the page cache, which is at the file level.
  - rascul 4y ago
    Are you the Kent Overstreet working on bcachefs? If so, I've been watching it and looking forward for it to get into the kernel. Keep up the good work. I'll have a donation coming in a little while.
    - koverstreet 4y ago
      The one and only :)
  - yencabulator 4y ago
    I guess the real question there is, does a directory count as a file for the purposes of this discussion. It seems necessary that the blocks storing the directory entries are cached too. Otherwise every non-existent lookup (that doesn't hit a negative dentry) would hit disk, each separately. So yeah, the page cache is keyed by file, but the system should still cache the directory structure.

If we hit a lot of random sub folder URLs (that should all 404), would that trigger the webserver to check the drive to see if the file exists and skip the cache?

jewel 4y ago

No. The cache is at the block layer, not the file/directory layer. So the filesystem will look up the directory structure, which will be cached.

asveikau 4y ago
On top of the block layer cache there's also the namei cache for filename to inode lookups, I'm not sure if that covers a file not found case or just a success path, but it may apply here too.
- Vogtinator 4y ago
  Yep, that's called a "negative dentry".
  - asveikau 4y ago
    But if a negative dentry does not yet exist, and a filename is requested for the first time, does it read from the fs? The DDoS scenario would be doing GET requests of random nonexistent filenames. You could change the name at every step so that the check for a negative dentry is never a cache hit.
    - vocram 4y ago
      If requesting http paths result in accessing the equivalent fs paths, then yes, you can dos it. But nowadays it’s uncommon for a webserver.
      - asveikau 4y ago
        Well the point under discussion is that i/o from a floppy disk is slow. Most servers have much faster disks and it'd be absurd to consider this. Also the page/buffer cache would probably cache the FAT direntries so probably not a huge issue here.
- bch 4y ago
  Still backed by block layer cache for (hopefully) quick response regardless of outcome
  - danachow 4y ago
    What block layer cache?
    - bch 4y ago
      I think I was wrong, typed too quickly. There’s (potentially) a file buffer block cache available, but that’s not for caching raw disk blocks, but filesystem blocks of content.
koverstreet 4y ago
Actually, no. There is no general block layer cache - the closest thing is the page cache, which is at the file level.
- rascul 4y ago
  Are you the Kent Overstreet working on bcachefs? If so, I've been watching it and looking forward for it to get into the kernel. Keep up the good work. I'll have a donation coming in a little while.
  - koverstreet 4y ago
    The one and only :)
- yencabulator 4y ago
  I guess the real question there is, does a directory count as a file for the purposes of this discussion. It seems necessary that the blocks storing the directory entries are cached too. Otherwise every non-existent lookup (that doesn't hit a negative dentry) would hit disk, each separately. So yeah, the page cache is keyed by file, but the system should still cache the directory structure.