4 ms·

The problem with locate is that it requires the index db to be updated periodically (I think this happens daily by default?). For some use cases, especially tho

The problem with locate is that it requires the index db to be updated periodically (I think this happens daily by default?). For some use cases, especially those where I'm searching for files in a tree that I'm actively working in, this forces me to fall back to find (or maybe git ls-files now). I feel like it should be the job of the filesystem to maintain an index and incrementally update it whenever files are created or removed, but afaict no modern filesystem offers it.

spicybright 5y ago
It really is strange we don't have that yet. We already pretend files are on the physical disk and sync it in the background. Adding smarter indexing to that shouldn't be a huge deal.
ziml77 5y ago
As mentioned elsewhere in these comments, NTFS actually does do that. There are a couple tools out there that take advantage of it like Everything Search and WizTree. I don't know why no open source filesystem has done the same thing.
Karellen 5y ago
Sarcasm?
bee_rider 5y ago
I suppose this is sort of a problem depending on how you use it. I find that I don't really need to locate things that I've recently used, because I already know where they are as a result of having recently used them. But, other people have different workflows of course (I can imagine, for example, if somebody's workflow involved creating lots of files, only some of which are immediately useful, they might want the ability to scan through them for particular outputs).
varenc 5y ago
Not a direct part of the filesystem, but macOS does a good job of keeping a constantly up to date filesystem index. It powers Spotlight but is also used for other less user facing things. Running `mdfind` will locate whatever file I want near instantly and the index usually gets updated in a couple seconds. The background indexing does get bogged down when there’s a massive number of file changes, so I exclude things like “node_modules” folders. It also indexes the actual content of text files and not just filenames. Sadly the CLI interface for `mdfind` is incredibly obtuse and verbose, and very under documented [0]. When querying anything besides just a filename I still have to lookup the correct incantation. For example, here's how to do a case insensitive search for any folder containing "apple" modified in the last year: mdfind 'kMDItemDisplayName = "*apple*"c && kMDItemFSContentChangeDate <= $time.today(-365) && kMDItemKind == "Folder"' The mysterious `c` after `"apple"c` is how you specify a case insensitive search...of course. Compare that to fd: fd --changed-within=1y -t d -i apple [0] Nothing in the macOS man pages actually explains the `mdfind` query language, nor is there even a link or suggestion of where to learn more. This developer documentation sort of covers it, but it's not quite the same: https://developer.apple.com/documentation/corespotlight/cssearchquery https://developer.apple.com/documentation/corespotlight/csse...
- nerdponx 5y ago
  Is there an API for mdfind? If so, maybe it'd be worthwhile to try writing an alternative CLI frontend for it, if anyone is looking for a project.
  - varenc 5y ago
    There is! Found a simple example: https://gist.github.com/dagronf/3d03094a7ee79c1f91607f4c365fba91 https://gist.github.com/dagronf/3d03094a7ee79c1f91607f4c365f... I've thought about just making a simpler shell script that turns a valid `fd` query into the obscure `mdfind` incantation. And you can use `-onlyin ...` to limit the search to a particular dir just like fd/find. Might get around to it!
    - nerdponx 5y ago
      That would be interesting, and I'd probably use it!

spicybright 5y ago

It really is strange we don't have that yet. We already pretend files are on the physical disk and sync it in the background. Adding smarter indexing to that shouldn't be a huge deal.

ziml77 5y ago

As mentioned elsewhere in these comments, NTFS actually does do that. There are a couple tools out there that take advantage of it like Everything Search and WizTree. I don't know why no open source filesystem has done the same thing.

Karellen 5y ago

Sarcasm?

bee_rider 5y ago

I suppose this is sort of a problem depending on how you use it. I find that I don't really need to locate things that I've recently used, because I already know where they are as a result of having recently used them. But, other people have different workflows of course (I can imagine, for example, if somebody's workflow involved creating lots of files, only some of which are immediately useful, they might want the ability to scan through them for particular outputs).

varenc 5y ago

Not a direct part of the filesystem, but macOS does a good job of keeping a constantly up to date filesystem index. It powers Spotlight but is also used for other less user facing things. Running `mdfind` will locate whatever file I want near instantly and the index usually gets updated in a couple seconds. The background indexing does get bogged down when there’s a massive number of file changes, so I exclude things like “node_modules” folders. It also indexes the actual content of text files and not just filenames. Sadly the CLI interface for `mdfind` is incredibly obtuse and verbose, and very under documented [0]. When querying anything besides just a filename I still have to lookup the correct incantation. For example, here's how to do a case insensitive search for any folder containing "apple" modified in the last year: mdfind 'kMDItemDisplayName = "*apple*"c && kMDItemFSContentChangeDate <= $time.today(-365) && kMDItemKind == "Folder"' The mysterious `c` after `"apple"c` is how you specify a case insensitive search...of course. Compare that to fd: fd --changed-within=1y -t d -i apple [0] Nothing in the macOS man pages actually explains the `mdfind` query language, nor is there even a link or suggestion of where to learn more. This developer documentation sort of covers it, but it's not quite the same: https://developer.apple.com/documentation/corespotlight/cssearchquery https://developer.apple.com/documentation/corespotlight/csse...

nerdponx 5y ago
Is there an API for mdfind? If so, maybe it'd be worthwhile to try writing an alternative CLI frontend for it, if anyone is looking for a project.
- varenc 5y ago
  There is! Found a simple example: https://gist.github.com/dagronf/3d03094a7ee79c1f91607f4c365fba91 https://gist.github.com/dagronf/3d03094a7ee79c1f91607f4c365f... I've thought about just making a simpler shell script that turns a valid `fd` query into the obscure `mdfind` incantation. And you can use `-onlyin ...` to limit the search to a particular dir just like fd/find. Might get around to it!
  - nerdponx 5y ago
    That would be interesting, and I'd probably use it!