5 ms·

Make tags not trees - filesystem based on tags instead of directories

27 points by 10ren 17y ago

dlib 17y ago
I've been thinking of pretty much the same thing ever since I got a Fujitsu Scansnap scanner to digitize all the documents I have. Tagging is often more appropriate to categorize my documents than hierarchical folders. Spotlight comments may make up for this although they are a bit cumbersome. Right now I only enter a comment for files I know I want to find back. I have also considered Evernote to handle my document storage (and always accessible, including search) accompanied with its tagging features but I found it too slow and plain directories combined with Spotlight worked better for me. These directories are in my Dropbox folder so I can access them anywhere, some hierarchies are applied for Bank statements etc. but most files are in one big folder. It's a pity there is no good searching in documents from the Dropbox site.
- gry 17y ago
  I want to be able to file documents in one place, but tag them for action. I've been meaning to find which one of these fits my workflow: Quicksilver tag plugin [http://lifehacker.com/169971/metadata-as-a-filing-system http://lifehacker.com/169971/metadata-as-a-filing-system] TagBot [http://bigrobotsoftware.com/ http://bigrobotsoftware.com/] Punakea [http://www.nudgenudge.eu/punakea http://www.nudgenudge.eu/punakea] They all use Spotlight metadata to accomplish the tagging, Punakea seems the most modal, which I like less.
windsurfer 17y ago
What's wrong with the traditional folder full of symlinks? This article doesn't talk about symbolic links at all, and they would solve all the "problems" discussed here.
- moe 17y ago
  This article doesn't talk about symbolic links at all, and they would solve all the "problems" discussed here. The only problem symlinks can address would that of having files available under multiple locations. But that's still strictly hierarchical and bound to become a maintenance nightmare soon (just think about file renames). Tag based filesystems are indeed an interesting idea and a somewhat logical next step from the traditional hierarchical fs + desktop search tandem that is commonplace now.
  - ThinkWriteMute 17y ago
    Never going to happen. Far too many people like the person you replied too are going to scream that we're just not using the regular tools 'right'.
    - TomasSedovic 17y ago
      True -- on the desktop. But most web-based storages are doing this already (gmail, flickr, delicious). I do believe that desktops will lose their significance over time and the web experience will be the next thing. And then it may even migrate to the OS itself, because it will be the familiar thing.
  - thwarted 17y ago
    If you were going to do it like this, hard links would make a better option. And in a tag based file system, the actual on-filesystem "filename" isn't as important as the path, er list of tags. This would end up being a maintenance nightmare for different reason, though. You'd end up with two dirs, for example work/projectX and projectX/work, with some files in one and some files in another. Strict hierarchies don't map to tags.
  - isb 17y ago
    A while back I wrote a Fuse filesystem for Mac (http://code.google.com/p/xtagfs/ http://code.google.com/p/xtagfs/) that does exactly what the blog proposes. The fs just provides the directory structure on top of an existing filesystem and the files are symlinks to the actual files on the filesystem.
j_baker 17y ago
Windows 7 has essentially added a much more basic version of this called "libraries".
xtho 17y ago
Haven't there been some efforts in implementing "database filesystems"? Wasn't this supposed to be a key feature of Windows Vista?
- julio_the_squid 17y ago
  An FS that supported indexing and querying of files through metadata was a key feature of BeOS, actually. And yes, MS claimed their project, 'WinFS', would be a key part of Vista (aka Longhorn), but then dropped it in 2006.
  - agbell 17y ago
    Seems like MS has had go at this problem many times since 1990 and never cracked it. http://en.wikipedia.org/wiki/WinFS#Development http://en.wikipedia.org/wiki/WinFS#Development
  - shawndumas 17y ago
    I thought of BeOS immediately. Then iTunes, actually. I can always quickly find what I am looking for in itunes on account of the column browser.
- gnosis 17y ago
  A regular filesystem really is a database, actually... a hierarchical, tree-structured one.
oconnore 17y ago
It seems like you could easily build a file browser like this on top of the file system. That way you preserve operating system hierarchies while getting all the benefits the article talks about. If you kept a database of tags mapped to files, you could keep every file in the same folder, without suffering from linear search of the directory. But really I think the author just needs to discover 'locate'...
- access_denied 17y ago
  The problem with that: retain meta-info when copying from machine to machine. Otherwise: yes.
- gregdetre 17y ago
  Yup! The question was: how might we move to a tag-based infrastructure while preserving our current interface, so that all of our existing applications and File/Open dialogs would still work.
  - cgcgcgcg 17y ago
    I've thought about this a little. The 3 options I could think of, along with their issues: 1. Using an OS's comments/metadata field to store tags and include them in a desktop-search program -- the comments/metadata field is not compatible between operating systems and can be lost while copying files between filesystems or operating systems, so this is not a robust solution. And currently only OS X supports comments/metadata field for all file types. 2. Using a central database that does all the file-to-tag association -- this may be 'less universal' or robust than #1; to be cross-platform this requires someone to write a new program that plugs into a given OS and runs constantly monitoring files in case you move or rename them. 3. Storing the tags in the filename itself -- this will give you long, ugly filenames, but should be the most robust because all of the modern OS's and filesystems support 256 character filenames (usually including the path) which gives plenty of room for a number of tags separated by some kind of delimiter character e.g. filename###tag1#tag2#tag3.ext . This would need just a simple tag-management program to help you tag files (and can function as a nice file explorer as well, like http://www.nudgenudge.eu http://www.nudgenudge.eu and its clone http://lunarfrog.com/taggedfrog http://lunarfrog.com/taggedfrog) which does not need to be constantly running, and could use whatever desktop-search program to use the files. Therefore at the moment I think #3 would be the most robust and versatile solution. Then someone just has to make a frontend for file renaming and tag-database storage/rebuilding from filenames/viewing...
bayareaguy 17y ago
The original goal of the Namesys filesystem (what eventually became ReiserFS) was a similar attempt to replace fixed hierarchies with a form of tagging. While the original paper is no longer accessible, I found a copy here: http://www.dmi.me.uk/blog/wp-content/uploads/2007/11/reiser-the-naming-system-venture.pdf http://www.dmi.me.uk/blog/wp-content/uploads/2007/11/reiser-...
epochwolf 17y ago
I love this idea but plain tags are simply not enough. I still have a need for hierarchy. If I can have tags like "programming/ruby" and "school/2008-fall/ecology" be organized like mini-filesystems I would be completely sold.
- moe 17y ago
  Why not just "programming ruby" and "school 2008 fall ecology"?
- gregdetre 17y ago
  Hey Epochwolf, I hope I'm not misunderstanding you, but I think that's exactly what I was trying to suggest in the article. The idea was that you could have: - school/2008-fall/ecology and that would be identical to: - ecology/2008-fall/school - 2008-fall/ecology/school etc. To put this broadly in terms of data structures, our current filesystems use an ordered list, whereas I'm proposing using an (unordered) set. g
gnosis 17y ago
Some related projects: http://www.pytagsfs.org/Introduction http://www.pytagsfs.org/Introduction http://code.google.com/p/dhtfs/wiki/UserGuide http://code.google.com/p/dhtfs/wiki/UserGuide http://pages.stern.nyu.edu/~marriaga/software/oyepa/ http://pages.stern.nyu.edu/~marriaga/software/oyepa/ http://manishrjain.googlepages.com/flickrfs http://manishrjain.googlepages.com/flickrfs

27 points by 10ren 17y ago

dlib 17y ago

I've been thinking of pretty much the same thing ever since I got a Fujitsu Scansnap scanner to digitize all the documents I have. Tagging is often more appropriate to categorize my documents than hierarchical folders. Spotlight comments may make up for this although they are a bit cumbersome. Right now I only enter a comment for files I know I want to find back. I have also considered Evernote to handle my document storage (and always accessible, including search) accompanied with its tagging features but I found it too slow and plain directories combined with Spotlight worked better for me. These directories are in my Dropbox folder so I can access them anywhere, some hierarchies are applied for Bank statements etc. but most files are in one big folder. It's a pity there is no good searching in documents from the Dropbox site.

gry 17y ago
I want to be able to file documents in one place, but tag them for action. I've been meaning to find which one of these fits my workflow: Quicksilver tag plugin [http://lifehacker.com/169971/metadata-as-a-filing-system http://lifehacker.com/169971/metadata-as-a-filing-system] TagBot [http://bigrobotsoftware.com/ http://bigrobotsoftware.com/] Punakea [http://www.nudgenudge.eu/punakea http://www.nudgenudge.eu/punakea] They all use Spotlight metadata to accomplish the tagging, Punakea seems the most modal, which I like less.

windsurfer 17y ago

What's wrong with the traditional folder full of symlinks? This article doesn't talk about symbolic links at all, and they would solve all the "problems" discussed here.

moe 17y ago
This article doesn't talk about symbolic links at all, and they would solve all the "problems" discussed here. The only problem symlinks can address would that of having files available under multiple locations. But that's still strictly hierarchical and bound to become a maintenance nightmare soon (just think about file renames). Tag based filesystems are indeed an interesting idea and a somewhat logical next step from the traditional hierarchical fs + desktop search tandem that is commonplace now.
- ThinkWriteMute 17y ago
  Never going to happen. Far too many people like the person you replied too are going to scream that we're just not using the regular tools 'right'.
  - TomasSedovic 17y ago
    True -- on the desktop. But most web-based storages are doing this already (gmail, flickr, delicious). I do believe that desktops will lose their significance over time and the web experience will be the next thing. And then it may even migrate to the OS itself, because it will be the familiar thing.
- thwarted 17y ago
  If you were going to do it like this, hard links would make a better option. And in a tag based file system, the actual on-filesystem "filename" isn't as important as the path, er list of tags. This would end up being a maintenance nightmare for different reason, though. You'd end up with two dirs, for example work/projectX and projectX/work, with some files in one and some files in another. Strict hierarchies don't map to tags.
- isb 17y ago
  A while back I wrote a Fuse filesystem for Mac (http://code.google.com/p/xtagfs/ http://code.google.com/p/xtagfs/) that does exactly what the blog proposes. The fs just provides the directory structure on top of an existing filesystem and the files are symlinks to the actual files on the filesystem.

j_baker 17y ago

Windows 7 has essentially added a much more basic version of this called "libraries".

xtho 17y ago

Haven't there been some efforts in implementing "database filesystems"? Wasn't this supposed to be a key feature of Windows Vista?

julio_the_squid 17y ago
An FS that supported indexing and querying of files through metadata was a key feature of BeOS, actually. And yes, MS claimed their project, 'WinFS', would be a key part of Vista (aka Longhorn), but then dropped it in 2006.
- agbell 17y ago
  Seems like MS has had go at this problem many times since 1990 and never cracked it. http://en.wikipedia.org/wiki/WinFS#Development http://en.wikipedia.org/wiki/WinFS#Development
- shawndumas 17y ago
  I thought of BeOS immediately. Then iTunes, actually. I can always quickly find what I am looking for in itunes on account of the column browser.
gnosis 17y ago
A regular filesystem really is a database, actually... a hierarchical, tree-structured one.

oconnore 17y ago

It seems like you could easily build a file browser like this on top of the file system. That way you preserve operating system hierarchies while getting all the benefits the article talks about. If you kept a database of tags mapped to files, you could keep every file in the same folder, without suffering from linear search of the directory. But really I think the author just needs to discover 'locate'...

access_denied 17y ago
The problem with that: retain meta-info when copying from machine to machine. Otherwise: yes.
gregdetre 17y ago
Yup! The question was: how might we move to a tag-based infrastructure while preserving our current interface, so that all of our existing applications and File/Open dialogs would still work.
- cgcgcgcg 17y ago
  I've thought about this a little. The 3 options I could think of, along with their issues: 1. Using an OS's comments/metadata field to store tags and include them in a desktop-search program -- the comments/metadata field is not compatible between operating systems and can be lost while copying files between filesystems or operating systems, so this is not a robust solution. And currently only OS X supports comments/metadata field for all file types. 2. Using a central database that does all the file-to-tag association -- this may be 'less universal' or robust than #1; to be cross-platform this requires someone to write a new program that plugs into a given OS and runs constantly monitoring files in case you move or rename them. 3. Storing the tags in the filename itself -- this will give you long, ugly filenames, but should be the most robust because all of the modern OS's and filesystems support 256 character filenames (usually including the path) which gives plenty of room for a number of tags separated by some kind of delimiter character e.g. filename###tag1#tag2#tag3.ext . This would need just a simple tag-management program to help you tag files (and can function as a nice file explorer as well, like http://www.nudgenudge.eu http://www.nudgenudge.eu and its clone http://lunarfrog.com/taggedfrog http://lunarfrog.com/taggedfrog) which does not need to be constantly running, and could use whatever desktop-search program to use the files. Therefore at the moment I think #3 would be the most robust and versatile solution. Then someone just has to make a frontend for file renaming and tag-database storage/rebuilding from filenames/viewing...

bayareaguy 17y ago

The original goal of the Namesys filesystem (what eventually became ReiserFS) was a similar attempt to replace fixed hierarchies with a form of tagging. While the original paper is no longer accessible, I found a copy here: http://www.dmi.me.uk/blog/wp-content/uploads/2007/11/reiser-the-naming-system-venture.pdf http://www.dmi.me.uk/blog/wp-content/uploads/2007/11/reiser-...

epochwolf 17y ago

I love this idea but plain tags are simply not enough. I still have a need for hierarchy. If I can have tags like "programming/ruby" and "school/2008-fall/ecology" be organized like mini-filesystems I would be completely sold.

moe 17y ago
Why not just "programming ruby" and "school 2008 fall ecology"?
gregdetre 17y ago
Hey Epochwolf, I hope I'm not misunderstanding you, but I think that's exactly what I was trying to suggest in the article. The idea was that you could have: - school/2008-fall/ecology and that would be identical to: - ecology/2008-fall/school - 2008-fall/ecology/school etc. To put this broadly in terms of data structures, our current filesystems use an ordered list, whereas I'm proposing using an (unordered) set. g

gnosis 17y ago

Some related projects: http://www.pytagsfs.org/Introduction http://www.pytagsfs.org/Introduction http://code.google.com/p/dhtfs/wiki/UserGuide http://code.google.com/p/dhtfs/wiki/UserGuide http://pages.stern.nyu.edu/~marriaga/software/oyepa/ http://pages.stern.nyu.edu/~marriaga/software/oyepa/ http://manishrjain.googlepages.com/flickrfs http://manishrjain.googlepages.com/flickrfs