4 ms·

A file naming convention (lightweight markup) that would allow us to store structured (meta)data right inside file names. Obviously inspired by Markdown and CSV

A file naming convention (lightweight markup) that would allow us to store structured (meta)data right inside file names. Obviously inspired by Markdown and CSV. We could then build lean, database-less asset management applications, while the user data (i.e. the files and their metadata) would always be portable, across platforms. Take for example: J.M.W. Turner | Rain, Steam and Speed | ···· 1844.jpg W. Blake ···· | Newton ·············· | 1795–1805.jpg as compared to the clutter we now must deal with: _IMG00123.JPG Turner_-_Rain%2C_Steam_and_Speed_-_National_Gallery_file.jpg My practical use case: take snapshots of my incoming receipts, bills, etc., name the jpgs using the proposed file naming convention (including fields for VAT, net amount, etc.), put them in Dropbox, build a parser and accompanying GUI to edit file names (and their corresponding metadata; have total amounts etc. being calculated in real time), drop a link to that (web app) interface to my accountant. It’s just an idea for HN Idea Sunday; I did a somewhat more detailed write-up: https://gist.github.com/rhythmus/11118629 https://gist.github.com/rhythmus/11118629

tfgg 12y ago
I like this. I've often had a similar issue in computational physics when handling lots of different calculation input/output files with variations in parameters. I've tended to automatically generate directory hierarchies, e.g. rc=1.0/ecut=60/kp=4,4,4/ rc=1.0/ecut=80/kp=4,4,4/ rc=2.0/ecut=60/kp=4,4,4/ rc=2.0/ecut=80/kp=4,4,4/ but some standard way of expressing it in the file name would be great, and nicer to parse out later. There are filesystems that allow metadata, but I've never seen one really being used for that purpose. You could make some associated command line tools that are equivalent to 'ls' that allow splitting/slicing the files down different parameters.
- yummyfajitas 12y ago
  Take a look at hdf5.
  - tfgg 12y ago
    Thanks, I'll take a look, but it seems like I'd have to change the format of the files, right? Unfortunately that's not really feasible, since their format is determined by the scientific codes I use -- that's why I was attracted to the idea of using the file name. I can, however, have a go at HDF5 for code that I write myself.
jerrytsai 12y ago
+1 to this idea. Organizing photos, I would like to impart date taken, location, and context (e.g., 2014-04-29; San Francisco, CA (or geolocation); "California trip"). And I'm sure other people would love other types of metadata associated with photos. And that's just image files... plenty of other metadata-needy situations (e.g., the receipts you mentioned; I have a similar problem too).
- lnanek2 12y ago
  You can just put this in the EXIF data block of the image. File explorers like Windows Explorer show this meta data just like any other meta data about the file like file size.
lnanek2 12y ago
MetaData is already built-in to most modern file systems. On OSX (HFS Plus) you have the xattr command, and there are things for ext2, ext3, XFS, ReiserFS...
- rhythmvs 12y ago
  Sure, but it’s neither portable across platforms, nor user friendly (for non-techies). Imagine managing a collection of photos (or a mixed collection of multiple file formats, like scans of receipts, and pdfs with invoices) in something like Dropbox: no access to hidden low-level metadata blocks, etc., let alone easy editing. Not to speak of cross-platform support. File names are always visible and editable, and they’re easily parsable. I discussed such objections here: https://gist.github.com/rhythmus/11118629#problem-metadata-portability https://gist.github.com/rhythmus/11118629#problem-metadata-p...
nanofortnight 12y ago
You'd be interested in the Be File System (from BeOS / Haiku) for inspiration: http://arstechnica.com/information-technology/2010/06/the-beos-filesystem/ http://arstechnica.com/information-technology/2010/06/the-be... It has support for database-like extended file attributes, which can be custom attributes! For example, a `person` file would have all of the normal fields you'd expect from an address book as file attributes: http://www.birdhouse.org/beos/byte/24-scripting_the_bfs/ http://www.birdhouse.org/beos/byte/24-scripting_the_bfs/ https://www.haiku-os.org/docs/userguide/en/attributes.html https://www.haiku-os.org/docs/userguide/en/attributes.html Technical: http://www.letterp.com/~dbg/practical-file-system-design.pdf http://www.letterp.com/~dbg/practical-file-system-design.pdf Though you seem to be against specific file systems due to non-portability and that is understandable.
rbonvall 12y ago
I love this idea. I rely too much on custom file naming conventions, and I always end up ditching applications for managing file metadata (for example Calibre for ebooks or several others for photos and music) because using `find` covers most of the use cases and organizing the metadata takes much time.

tfgg 12y ago

I like this. I've often had a similar issue in computational physics when handling lots of different calculation input/output files with variations in parameters. I've tended to automatically generate directory hierarchies, e.g. rc=1.0/ecut=60/kp=4,4,4/ rc=1.0/ecut=80/kp=4,4,4/ rc=2.0/ecut=60/kp=4,4,4/ rc=2.0/ecut=80/kp=4,4,4/ but some standard way of expressing it in the file name would be great, and nicer to parse out later. There are filesystems that allow metadata, but I've never seen one really being used for that purpose. You could make some associated command line tools that are equivalent to 'ls' that allow splitting/slicing the files down different parameters.

yummyfajitas 12y ago
Take a look at hdf5.
- tfgg 12y ago
  Thanks, I'll take a look, but it seems like I'd have to change the format of the files, right? Unfortunately that's not really feasible, since their format is determined by the scientific codes I use -- that's why I was attracted to the idea of using the file name. I can, however, have a go at HDF5 for code that I write myself.

jerrytsai 12y ago

+1 to this idea. Organizing photos, I would like to impart date taken, location, and context (e.g., 2014-04-29; San Francisco, CA (or geolocation); "California trip"). And I'm sure other people would love other types of metadata associated with photos. And that's just image files... plenty of other metadata-needy situations (e.g., the receipts you mentioned; I have a similar problem too).

lnanek2 12y ago
You can just put this in the EXIF data block of the image. File explorers like Windows Explorer show this meta data just like any other meta data about the file like file size.

lnanek2 12y ago

MetaData is already built-in to most modern file systems. On OSX (HFS Plus) you have the xattr command, and there are things for ext2, ext3, XFS, ReiserFS...

rhythmvs 12y ago
Sure, but it’s neither portable across platforms, nor user friendly (for non-techies). Imagine managing a collection of photos (or a mixed collection of multiple file formats, like scans of receipts, and pdfs with invoices) in something like Dropbox: no access to hidden low-level metadata blocks, etc., let alone easy editing. Not to speak of cross-platform support. File names are always visible and editable, and they’re easily parsable. I discussed such objections here: https://gist.github.com/rhythmus/11118629#problem-metadata-portability https://gist.github.com/rhythmus/11118629#problem-metadata-p...

nanofortnight 12y ago

You'd be interested in the Be File System (from BeOS / Haiku) for inspiration: http://arstechnica.com/information-technology/2010/06/the-beos-filesystem/ http://arstechnica.com/information-technology/2010/06/the-be... It has support for database-like extended file attributes, which can be custom attributes! For example, a `person` file would have all of the normal fields you'd expect from an address book as file attributes: http://www.birdhouse.org/beos/byte/24-scripting_the_bfs/ http://www.birdhouse.org/beos/byte/24-scripting_the_bfs/ https://www.haiku-os.org/docs/userguide/en/attributes.html https://www.haiku-os.org/docs/userguide/en/attributes.html Technical: http://www.letterp.com/~dbg/practical-file-system-design.pdf http://www.letterp.com/~dbg/practical-file-system-design.pdf Though you seem to be against specific file systems due to non-portability and that is understandable.

rbonvall 12y ago

I love this idea. I rely too much on custom file naming conventions, and I always end up ditching applications for managing file metadata (for example Calibre for ebooks or several others for photos and music) because using `find` covers most of the use cases and organizing the metadata takes much time.