3 ms·

I think this is just a general lack of software for rare cases like clinical data? I mean, it sounds like what you have described is solved for photos. I use d

by theamk 5y ago

I think this is just a general lack of software for rare cases like clinical data? I mean, it sounds like what you have described is solved for photos. I use digikam photo manager, and it automatically discovers all the photos on multiple the volumes, and supports showing files by date, location, tags, path -- whatever you like. And it is not very fragile at all -- it identifies photos by metadata-excluded hash, so moving the photos around does not break the links. And back in Winamp days, it had an MP3 database which had basically the same properties. I have no doubt that users could use more and better organization, but it seems like UX problem, not a filesystem one.

fluidcruft 5y ago
The problem is that all the data is not the clinical data. Images are fairly well understood for clinical use, it's all the non-imaging things that need to be associated with the images. It's images and a whole bunch of stuff that's associated with them that people want to do with the data that are non-clinical. For example now there's things like BIDS[1] which are basically applying rigid schema to a filesystem. But that's only because so many people got frustrated with everyone doing their own thing and having to spend time restructuring data to different site workflows. But even that only works for neuro, what about cardiac or liver or spectroscopy or... software that doesn't use BIDS etc? And even with BIDS mostly it's just copy rename to the layouts the software you're running actually expects. There's also a huge push for "Vendor Neutral Archives" to augment PACS which will allow things to be sort of structured and managed but that always seems to be uploading and downloading from websites so people still keep copying everything out because everything is so opaque vs filesystem. > it identifies photos by metadata-excluded hash, so moving the photos around does not break the links That only solves the problem in one direction. If you run across an image after it's been moved, you know which image it is and can index into the database based on that. But if you want to find an image starting from the metadata after it's been moved, then you're stuck trawling everywhere. The metadata-hash thing exists in medical images DICOM as various GUIDs (in a different format), so you can track things and updates that using that key. But if you have to visit 30 TB of files just to find one that's been renamed or updated, it's basically impossible. [1] http://fmri.ucsd.edu/pdf/BIDS_Presentation_14NOV2018.pdf http://fmri.ucsd.edu/pdf/BIDS_Presentation_14NOV2018.pdf

by theamk 5y ago

fluidcruft 5y ago

The problem is that all the data is not the clinical data. Images are fairly well understood for clinical use, it's all the non-imaging things that need to be associated with the images. It's images and a whole bunch of stuff that's associated with them that people want to do with the data that are non-clinical. For example now there's things like BIDS[1] which are basically applying rigid schema to a filesystem. But that's only because so many people got frustrated with everyone doing their own thing and having to spend time restructuring data to different site workflows. But even that only works for neuro, what about cardiac or liver or spectroscopy or... software that doesn't use BIDS etc? And even with BIDS mostly it's just copy rename to the layouts the software you're running actually expects. There's also a huge push for "Vendor Neutral Archives" to augment PACS which will allow things to be sort of structured and managed but that always seems to be uploading and downloading from websites so people still keep copying everything out because everything is so opaque vs filesystem. > it identifies photos by metadata-excluded hash, so moving the photos around does not break the links That only solves the problem in one direction. If you run across an image after it's been moved, you know which image it is and can index into the database based on that. But if you want to find an image starting from the metadata after it's been moved, then you're stuck trawling everywhere. The metadata-hash thing exists in medical images DICOM as various GUIDs (in a different format), so you can track things and updates that using that key. But if you have to visit 30 TB of files just to find one that's been renamed or updated, it's basically impossible. [1] http://fmri.ucsd.edu/pdf/BIDS_Presentation_14NOV2018.pdf http://fmri.ucsd.edu/pdf/BIDS_Presentation_14NOV2018.pdf