9 ms·
Parsing the .DS_Store File Format (2018)
- kakwa_ 4y agoThis article makes me wonder: what tools do you use when reversing a custom file format? Personally, having done so recently to reverse a custom game archive format, I have only used the usual hexdump, strings, grep, sed, etc. I'm wondering if there are more powerful tools to do this kind of job?
- eXpl0it3r 4y agoYou usually want something for which you can define a structure and parse the file with it. There are a bunch of commercial and free tools, I've used the following with success in the past: - 010 Editor (commercial) - Kaitai Struct (free & open source) Also has the advantage of directly generating parsers from the defined structures.
- dancemethis 4y ago...It's _not_ from the Nintendo DS?
- _ZeD_ 4y agoIs there a reliable way to make the os NOT create these `.DS_Store` files? I really don't like them and the fact that they pollute every folder. I'm on Big Sur (11.7.1) and I didn't find a way to make finder NOT create them. It's (one of the many) irritating idiosyncrasies of this OS
- suction 4y ago[dead]
- lucideer 4y agoI'm not aware of any reliable way, but anecdotally I've found that they seem to be mostly triggered by Finder usage, rather than the OS itself. I've gotten out of the habit of using Finder (instead mostly just use terminal or file-hierarchy UIs in whatever relevant app/IDE), and it's a long time since I've had a .DS_Store issue.
- alin23 4y agoI don't think there's any way to do that, the files are needed by Finder to store per-folder layouts and settings. You can however create a cronjob that deletes the files as often as you want, the operation takes 11 seconds on my 1TB SSD with a fairly large home folder: fd -u --type file --fixed-strings .DS_Store $HOME -x echo rm {} obviously, remove the echo to actually delete the files
- ryandrake 4y ago> I don't think there's any way to do that, the files are needed by Finder to store per-folder layouts and settings. I've always hated this excuse from software developers (I know it's not your excuse, alin23). User: How do I stop my computer from doing Unwanted Thing X? Dev: You can't. Unwanted Thing X is needed so we can do Unwanted Thing Y. Fucking A! Stop doing unwanted things on my computer! My computer should only be creating files I command it to create. It shouldn't be doing things I don't want it to do simply because it conveniences a developer living 1000 miles away in Silicon Valley. My filesystem is there to store my files not some developer's metadata. It should not contain this trash from my operating system. At least give me a DONT_CREATE_THIS_DS_STORE_CRAP environment variable or System Preference to set! I'll be generous and grant one root directory tree to the OS for its litter. Linux can have /var, macOS can have /System and maybe /Library too. Windows can have C:/WINDOWS. Everywhere else in my OS, hands off, please!
- plorkyeran 4y agoI don't want finder to store per-folder layouts and settings. I don't even care about the .DS_Store files very much. I just don't want the functionality that they exist to enable.
- narwally 4y agoI don't even use finder and I still find these things popping up every now or then.
- nielsbot 4y agoI think the .DS_Store file is in the classic macOS resource file format: https://dev.os9.ca/techpubs/mac/MoreToolbox/MoreToolbox-99.html https://dev.os9.ca/techpubs/mac/MoreToolbox/MoreToolbox-99.h...
- m348e912 4y ago>>Apple's operating system creates this file in apparently all directories to store meta information about its contents. In fact, it contains the names of all files (and also directories) in that folder. Anyone have an idea why modern versions of MacOS still uses a .DS_Store file? The OS could just as easily read and display the contents of a directory. And deleting a .DS_Store file doesn't seem to have any noticeable negative affect.
- GeekyBear 4y ago>Anyone have an idea why modern versions of MacOS still uses a .DS_Store file? For the exact same reason Windows has a Desktop.ini file? They store per directory settings for the Mac Finder and Windows Explorer.
- alin23 4y agoThe file is used for storing layouts (show as icons/list/grid etc.) and various settings (sorting, grouping etc.). Storing a local file inside the directory allows for moving that directory around while also keeping those settings. The file is invisible for users of Finder so it is still the best solution for this task. I have a .DS_Store line in a global .gitignore and don't notice it much but when I'm annoyed by DS_Store files in a specific directory, I usually bulk delete them using `fd`: fd -u --type file --fixed-strings .DS_Store $HOME/Projects/some-file-server-project -x echo rm {}
- gcr 4y agoi'm surprised that Apple doesn't do directory-level metadata directly in the filesystem itself, e.g. with xattrs on the directory inode.
- Tagbert 4y agoIt does store that metadata in the directing in Mac OS native file systems. The DS files are only for non-Mac file systems like FAT32.
- 4y ago
- SOLAR_FIELDS 4y agoCurious what creative use cases people have around this bizarre file format. The only reason myself and millions of other people have to deal with this stupid file is adding it to .gitignore as part of the ritual of spinning up a new codebase
- srhtftw 4y agoI instinctively delete these .DS_Store files whenever I see them but your "creative uses" comment makes me think given common .gitignore practice this might be a great place for AWS keys or whatever you never want checked into VCS /s
- pilif 4y agonitpick: As it’s a platform and machine specific thing, technically it shouldn’t be in .gitignore but in .git/info/exclude People working on Linux or Windows do not need to know about garbage your dev env leaves on your machine. I’m saying this half-seriously as I fully understand that these are so common, it’s more convenient to have the exclusion synced between clones. That said, as a purist, none of the repos I’m watching over have .DS_Store in their gitignore
- account42 4y agoI agree that platform-specific files shouldn't be in .gitignore, but it does make sense to put .* there and then only allow specific .files that you actually want to check in.
- MBCook 4y agoAlso there is a preference you can set with the defaults command to stop MacOS from putting them on file systems it encounters.
- dopidopHN 4y agoInteresting, I guess I live in impur lands :) I understand the pollution concern… but unless that .git/info/exclude is standard in the company, what prevents a intern to not have a correct exclusion there, and happily push a .ds file ? .gitignore looks more robust to the obvious user. And it’s not like it’s a file you need to look at attentively every day. But yes, I do see that you are correct!
- lapcat 4y ago(2018)
- mdmglr 4y agoRelated: If you change the folder icon, Finder will store an "Icon?" file in that folder. More info here: https://superuser.com/a/298798 https://superuser.com/a/298798 My question is: Linux and Mac both have the ability to store key-value pairs on files/folders with xattr. Why did or does Apple continue to use .DS_Store?
- dreamcompiler 4y agoGood question. Possibly because that approach wouldn't work on readonly files. (But in the case of readonly directories .DS_Store wouldn't work either). Possibly because not all copy programs copy metadata. Possibly it's just technical debt: .DS_Store was added to MacOS before xattr and rewriting the Finder to use xattr is not high priority.
- deleted 4y ago[deleted]
- flomo 4y agoWindows too with NTFS attributes. The answer is baseline FAT compatibility. But at the very least, they could set the FAT hidden attribute on this stuff. It must be some mentality like "Well, I'm not littering in my own yard?"
- ehPReth 4y agowhile we're here discussing what files should be pre-emptively added to your web server's deny entry; what else should be in there.. besides, say, .git? I guess you could block .* except .well-known 'just in case'
- pabs3 4y agoOther resources for this file format: http://fileformats.archiveteam.org/wiki/Desktop_Services_Store http://fileformats.archiveteam.org/wiki/Desktop_Services_Sto...
- eviks 4y agoIs there any chance for these dropping to go away and be replaced by a proper database of whatever Finder info is in there?
- deleted 4y ago[deleted]
- thrdbndndn 4y agoIt's not exactly the same thing, but Windows used to have a thumb.db to store, well, thumbnails, in each folder that has thumbnail-able files. I hated it, most of people hated it. MS finally got rid of it (in Vista or 7) and store thumbnails in a centralized place. However, I now start to miss it. I store lots of large image files (think 1200 dpi PNG scans) on my computer, and it took Windows years to create thumbnail for them, while locking the entire hard disk (and sometimes the whole explorer.exe) in the meantime. This process, while obviously is much slower on spinning HDD, it surprisingly isn't really that fast on NVME SSD, either. This would be fine if it's a one-time process, but unfortunately the aforementioned centralized thumbnail cache either have an expiration time, or have a size limit (or both), so the reality is if I didn't visit a folder for awhile, its thumbnails are all gone, so they have to be generated again, make the explorer unresponsive for some time again. I have been annoyed by this for long time but I don't have any solution. The best workaround is to only view such folders in list/detail mode, and rely on 3rd party image manager (I use XnView) to show the preview/thumbnail -- which has a much faster thumbnailing speed and more robust thumbnail cache management.
- simplotek 4y ago> This process, while obviously is much slower on spinning HDD, it surprisingly isn't really that fast on NVME SSD, either. Windows is renowned for having awful performance in usecases involving accessing many small files even for read-only applications. I know a team that even went as far as testing switching away from Windows to build Windows apps and instead switched to cross-builds on Linux just to avoid that performance sinkhole. Meanwhile I worked on a multiplatform project where macOS builds on a Mac mini for the exact same project took less than half the time to build than on a beefed up Windows workstation. Here's an interesting discussion on HN from a couple of years ago on this topic. https://news.ycombinator.com/item?id=18783525 https://news.ycombinator.com/item?id=18783525
- kevin_thibedeau 4y agoIt doesn't help that corporate Windows machines are infested with AV snakeoil scanners slowing down FS access.
- deleted 4y ago[deleted]