7 ms·

I had an idea to write a filesystem filter driver which would let you configure path remapping rules of sorts, depending on the application. Things like: - %us

by mastax 2y ago

I had an idea to write a filesystem filter driver which would let you configure path remapping rules of sorts, depending on the application. Things like: - %userprofile%\.vscode -> %appdata%\vscode - %CSIDL_MYDOCUMENTS%\Call of Duty -> %userprofile%\Saved Games\Call of Duty Because my documents and home directories filling up with a bunch of garbage which has a designated place on the filesystem filled me with impotent rage. I scaffolded out a project to write a filter driver in rust, read through the minifilter documentation, realized how much work it was going to be, and gave up. I have made my peace with the fact that a windows system is just going to be filled with garbage.

mastax 2y ago
Huh my Documents folder is weirdly empty, where did- > Shortcut to Documents (OneDrive - Personal) Oh good, now all that garbage is being synced across machines. Lovely.
rkagerer 2y ago
I just completely ignore the My Documents toxic waste dump, and use a different place (with a shorter path) for files I actually care about. I might have created directory junctions to redirect one or two programs that hardcode their data location.
pjmlp 2y ago
Just like a UNIX file system ends up filled with .dotfiles garbage I didn't ask about to have around. And better know where they come from, as there is no clean way to safely remove them.
- nicce 2y ago
  > Just like a UNIX file system ends up filled with .dotfiles garbage I didn't ask about to have around. Which one is better - a configuration file in some place or default configuration buried inside binary blob?
  - pjmlp 2y ago
    Hardly makes a difference when the outcome is the same. Also no one edits text files by hand, we use tools for that, tools that manipulate what is anyway a stream of bytes, whose meaning is given by whatever tool is used to manipulate such stream of bytes.
    - Zopieux 2y ago
      Speak for yourself.
      - pjmlp 2y ago
        I am indeed.
  - ddulaney 2y ago
    Default configuration IMO. I actually really like the way git does it, where it reads each of these in order, last one wins. - Default configuration compiled in - Global configuration - Per-user configuration - Per-project configuration You can opt-in to however much configuration complexity you need. Just cloning the occasional thing? Don't bother configuring, the global defaults are probably good enough. Simple commits and pushes? Make a user-level dotfile in your $XDG_CONFIG_HOME to set some basic info. Complex per-project aliases? If you want that, go ahead and opt in. Contrast that with programs that just dump their whole default config into my home dir on first run. Just filling up with nonsense, often no way to tell what I changed or didn’t.
    - nicce 2y ago
      That is good one, I agree. I have seen some odd cases, where the configuration options were hided too well when they were compiled to the binary itself. But that is probably an another issue.
    - hulitu 2y ago
      > You can opt-in to however much configuration complexity you need. TO DO. /s
- p_ing 2y ago
  Just get a registry! Problem solved ;) Apps are terrible at this on Windows. Sometimes it's in %USERPROFILE%, sometimes in one of the three %appdata% folders, then My Docs, other times in %ProgramData%, and even worse, %PUBLIC%. There's a registry for a reason, but even Microsoft doesn't use it for x-platform apps.
  - pjmlp 2y ago
    Like GNOME? There are rules when to use what, but usually folks rather code away instead of reading documentation.
  - keyringlight 2y ago
    The only way I can see it getting solved is the OS sandboxing everything behind the scenes, redirecting everything the application spews in random locations to The One True Profile Folder per application. Even then it wouldn't be without challenges, off the top of my head some method if you actually wanted an application to get out of its sandbox and look at the real documents folder, another method for App1 to look at files in App2's sandbox (and detecting when this is desired), or what to do if an application is able to change how it identifies itself to the OS (new versions?).
    - pjmlp 2y ago
      Hence UWP, with its rampdown, the newly introduced Win32 app sandbox model for .msix distribution.
  - the8472 2y ago
    maybe if microsoft had a registryfs...
    - p_ing 2y ago
      You can browse/manipulate the registry with PowerShell. There's an OOTB provider for it. cd HKCU: https://learn.microsoft.com/powershell/scripting/samples/working-with-registry-entries https://learn.microsoft.com/powershell/scripting/samples/wor...
- jandrese 2y ago
  Dotfiles are going out of fashion as most stuff ends up in .config now.
  - int_19h 2y ago
    The equivalent of .config on Windows is ~/AppData. The problem, as usual, is that apps ignore it and do whatever they want. Such as e.g. using dotfiles directly in ~ instead.
  - adastra22 2y ago
    That's a dotfile...
    - paride5745 2y ago
      What is meant is that instead of ~/.app/config, you are getting now ~/.config/app/config. It’s much cleaner and it comes with a more logical and useful split of config and cache stuff, e.g. you can just skip .cache when rsync-ing to a remote storage instead of having a long list of per-app escapes…
      - queuebert 2y ago
        Not really. Now when I want to uninstall something, I have to look in multiple places for its artifacts, rather than just nuking .<appname>. And there is no universal pattern of use. It's even worse than /usr vs /opt.
        ablob 2y ago
        This is an issue that really can't be fixed imo. If use the scheme "app/storage" then migration/deletion of singular apps is easy, but if you want to touch just a part of every program (e.g. delete the cache) then its filled with the same tedium you were initially complaining about with regards to uninstalling. On the other hand, the scheme "storage/app" allows you to "nuke" the cache for every installed program without touching the config at all (while lessening the ease of complete removal). As I don't usually remove programs (or install new ones, for that matter) the scheme that works best for me and many others is "storage/app". In an ideal world a hybrid scheme might be possible, but as it stands now it is not an option and one way has to be chosen over the other.
- hulitu 2y ago
  > And better know where they come from, as there is no clean way to safely remove them. Add a cron job to delete them ? /s
- queuebert 2y ago
  You mean you don't need .local, .config, .cache, .fonts, and .<appname> x <# of apps>? /s Makes me want to run off a read-only DVD.
antithesis-nl 2y ago
> I have made my peace with the fact that a windows system is just going to be filled with garbage Or, you know, create a "%USERPROFILE%\Desktop\Actual Documents" folder and set that as the default open/save location in the applications you care about? The pollution of the "My Documents" folder is unfortunate, yet nothing new, and not exactly limited to Windows (I mean, even on supposedly-perfect MacOS, I have to tolerate various game save crap under 'Documents', not to mention various 'helpful' vendor detritus), and not something that you would want to write a device driver for. (That being said, hello Rust people, welcome to the Windows kernel! Personally, I've been enjoying the freedom to write C# drivers for years, with a minimal C++ wrapper, and I'm looking forward to the fruits of your creativity!)
- lostmsu 2y ago
  He could also replace the folders with hidden junctions or symlinks.
the8472 2y ago
symlink + mark it as hidden?
7bit 2y ago
> - %CSIDL_MYDOCUMENTS%\Call of Duty -> %userprofile%\Saved Games\Call of Duty That is a fight not worth fighting. Even Microsoft does not give a care in the world when deciding on storage locations and they put files directly in your %USERPROFILE% folder. Often also naming them like Linux/Unix dotfiles. I have opened tickets, pointing them to Microsoft documentation clearly stating that this is not how it should be, but they just don't care. Still, it is infuriating seeing so much willful ignorance, when choosing the correct location is a decision just as quick as choosing the wrong location.
ww520 2y ago
May be using a symlink or a junction to redirect the folders? mklink /d link_src target_dir mklink /J link_src target_dir This has good discussion on the plus and minus of both. https://superuser.com/questions/343074/directory-junction-vs-directory-symbolic-link#343079 https://superuser.com/questions/343074/directory-junction-vs...
- ninkendo 2y ago
  Their point is that the files shouldn’t be there in their home directory in the first place. Your approach would still leave symlinks/junctions in their homedir. Their proposed driver-based approach would allow apps to think they’re writing to your homedir when they’re actually writing under appdata, leaving the homedir clean.
dist-epoch 2y ago
Microsoft/Windows itself puts a bunch of junk in the home folder, and to make things worse, they have extra long names: ntuser.ini ntuser.dat.LOG1 ntuser.dat.LOG2 NTUSER.DAT NTUSER.DAT{b2352f18-cdbf-1122-8680-002248483d79}.TM.blf NTUSER.DAT{b2352f18-cdbf-1122-8680-002248483d79}.TMContainer00000000000000000001.regtrans-ms NTUSER.DAT{b2352f18-cdbf-1122-8680-002248483d79}.TMContainer00000000000000000002.regtrans-ms
- zamadatix 2y ago
  You have to go pretty out of your way as a user to actually see those transaction logs though vs what GP is talking about with blatant pollution the user is actually expected to see and interact with.
  - wink 2y ago
    By "go out of your way" you mean "trying to copy the contents of any "system" folder in your HOME to a new machine? :)
- delta_p_delta_x 2y ago
  These files correspond to the HKCU registry hive. There is really no better place to put them, because even the location of %LOCALAPPDATA% and %APPDATA% can be controlled... in the registry.
  - dist-epoch 2y ago
    they could have shorter names
    - p_ing 2y ago
      They're transaction log files for HKCU. Microsoft typically uses GUIDs to prevent name collisions.
  - eviks 2y ago
    Of course there is a better way, don't give up so easily. One immediate improvement - use a subfolder for organization, so just 1 "garbage" item instead of many.
ryandrake 2y ago
Windows is not the only operating system treating the user's filesystem as a dumping ground. I don't know how many times I keep deleting .DS_Store, .fseventsd and extra files named ._xxxx but that doesn't keep Apple from dumping them all over my filesystem. At least macOS has one place to install applications, one place for the user's documents, and most apps somehow respect them. Then, you have designated dumping grounds like ~/Library that contains tons of junk I have no idea whether I need or not.
- p_ing 2y ago
  Two places. /Applications and ~/Applications. More, if you count UN*X binaries.
  - _blk 2y ago
    ...And you'll start having dot files and dot folders in your home when you start using brew...
    - bolognafairy 2y ago
      I mean…kind of. It depends on what you install. Homebrew is quite good at keeping itself contained, when it’s in a position to do so.
  - filleokus 2y ago
    I didn't even realise I still had stuff in ~/Applications. Probably got there via some Installer wizard... ls ~/Applications Autodesk Fusion Service Utility.app Autodesk Fusion.app Chrome Apps.localized Edge Apps.localized Remove Autodesk Fusion.app ls -ls /Applications | wc -l 94
  - maccard 2y ago
    And then all of the actual stuff that they put in Library Support
    - hobs 2y ago
jeroenhd 2y ago
Marking files as hidden seems easier on Windows. I find it much more annoying to hide files on Linux (especially folders like ~/snap which can't be renamed). If there's anything I've heard about drivers, though, it's that filesystem drivers are particularly annoying to write. If you want to simply hide the files from view, you may have an easier time writing a shell plug-in. You have to deal with COM, but you're less likely to take down the whole system.
- deleted 2y ago
  [deleted]
jiggawatts 2y ago
There are several ways of accomplishing this in user mode. The Detours library lets you attach a custom DLL during the launch of any process, intercepting Win32 calls to file system APIs by writing a custom hook. There's the built-in "compatibility shims", which are borderline undocumented, but they work similarly. They can toggle compatibility flags and redirect file and registry paths. Conveniently, these are designed to trigger only for specific EXEs based on heuristics. To support technology like App-V and later Docker containers, Windows has an entire API surface for "virtualising" the filesystem and the registry, as well as any other NT kernel namespace. Then there's the User Mode Filesystem and probably more approaches I'm not aware of or that I forgot about...
- mastax 2y ago
  Thanks. A looong time ago I looked into accomplishing it using AppContainers but that approach looked even more confusing and difficult than the kernel driver.
- davikr 2y ago
  Can you describe those options further or send references to where I can read more, specially about the API for App-V you mention? I was looking, like OP, into writing a minifilter to overlay files and folders for game modding purposes, mostly (I know of Detours - but I think injecting is somewhat dirty), but I'd rather it be done in usermode given driver signing costs and all.
  - EvanAnderson 2y ago
    You're looking for AppCompat shims. I used them to make old applications work on newer versions of Windows (in the context of educational programs that K-12 teachers wanted to keep as they moved from Win9X to NT-based Windows). Here's a decent starting place: https://techcommunity.microsoft.com/blog/askperf/demystifying-shims---or---using-the-app-compat-toolkit-to-make-your-old-stuff-wo/374947 https://techcommunity.microsoft.com/blog/askperf/demystifyin...
int_19h 2y ago
Funnily enough, Windows itself does stuff like this to allow applications written for Win9x (where e.g. Program Files were writable, so many apps would create files in their install folder). This is what the stuff in ~/AppData/Local/VirtualStore is.
yonatan8070 2y ago
I wish there was something like that for Linux so programs would stop littering my home directory with stuff that should clearly go into .config or .local.

by mastax 2y ago

mastax 2y ago

Huh my Documents folder is weirdly empty, where did- > Shortcut to Documents (OneDrive - Personal) Oh good, now all that garbage is being synced across machines. Lovely.

rkagerer 2y ago

I just completely ignore the My Documents toxic waste dump, and use a different place (with a shorter path) for files I actually care about. I might have created directory junctions to redirect one or two programs that hardcode their data location.

pjmlp 2y ago

Just like a UNIX file system ends up filled with .dotfiles garbage I didn't ask about to have around. And better know where they come from, as there is no clean way to safely remove them.

nicce 2y ago
> Just like a UNIX file system ends up filled with .dotfiles garbage I didn't ask about to have around. Which one is better - a configuration file in some place or default configuration buried inside binary blob?
- pjmlp 2y ago
  Hardly makes a difference when the outcome is the same. Also no one edits text files by hand, we use tools for that, tools that manipulate what is anyway a stream of bytes, whose meaning is given by whatever tool is used to manipulate such stream of bytes.
  - Zopieux 2y ago
    Speak for yourself.
    - pjmlp 2y ago
      I am indeed.
- ddulaney 2y ago
  Default configuration IMO. I actually really like the way git does it, where it reads each of these in order, last one wins. - Default configuration compiled in - Global configuration - Per-user configuration - Per-project configuration You can opt-in to however much configuration complexity you need. Just cloning the occasional thing? Don't bother configuring, the global defaults are probably good enough. Simple commits and pushes? Make a user-level dotfile in your $XDG_CONFIG_HOME to set some basic info. Complex per-project aliases? If you want that, go ahead and opt in. Contrast that with programs that just dump their whole default config into my home dir on first run. Just filling up with nonsense, often no way to tell what I changed or didn’t.
  - nicce 2y ago
    That is good one, I agree. I have seen some odd cases, where the configuration options were hided too well when they were compiled to the binary itself. But that is probably an another issue.
  - hulitu 2y ago
    > You can opt-in to however much configuration complexity you need. TO DO. /s
p_ing 2y ago
Just get a registry! Problem solved ;) Apps are terrible at this on Windows. Sometimes it's in %USERPROFILE%, sometimes in one of the three %appdata% folders, then My Docs, other times in %ProgramData%, and even worse, %PUBLIC%. There's a registry for a reason, but even Microsoft doesn't use it for x-platform apps.
- pjmlp 2y ago
  Like GNOME? There are rules when to use what, but usually folks rather code away instead of reading documentation.
- keyringlight 2y ago
  The only way I can see it getting solved is the OS sandboxing everything behind the scenes, redirecting everything the application spews in random locations to The One True Profile Folder per application. Even then it wouldn't be without challenges, off the top of my head some method if you actually wanted an application to get out of its sandbox and look at the real documents folder, another method for App1 to look at files in App2's sandbox (and detecting when this is desired), or what to do if an application is able to change how it identifies itself to the OS (new versions?).
  - pjmlp 2y ago
    Hence UWP, with its rampdown, the newly introduced Win32 app sandbox model for .msix distribution.
- the8472 2y ago
  maybe if microsoft had a registryfs...
  - p_ing 2y ago
    You can browse/manipulate the registry with PowerShell. There's an OOTB provider for it. cd HKCU: https://learn.microsoft.com/powershell/scripting/samples/working-with-registry-entries https://learn.microsoft.com/powershell/scripting/samples/wor...
jandrese 2y ago
Dotfiles are going out of fashion as most stuff ends up in .config now.
- int_19h 2y ago
  The equivalent of .config on Windows is ~/AppData. The problem, as usual, is that apps ignore it and do whatever they want. Such as e.g. using dotfiles directly in ~ instead.
- adastra22 2y ago
  That's a dotfile...
  - paride5745 2y ago
    What is meant is that instead of ~/.app/config, you are getting now ~/.config/app/config. It’s much cleaner and it comes with a more logical and useful split of config and cache stuff, e.g. you can just skip .cache when rsync-ing to a remote storage instead of having a long list of per-app escapes…
    - queuebert 2y ago
      Not really. Now when I want to uninstall something, I have to look in multiple places for its artifacts, rather than just nuking .<appname>. And there is no universal pattern of use. It's even worse than /usr vs /opt.
      - ablob 2y ago
        This is an issue that really can't be fixed imo. If use the scheme "app/storage" then migration/deletion of singular apps is easy, but if you want to touch just a part of every program (e.g. delete the cache) then its filled with the same tedium you were initially complaining about with regards to uninstalling. On the other hand, the scheme "storage/app" allows you to "nuke" the cache for every installed program without touching the config at all (while lessening the ease of complete removal). As I don't usually remove programs (or install new ones, for that matter) the scheme that works best for me and many others is "storage/app". In an ideal world a hybrid scheme might be possible, but as it stands now it is not an option and one way has to be chosen over the other.
hulitu 2y ago
> And better know where they come from, as there is no clean way to safely remove them. Add a cron job to delete them ? /s
queuebert 2y ago
You mean you don't need .local, .config, .cache, .fonts, and .<appname> x <# of apps>? /s Makes me want to run off a read-only DVD.

antithesis-nl 2y ago

> I have made my peace with the fact that a windows system is just going to be filled with garbage Or, you know, create a "%USERPROFILE%\Desktop\Actual Documents" folder and set that as the default open/save location in the applications you care about? The pollution of the "My Documents" folder is unfortunate, yet nothing new, and not exactly limited to Windows (I mean, even on supposedly-perfect MacOS, I have to tolerate various game save crap under 'Documents', not to mention various 'helpful' vendor detritus), and not something that you would want to write a device driver for. (That being said, hello Rust people, welcome to the Windows kernel! Personally, I've been enjoying the freedom to write C# drivers for years, with a minimal C++ wrapper, and I'm looking forward to the fruits of your creativity!)

lostmsu 2y ago
He could also replace the folders with hidden junctions or symlinks.

the8472 2y ago

symlink + mark it as hidden?

7bit 2y ago

> - %CSIDL_MYDOCUMENTS%\Call of Duty -> %userprofile%\Saved Games\Call of Duty That is a fight not worth fighting. Even Microsoft does not give a care in the world when deciding on storage locations and they put files directly in your %USERPROFILE% folder. Often also naming them like Linux/Unix dotfiles. I have opened tickets, pointing them to Microsoft documentation clearly stating that this is not how it should be, but they just don't care. Still, it is infuriating seeing so much willful ignorance, when choosing the correct location is a decision just as quick as choosing the wrong location.

ww520 2y ago

May be using a symlink or a junction to redirect the folders? mklink /d link_src target_dir mklink /J link_src target_dir This has good discussion on the plus and minus of both. https://superuser.com/questions/343074/directory-junction-vs-directory-symbolic-link#343079 https://superuser.com/questions/343074/directory-junction-vs...

ninkendo 2y ago
Their point is that the files shouldn’t be there in their home directory in the first place. Your approach would still leave symlinks/junctions in their homedir. Their proposed driver-based approach would allow apps to think they’re writing to your homedir when they’re actually writing under appdata, leaving the homedir clean.

dist-epoch 2y ago

Microsoft/Windows itself puts a bunch of junk in the home folder, and to make things worse, they have extra long names: ntuser.ini ntuser.dat.LOG1 ntuser.dat.LOG2 NTUSER.DAT NTUSER.DAT{b2352f18-cdbf-1122-8680-002248483d79}.TM.blf NTUSER.DAT{b2352f18-cdbf-1122-8680-002248483d79}.TMContainer00000000000000000001.regtrans-ms NTUSER.DAT{b2352f18-cdbf-1122-8680-002248483d79}.TMContainer00000000000000000002.regtrans-ms

zamadatix 2y ago
You have to go pretty out of your way as a user to actually see those transaction logs though vs what GP is talking about with blatant pollution the user is actually expected to see and interact with.
- wink 2y ago
  By "go out of your way" you mean "trying to copy the contents of any "system" folder in your HOME to a new machine? :)
delta_p_delta_x 2y ago
These files correspond to the HKCU registry hive. There is really no better place to put them, because even the location of %LOCALAPPDATA% and %APPDATA% can be controlled... in the registry.
- dist-epoch 2y ago
  they could have shorter names
  - p_ing 2y ago
    They're transaction log files for HKCU. Microsoft typically uses GUIDs to prevent name collisions.
- eviks 2y ago
  Of course there is a better way, don't give up so easily. One immediate improvement - use a subfolder for organization, so just 1 "garbage" item instead of many.

ryandrake 2y ago

Windows is not the only operating system treating the user's filesystem as a dumping ground. I don't know how many times I keep deleting .DS_Store, .fseventsd and extra files named ._xxxx but that doesn't keep Apple from dumping them all over my filesystem. At least macOS has one place to install applications, one place for the user's documents, and most apps somehow respect them. Then, you have designated dumping grounds like ~/Library that contains tons of junk I have no idea whether I need or not.

p_ing 2y ago
Two places. /Applications and ~/Applications. More, if you count UN*X binaries.
- _blk 2y ago
  ...And you'll start having dot files and dot folders in your home when you start using brew...
  - bolognafairy 2y ago
    I mean…kind of. It depends on what you install. Homebrew is quite good at keeping itself contained, when it’s in a position to do so.
- filleokus 2y ago
  I didn't even realise I still had stuff in ~/Applications. Probably got there via some Installer wizard... ls ~/Applications Autodesk Fusion Service Utility.app Autodesk Fusion.app Chrome Apps.localized Edge Apps.localized Remove Autodesk Fusion.app ls -ls /Applications | wc -l 94
- maccard 2y ago
  And then all of the actual stuff that they put in Library Support
  - hobs 2y ago

jeroenhd 2y ago

Marking files as hidden seems easier on Windows. I find it much more annoying to hide files on Linux (especially folders like ~/snap which can't be renamed). If there's anything I've heard about drivers, though, it's that filesystem drivers are particularly annoying to write. If you want to simply hide the files from view, you may have an easier time writing a shell plug-in. You have to deal with COM, but you're less likely to take down the whole system.

deleted 2y ago
[deleted]

jiggawatts 2y ago

There are several ways of accomplishing this in user mode. The Detours library lets you attach a custom DLL during the launch of any process, intercepting Win32 calls to file system APIs by writing a custom hook. There's the built-in "compatibility shims", which are borderline undocumented, but they work similarly. They can toggle compatibility flags and redirect file and registry paths. Conveniently, these are designed to trigger only for specific EXEs based on heuristics. To support technology like App-V and later Docker containers, Windows has an entire API surface for "virtualising" the filesystem and the registry, as well as any other NT kernel namespace. Then there's the User Mode Filesystem and probably more approaches I'm not aware of or that I forgot about...

mastax 2y ago
Thanks. A looong time ago I looked into accomplishing it using AppContainers but that approach looked even more confusing and difficult than the kernel driver.
davikr 2y ago
Can you describe those options further or send references to where I can read more, specially about the API for App-V you mention? I was looking, like OP, into writing a minifilter to overlay files and folders for game modding purposes, mostly (I know of Detours - but I think injecting is somewhat dirty), but I'd rather it be done in usermode given driver signing costs and all.
- EvanAnderson 2y ago
  You're looking for AppCompat shims. I used them to make old applications work on newer versions of Windows (in the context of educational programs that K-12 teachers wanted to keep as they moved from Win9X to NT-based Windows). Here's a decent starting place: https://techcommunity.microsoft.com/blog/askperf/demystifying-shims---or---using-the-app-compat-toolkit-to-make-your-old-stuff-wo/374947 https://techcommunity.microsoft.com/blog/askperf/demystifyin...

int_19h 2y ago

Funnily enough, Windows itself does stuff like this to allow applications written for Win9x (where e.g. Program Files were writable, so many apps would create files in their install folder). This is what the stuff in ~/AppData/Local/VirtualStore is.

yonatan8070 2y ago

I wish there was something like that for Linux so programs would stop littering my home directory with stuff that should clearly go into .config or .local.