5 ms·
Byenow – A command-line Windows utility for faster folder removal
- jmnicolas 7y agoI always assumed that a delete command would execute at max disk capacity on a single thread and that multi-threading it would even slow the command (more random accesses on the disk). This utility would have been a godsend a couple months ago when I had millions of XML files to delete on one of our server (12 years of accumulation). It took several days (it's an old Windows 2003 server with SCSI disks I think).
- apankrat 7y agoThis is mine. I was meaning to do a Show HN post for it actually, just was waiting for the weekend :) I've been using this internally for several months. For local deletes YMMV. There seem to be some sort of central lock in the NTFS driver, so the speed up comes from being faster with scanning and generally idling less. For freshly mounted fast drives you may see a speed up of 1.5-2.5 times, but in other cases it will be just 10-20%. However for remote deletes this tool is really great. Due to the network latency, single-threaded deletes will spend most of their time idle - a request needs to get there and then back before the next one can be fired. But using threads allows to have multiple requests "in flight", forming a queue if needed, so the remote end is always doing something. A typical speed up for a LAN is around 2x, and for VPN links this should be even higher. In any case, AMA.
- ptah 7y agowill it delete files even if open in another process? this is my main bugbear with windows file deletion: it won't delete it and you need special tools to track down the process that has the file open
- apankrat 7y agoDepends on how they are opened. If they aren't opened in FILE_SHARE_DELETE mode, then they will in fact be locked against deletion... because that's exactly what file's current user wanted. Force-deleting a file in this case is not the wisest thing to do. [1] https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea#parameters https://docs.microsoft.com/en-us/windows/win32/api/fileapi/n..., Ctrl-F for FILE_SHARE_DELETE
- pjmlp 7y agoWhich is the right thing to do and the approach taken in all non-UNIX clones. In any case, since ages one does not need special tools, resource monitor listing of open handles is enough.
- BubRoss 7y agoIt's the right thing to give an error that some process has the file open but never tell you which process that is?
- pjmlp 7y agoIt might not be the most user-friendly way, but sure tops removing the file altogether.
- latitude 7y agoThat’s an UI issue with the tool. The underlying behavior is correct though - people with open handles have the priority unless they have explicitly stated that deleting a file while they use it is OK.
- BubRoss 7y agoWhat tool? That's windows' behavior. I would agree that locking a file and preventing deletion is good or possibly queueing a file for deletion when it is free, but windows does not handle this scenario well.
- Ididntdothis 7y agoTelling you what process has a folder or file open should be added to windows explorer. It’s hard to explain why it isn’t.
- SAI_Peregrinus 7y agoA workaround to this is to use Process Explorer[1] to search for the filename, then kill the process that has a handle with that file name. [1] https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer https://docs.microsoft.com/en-us/sysinternals/downloads/proc...
- e12e 7y agoThank you for sharing. You write: > The source is incomplete, because it depends on several libraries that are not a part of this distribution. However, with an exception of simple_work_queue, these libraries are simple Win32 API wrappers and their functionality should be fairly obvious from their usage. Just curious - if they're trivial wrappers - why not share them?
- apankrat 7y agoA couple of reasons. It takes time to prep a public release. Review the sources, comment them, check formatting, add docs, legalese, build instructions, etc. I really don't like publishing stuff that's not in a proper order. Second is that a public release needs to be maintained and this complicates our own workflow. We can, in theory, do a release and then abandon it, but it doesn't seem like a right thing to do.
- stallmanite 7y agoFirst of all, great piece of software thanks for sharing it. Secondly re: releasing then abandoning some subset of code being “not the right thing”, I would submit that as a user I’d much prefer abandoned code than unreleased code. Please consider just slapping a disclaimer on it and releasing it.
- voltagex_ 7y agoOne of these days I hope to get a peek at how you build UIs - you've got some of the best looking software for Windows.
- apankrat 7y agoThanks, appreciate the compliment, but there's not much to it really. Do version A, use it, change what's off or annoying, rinse and repeat. At some point it always converges to a stable point. Pretty much the only criteria is to remove friction from fast paths and add some where safeguards are needed.
- fctorial 7y agoBtw is there a cli trash utility for linux that handles multiple partitions correctly?
- alkonaut 7y agoIt always amazed me that deleting a remote directory with thousands of files can be so much slower than remoting into the server, and deleting the directory. I don't want to recursively list the remote files and then delete them one by one. I just want to remove the directory from the remote filesystem! Can a tool solve that problem? (I guess powershell remote scripts could?)
- apankrat 7y agoThere's no way to delete multiple files with a single syscall. There's also no way to purge the folder of all its contents. At least not on Windows, using an API.
- alkonaut 7y agoThe syscalls are quick enough when done locally. What I think is going on is that for a directory with N files on a share, I get N (or likely multiples of N) network roundtrips. If the network round trip time is 1second and the time to delete the directory with 1000 files is 1 second when done on the local drive, then I want the delete on the network share to be 2 seconds not 2000 or 4000 seconds.
- apankrat 7y agoFor deleting - yes, you will get a round-trip per delete request. So if they are serialized (which is the case with standard tools), it will be excruciatingly slow indeed. For scanning - you'll get the list of content in chunks. The size of a chunk depends on the size of the buffer you provide. For smaller folders it'll normally fit in a single request.
- pbhjpbhj 7y ago(I'm just a hobbyist) Is that how .+n[i|u]x's unlink command works? IIRC you can unlink a directory. If that's true, how does MS's WSL handle unlink? Why doesn't Windows del work like that, surely at the physical layer the drive is only removing allocation entries?
- mvladic 7y agoFYI builtin command "RMDIR /Q/S foldername" is fast, at least much faster then using windows file explorer.
- apankrat 7y agoIt's mentioned on the homepage. There's also "del /f /s /q" to delete just files, followed by "rmdir /q /s" to delete just folders; and there's also "robocopy /mir" with an empty folder as a source. From what I've seen straight "rmdir" is the fastest of these three.
- 5bolts 7y agojust "RD Foo /S" if you know you want it gone why all the extra flags and steps
- jussij 7y agoNow I don't know is this makes things slower, but Windows File Explorer by default never deletes files. It just moves them to the recycle bin. If you want to bypass that step you need to use the Shift + Del keys and for that scenario Windows even warns you the action you are request is a permanent delete.
- satya71 7y agoCame to say the same. `rmdir /s` is an order of magnitude faster on windows than other methods I have tried.
- alex3165 7y agobyenow node_modules - let's see if it can keep up
- thunderbong 7y agoThis was my test actually. Because that is the exact reason that I would use this for. And it was spectacular. I'm amazed at the speed at which it clears out an entire folder. Another nice touch is if I ask it to delete a folder and that folder is open in a command prompt somewhere, it just goes ahead and deletes it's contents cleanly. Excellent piece of software.
- posedge 7y agoFunny that you need a third-party tool to do that, isn't it.
- SeriousM 7y agoIs there any difference to npm rimraf? This is removing the node modules folder in lightning speed