11 ms·

PATH should be a system call

18 points by todsacerdoti 2y ago

jacknews 2y ago
Maybe filesystems should be object stores with metadata in a database.
- chrchr 2y ago
  https://en.m.wikipedia.org/wiki/WinFS https://en.m.wikipedia.org/wiki/WinFS
  - nirui 2y ago
    > Windows Vista, then called by its codename "Longhorn", given to developers at the Microsoft Professional Developers Conference in 2003, included WinFS, but it suffered from significant performance issues Odd, I did tried out a few Longhorn builds. Not knowing if they were shipped with WinFS, but in my case the "performance issues" of the system was caused by increased RAM demand (I only had 512MB RAM at the time). The referenced sourced by Paul Thurrott from winsupersite.com mentioned: > (https://web.archive.org/web/20070702131752/http://www.winsupersite.com/showcase/winfs_preview.asp) > > it was pretty clear that WinFS wasn't ready for prime time. As one might expect, WinFS suffered from huge performance issues and simply bogged down the regular builds. And while WinFS was indeed included with the WinHEC 2004 Longhorn build that Microsoft shipped in May 2004, Microsoft was surprisingly quiet about WinFS at that time. A few months later, we found out why. One old discussion on the subject suggested: > (https://ask.metafilter.com/129685/Why-did-WinFS-fail) > > posted by @troy at 3:03 PM on August 9, 2009: > I read that as "slow as a wee lassie on anything less than 16GB and quad dual-cores". They're waiting for PCs to be fast enough. > > posted by Ptrin at 7:13 PM on August 10, 2009: > Because Longhorn was cancelled. The WinFS project was a part of Longhorn, and when Longhorn died, WinFS did as well But the exact cause for the issues remain undisclosed. Don't you just like these close sourced hypes? LOL
    - compressedgas 2y ago
      I think WinFS failed for the same reason the Cairo Object filesystem before it did. Microsoft required WinFS to use their SQL server rather than implementing the limited structures directly in the filesystem.
IcyWindows 2y ago
Like this? https://learn.microsoft.com/en-us/windows/win32/api/shlwapi/nf-shlwapi-pathfindonpatha https://learn.microsoft.com/en-us/windows/win32/api/shlwapi/...
- LegionMammal978 2y ago
  That's not a single system call, it's just a userspace library function built on GetEnvironmentVariable() followed by a GetFileAttributes() loop, alongside a few other special cases for compatibility.
tedunangst 2y ago
Make it happen? Doesn't sound very hard to code up. Then have bash call execsomewhere() and benchmark the results.
- spyrja 2y ago
  Challenge accepted! https://news.ycombinator.com/item?id=43808428 https://news.ycombinator.com/item?id=43808428
sour-taste 2y ago
This is just for the bash case, but I was under the impression that shells cache and hash the binaries they find in the path to avoid doing these lookups. Surprised by the need for strace showing lookups.
- rascul 2y ago
  The caching is per session so starting up a new session will have an empty cache. To verify: strace bash -c 'ls;ls' You should not see the lookups for the second ls invocation.
- yencabulator 2y ago
  The hash is probably not very relevant in the age of plentiful RAM & NVMe. Mine is disabled.
bediger4000 2y ago
> the latter [Windows] is somewhat known of its less than excellent performance when it comes to small files. Not at all a Windows guy here, but my understanding was that NTFS kept the contents of small files in the FILE record itself. How does this cause low performance?
- LegionMammal978 2y ago
  Looking up and opening a file is infamously slow on Windows, due to the stack of drivers, compatibility layers, etc. it must go through. So performant Windows programs typically use fewer larger files instead of the many smaller files seen in Unix-like programming.
yencabulator 2y ago
https://www.man7.org/linux/man-pages/man3/io_uring_prep_statx.3.html https://www.man7.org/linux/man-pages/man3/io_uring_prep_stat... is there when you need it.
spyrja 2y ago
For what it is worth, I did actually try to tackle the problem after reading this. So thank you! https://news.ycombinator.com/item?id=43808428 https://news.ycombinator.com/item?id=43808428

18 points by todsacerdoti 2y ago

jacknews 2y ago

Maybe filesystems should be object stores with metadata in a database.

chrchr 2y ago
https://en.m.wikipedia.org/wiki/WinFS https://en.m.wikipedia.org/wiki/WinFS
- nirui 2y ago
  > Windows Vista, then called by its codename "Longhorn", given to developers at the Microsoft Professional Developers Conference in 2003, included WinFS, but it suffered from significant performance issues Odd, I did tried out a few Longhorn builds. Not knowing if they were shipped with WinFS, but in my case the "performance issues" of the system was caused by increased RAM demand (I only had 512MB RAM at the time). The referenced sourced by Paul Thurrott from winsupersite.com mentioned: > (https://web.archive.org/web/20070702131752/http://www.winsupersite.com/showcase/winfs_preview.asp) > > it was pretty clear that WinFS wasn't ready for prime time. As one might expect, WinFS suffered from huge performance issues and simply bogged down the regular builds. And while WinFS was indeed included with the WinHEC 2004 Longhorn build that Microsoft shipped in May 2004, Microsoft was surprisingly quiet about WinFS at that time. A few months later, we found out why. One old discussion on the subject suggested: > (https://ask.metafilter.com/129685/Why-did-WinFS-fail) > > posted by @troy at 3:03 PM on August 9, 2009: > I read that as "slow as a wee lassie on anything less than 16GB and quad dual-cores". They're waiting for PCs to be fast enough. > > posted by Ptrin at 7:13 PM on August 10, 2009: > Because Longhorn was cancelled. The WinFS project was a part of Longhorn, and when Longhorn died, WinFS did as well But the exact cause for the issues remain undisclosed. Don't you just like these close sourced hypes? LOL
  - compressedgas 2y ago
    I think WinFS failed for the same reason the Cairo Object filesystem before it did. Microsoft required WinFS to use their SQL server rather than implementing the limited structures directly in the filesystem.

IcyWindows 2y ago

Like this? https://learn.microsoft.com/en-us/windows/win32/api/shlwapi/nf-shlwapi-pathfindonpatha https://learn.microsoft.com/en-us/windows/win32/api/shlwapi/...

LegionMammal978 2y ago
That's not a single system call, it's just a userspace library function built on GetEnvironmentVariable() followed by a GetFileAttributes() loop, alongside a few other special cases for compatibility.

tedunangst 2y ago

Make it happen? Doesn't sound very hard to code up. Then have bash call execsomewhere() and benchmark the results.

spyrja 2y ago
Challenge accepted! https://news.ycombinator.com/item?id=43808428 https://news.ycombinator.com/item?id=43808428

sour-taste 2y ago

This is just for the bash case, but I was under the impression that shells cache and hash the binaries they find in the path to avoid doing these lookups. Surprised by the need for strace showing lookups.

rascul 2y ago
The caching is per session so starting up a new session will have an empty cache. To verify: strace bash -c 'ls;ls' You should not see the lookups for the second ls invocation.
yencabulator 2y ago
The hash is probably not very relevant in the age of plentiful RAM & NVMe. Mine is disabled.

bediger4000 2y ago

> the latter [Windows] is somewhat known of its less than excellent performance when it comes to small files. Not at all a Windows guy here, but my understanding was that NTFS kept the contents of small files in the FILE record itself. How does this cause low performance?

LegionMammal978 2y ago
Looking up and opening a file is infamously slow on Windows, due to the stack of drivers, compatibility layers, etc. it must go through. So performant Windows programs typically use fewer larger files instead of the many smaller files seen in Unix-like programming.

yencabulator 2y ago

https://www.man7.org/linux/man-pages/man3/io_uring_prep_statx.3.html https://www.man7.org/linux/man-pages/man3/io_uring_prep_stat... is there when you need it.

spyrja 2y ago

For what it is worth, I did actually try to tackle the problem after reading this. So thank you! https://news.ycombinator.com/item?id=43808428 https://news.ycombinator.com/item?id=43808428