10 ms·

A few random thoughts: - Wow, hell is really freezing over! - The hardest part of running bash and other posix things under windows is filesystem access. Wind

A few random thoughts: - Wow, hell is really freezing over! - The hardest part of running bash and other posix things under windows is filesystem access. Windows uses drive letters and backslashes, unix has a root filesystem with forward slashes. It seems they are taking the same route as cygwin by "mounting" windows drives in /mnt/c (or /cygdrive/c). - If you just wanted bash and some posix tools, the harder but nicer way would be to patch them to understand windows paths. It is not clear to me that it is even possible, for example many tools assume a path that does not start with a slash is a relative path - while "C:\" is absolute. You would also want to make more windows apps understand forward slashes like "C:/Windows". To make things even more complicated, there are NT native paths "\Device\HarddiskVolume4\Users\Bill", UNC paths "\\Server\share", and the crazy syntax "\\?\C:\MyReallyLongPath\File.txt". - I am really surprised this works in an appx container. From my little dabbling with modern apps in Visual Studio, I've found that they are incredibly sandboxed - no filesystem access unless you go through a file picker, no network connections to localhost (!?), no control of top-level windows, no loading of external DLLs. You can get around most restrictions for sideloaded apps, but not for windows store apps. That they can now package such a complex application as a modern app (with maybe only the linux subsystem DLLs delivered externally) means that they are slowly moving the modern/universal apps and traditional Win32 apps together with regards to their powers. - Running a Linux kernel in windows, and then ELF executables on top (without virtualization) is nothing new, see CoLinux or andLinux. If I understand correctly, this new work uses a new Linux NT subsystem. It remains to be seen if this is better (more performant) or worse (if the Linux kernel is just another process, and it crashes, it doesn't take down the system). - If they actually wrote a NT subsystem for Linux, this opens a whole can of GPL licensing worms, as you'll need to include internal NT headers. However, they say it is closed source, so I wonder how they did it. - This really stands and falls with how well it is integrated in the rest of the system. I want to install tools in "Ubuntu" via apt and use them from cmd.exe, and vice versa. And long term, a X11/Wayland bridge would be nice too.

barrkel 11y ago
I expect they implemented a subsystem for Linux at the syscall level - i.e. they implemented the Linux kernel's interface at the ABI level. No headers would need to intermingle with open source; compile on Linux, run on Windows.
- whatever_dude 11y ago
  I've read comments where people wrote that NT always had some kind of POSIX subsystem, and it has probably just now been completed and expanded. Can't find any substantial writings on this, but it seems likely.
  - barrkel 11y ago
    Not only did NT have a POSIX subsystem, but once upon a time I used it (when it was packaged as SFU, Services for Unix). It wasn't a pleasant experience. This will have been quite a bit more work than just a POSIX layer, IMO; POSIX defines a bunch of low-level functions that lie just under the C runtime library, but Linux defines a bunch more that expect a certain view of the world - things like clone(2), which lets you selectively choose what the forked process gets to inherit. POSIX only specifies fork(2). Linux implements fork(2) in terms of clone(2). And POSIX defines the API at the level of C; this will have had to implement the Linux ABI, where it differs from Win64. A minor detail of a bunch of assembly stubs, but work nonetheless.
    - LukeShu 11y ago
      To nitpick: On Linux, glibc implements fork(3) in terms of clone(2). Linux still has an independent fork(2), but nothing uses it.
    - alwaysdownvoted 11y ago
      I still use SUA every day when I'm forced to use Windows. (I also run Linux binaries on BSD.) SUA is based on an old version of BSD, not GNU. tcsh, csh and sh. The compiler works. There is an old version of lex. For some of the userland, Windows binaries are provided, such as vi. It's better than nothing. Why did MS remove SUA from Windows 10? What harm would it do to remain an optional add-in as it was in Windows 7? Why do users have to upgrade to Windows 10 to use Linux binaries? Seems like Microsoft will do _anything_ to get users to upgrade. What are the privacy implications of Windows 10? Microsoft is very untrustworthy. Will users be able to run their own Linux binaries on Windows? Windows has never been a "pleasant experience". It's the unpleasantness of it that makes the alternative, UNIX, so appealing.
mmebane 11y ago
> It seems they are taking the same route as cygwin by "mounting" windows drives in /mnt/c (or /cygdrive/c). I wonder if this Unix filesystem layer will be able to break the Windows legacy path length limit. If so, the Linux version of Node.js will suddenly become much more useful than the Windows version. > It remains to be seen if this is better (more performant) or worse Sounds like performance, at least, will be better. TFA says "it's totally shit hot! The sysbench utility is showing nearly equivalent cpu, memory, and io performance." I'll reserve judgment until I see the fork() benchmarks. :)
- drinchev 11y ago
  Just a remark. npm's team fixed long paths in v3.0, so now you have "flat tree".
  - mmebane 11y ago
    The "maximally flat" package structure in npm 3 solves the problem in most common cases, so I suppose "much more useful" was an exaggeration - if you keep your code in a relatively short base directory such as C:\src\nodeprojects\left-pad-ng\, you'll probably be fine. Depending on what version ranges dependencies are locked to, you could still end up with a pathological situation which breaks the path length limit, but I've only had that happen once since switching to npm 3, and it was easy to resolve.
- bpye 11y ago
  http://stackoverflow.com/a/67293/3965517 http://stackoverflow.com/a/67293/3965517 indicates that NT does support fork properly, just not exposed to usermode normally.
  - pcwalton 11y ago
    fork() is exposed to user mode, but it's an undocumented private API. Pass the handle of the parent process as the 4th parameter and TRUE as the 5th parameter to NtCreateProcess: http://undocumented.ntinternals.net/index.html?page=UserMode%2FUndocumented%20Functions%2FNT%20Objects%2FProcess%2FNtCreateProcess.html http://undocumented.ntinternals.net/index.html?page=UserMode... The problem is that the Win32 user mode system will fight you every step of the way: CSRSS will not understand what you just did, for example. It's not generally worth it.
icefox 11y ago
Some long term thoughts: 1) Linux has won the server (web) market. Developers would like to use a Unix box to work on their server code so they typically move to OS X. This could prevent that switch because they can still use Windows to developer their Linux server software. 2) Many projects start out as Linux and stay Linux and are only ported after much time and effort to Windows. Enterprises when faced with a tool that they want to use will also look to switch off Windows. Now rather than the cost of switching they only have to pay to upgrade their windows boxes to use the tool. 3) There is now a major incentive for developers to only build Linux binaries because it will work more places. This might cause a faster drain of developers as they eventually remove all windows specific code and can more easily migrate elsewhere. This feels eerily similar to the OS2 story and no doubt in the next week I expect to see more than a few articles discussing this very thing. 4) It will be much easier for Microsoft to bring much loved Linux tools to Windows so you can expect to see a more rapid increase of tools announced that now work for Windows. 5) Games: What about the graphical layer? Can I write the majority of my game as a Linux binary and only have the rendering bit left over to separately implement for Linux/Windows? Will this spur an increase of cross platform games?
- deleted 11y ago
  [deleted]
- cbhl 11y ago
  I imagine that 3) and 5) are hedged by how well Visual Studio is doing.
- sbarre 11y ago
  Your first point describes me exactly (and many other developers I know). I grew up a Windows user, and switched to OS X because of the unix-like command line environment that more closely matched the servers I was working with in my job environments. I would most likely switch back to Windows as my primary/only machine (because I also like to play video games sometimes) if I had the same kind of unix-like command line environment that I get in OS X. Right now I basically need 2 computers at home to meet all my needs, but this would allow me to reduce it to one, so I could get a much better one (instead of the 2 mid-range ones I have now).
barsonme 11y ago
> - If you just wanted bash and some posix tools, the harder but nicer way would be to patch them to understand windows paths. It is not clear to me that it is even possible, for example many tools assume a path that does not start with a slash is a relative path - while "C:\" is absolute. You would also want to make more windows apps understand forward slashes like "C:/Windows". To make things even more complicated, there are NT native paths "\Device\HarddiskVolume4\Users\Bill", UNC paths "\\Server\share", and the crazy syntax "\\?\C:\MyReallyLongPath\File.txt". One of the good things about using "newer" languages than C for building cross-platform utilities[0] is that things like that come baked in[1]. [0] - https://github.com/EricLagergren/go-coreutils https://github.com/EricLagergren/go-coreutils [1] - https://golang.org/pkg/path/filepath/#IsAbs https://golang.org/pkg/path/filepath/#IsAbs
quotemstr 11y ago
> Windows uses drive letters and backslashes, unix has a root filesystem with forward slashes. Windows also has mountpoints. When I use a Windows system, I pretend that C: is the only drive, and mount external volumes under the root, Unix-style. I then install Cygwin in the C:\ root, creating the illusion of a fairly Unix-ish filesystem layout.
- JoBrad 11y ago
  And you can also use forward slashes. I've used them in applications (Java apps, NodeJS apps, etc) and in the command line since at least Windows 7.
  - jcrawfordor 11y ago
    The forward slash (/) as a directory separator has always been acceptable to NT. One source says that it's been acceptable to DOS since DOS 2.0, although I don't have a lot of backing on that. The only place that / as a directory separator doesn't work is when interpreted by cmd, as it's then ambiguous with DOS-style switches. And even then, modern cmd tries to understand / properly when possible. Most of the time it works as long as it is not the first character of a path (which you would not frequently see in Windows anyway), as there's then no way to tell it apart from a switch with a long name. The canonical representation of paths in Windows uses a backslash, but 95% of the time the slashes are interchangeable.
    - quotemstr 11y ago
      IIRC (it's been a while), the native NT API does _not_ accept backslashes; win32 converts forward slashes to backslashes when passing along filenames to NT. The distinction doesn't matter very much, though, since very few people use the NT native API.
    - bboreham 11y ago
      > One source says that it's been acceptable to DOS since DOS 2.0 I can confirm that from personal memory. At the time, the biggest vendor of UNIX-like systems was Microsoft (Xenix), so compatibility was a benefit. They soon sold that product off, though.
BrandonLive 11y ago
Regarding the AppX thing... Actually AppX packaged apps (even from the Store in AppContainer) have always had direct filesystem access, but only from their install directory and to/from their AppData directories. It's only access outside those areas which requires use of the picker and/or library capabilities (which both go through a broker process). They can also of course load DLLs, and there are APIs for moving your app Windows around (with some limitations). That said, I do not know exactly what they're doing here, and it may be using some new capabilities being added to the system in the rs1 release.
rsync 11y ago
"The hardest part of running bash and other posix things under windows is filesystem access. Windows uses drive letters and backslashes, unix has a root filesystem with forward slashes. It seems they are taking the same route as cygwin by "mounting" windows drives in /mnt/c (or /cygdrive/c)." How do these ubuntu tools interact with the rest of the windows system that they are running on ? Can I 'kill -9 explorer.exe' ? Can I touch a file in /mnt/c/(whatever)/Desktop ... and have that file actually show up on my windows desktop ?
- timgws 11y ago
  PIDs from Windows are not exposed when looking at /proc, so I am guessing you will not be able to `kill -9 explorer.exe` unless there is some real voodoo going on (and then it would not be POSIX compatible, anyways). I think the command that you meant to say was killall :) `kill` will only kill pids, not process names.
  - JdeBP 11y ago
    This was one of the selling points of the old Windows NT POSIX subsystem (the Interix-derived SFU/SFUA one) over its predecessor. Programs such as "ps" and "kill" can "see" Win32 processes, with POSIX process IDs. * https://technet.microsoft.com/en-gb/library/bb463219.aspx#EHAA https://technet.microsoft.com/en-gb/library/bb463219.aspx#EH... * https://technet.microsoft.com/en-gb/magazine/2005.05.interop.aspx https://technet.microsoft.com/en-gb/magazine/2005.05.interop...

barrkel 11y ago

I expect they implemented a subsystem for Linux at the syscall level - i.e. they implemented the Linux kernel's interface at the ABI level. No headers would need to intermingle with open source; compile on Linux, run on Windows.

whatever_dude 11y ago
I've read comments where people wrote that NT always had some kind of POSIX subsystem, and it has probably just now been completed and expanded. Can't find any substantial writings on this, but it seems likely.
- barrkel 11y ago
  Not only did NT have a POSIX subsystem, but once upon a time I used it (when it was packaged as SFU, Services for Unix). It wasn't a pleasant experience. This will have been quite a bit more work than just a POSIX layer, IMO; POSIX defines a bunch of low-level functions that lie just under the C runtime library, but Linux defines a bunch more that expect a certain view of the world - things like clone(2), which lets you selectively choose what the forked process gets to inherit. POSIX only specifies fork(2). Linux implements fork(2) in terms of clone(2). And POSIX defines the API at the level of C; this will have had to implement the Linux ABI, where it differs from Win64. A minor detail of a bunch of assembly stubs, but work nonetheless.
  - LukeShu 11y ago
    To nitpick: On Linux, glibc implements fork(3) in terms of clone(2). Linux still has an independent fork(2), but nothing uses it.
  - alwaysdownvoted 11y ago
    I still use SUA every day when I'm forced to use Windows. (I also run Linux binaries on BSD.) SUA is based on an old version of BSD, not GNU. tcsh, csh and sh. The compiler works. There is an old version of lex. For some of the userland, Windows binaries are provided, such as vi. It's better than nothing. Why did MS remove SUA from Windows 10? What harm would it do to remain an optional add-in as it was in Windows 7? Why do users have to upgrade to Windows 10 to use Linux binaries? Seems like Microsoft will do _anything_ to get users to upgrade. What are the privacy implications of Windows 10? Microsoft is very untrustworthy. Will users be able to run their own Linux binaries on Windows? Windows has never been a "pleasant experience". It's the unpleasantness of it that makes the alternative, UNIX, so appealing.

mmebane 11y ago

> It seems they are taking the same route as cygwin by "mounting" windows drives in /mnt/c (or /cygdrive/c). I wonder if this Unix filesystem layer will be able to break the Windows legacy path length limit. If so, the Linux version of Node.js will suddenly become much more useful than the Windows version. > It remains to be seen if this is better (more performant) or worse Sounds like performance, at least, will be better. TFA says "it's totally shit hot! The sysbench utility is showing nearly equivalent cpu, memory, and io performance." I'll reserve judgment until I see the fork() benchmarks. :)

drinchev 11y ago
Just a remark. npm's team fixed long paths in v3.0, so now you have "flat tree".
- mmebane 11y ago
  The "maximally flat" package structure in npm 3 solves the problem in most common cases, so I suppose "much more useful" was an exaggeration - if you keep your code in a relatively short base directory such as C:\src\nodeprojects\left-pad-ng\, you'll probably be fine. Depending on what version ranges dependencies are locked to, you could still end up with a pathological situation which breaks the path length limit, but I've only had that happen once since switching to npm 3, and it was easy to resolve.
bpye 11y ago
http://stackoverflow.com/a/67293/3965517 http://stackoverflow.com/a/67293/3965517 indicates that NT does support fork properly, just not exposed to usermode normally.
- pcwalton 11y ago
  fork() is exposed to user mode, but it's an undocumented private API. Pass the handle of the parent process as the 4th parameter and TRUE as the 5th parameter to NtCreateProcess: http://undocumented.ntinternals.net/index.html?page=UserMode%2FUndocumented%20Functions%2FNT%20Objects%2FProcess%2FNtCreateProcess.html http://undocumented.ntinternals.net/index.html?page=UserMode... The problem is that the Win32 user mode system will fight you every step of the way: CSRSS will not understand what you just did, for example. It's not generally worth it.

icefox 11y ago

Some long term thoughts: 1) Linux has won the server (web) market. Developers would like to use a Unix box to work on their server code so they typically move to OS X. This could prevent that switch because they can still use Windows to developer their Linux server software. 2) Many projects start out as Linux and stay Linux and are only ported after much time and effort to Windows. Enterprises when faced with a tool that they want to use will also look to switch off Windows. Now rather than the cost of switching they only have to pay to upgrade their windows boxes to use the tool. 3) There is now a major incentive for developers to only build Linux binaries because it will work more places. This might cause a faster drain of developers as they eventually remove all windows specific code and can more easily migrate elsewhere. This feels eerily similar to the OS2 story and no doubt in the next week I expect to see more than a few articles discussing this very thing. 4) It will be much easier for Microsoft to bring much loved Linux tools to Windows so you can expect to see a more rapid increase of tools announced that now work for Windows. 5) Games: What about the graphical layer? Can I write the majority of my game as a Linux binary and only have the rendering bit left over to separately implement for Linux/Windows? Will this spur an increase of cross platform games?

deleted 11y ago
[deleted]
cbhl 11y ago
I imagine that 3) and 5) are hedged by how well Visual Studio is doing.
sbarre 11y ago
Your first point describes me exactly (and many other developers I know). I grew up a Windows user, and switched to OS X because of the unix-like command line environment that more closely matched the servers I was working with in my job environments. I would most likely switch back to Windows as my primary/only machine (because I also like to play video games sometimes) if I had the same kind of unix-like command line environment that I get in OS X. Right now I basically need 2 computers at home to meet all my needs, but this would allow me to reduce it to one, so I could get a much better one (instead of the 2 mid-range ones I have now).

barsonme 11y ago

> - If you just wanted bash and some posix tools, the harder but nicer way would be to patch them to understand windows paths. It is not clear to me that it is even possible, for example many tools assume a path that does not start with a slash is a relative path - while "C:\" is absolute. You would also want to make more windows apps understand forward slashes like "C:/Windows". To make things even more complicated, there are NT native paths "\Device\HarddiskVolume4\Users\Bill", UNC paths "\\Server\share", and the crazy syntax "\\?\C:\MyReallyLongPath\File.txt". One of the good things about using "newer" languages than C for building cross-platform utilities[0] is that things like that come baked in[1]. [0] - https://github.com/EricLagergren/go-coreutils https://github.com/EricLagergren/go-coreutils [1] - https://golang.org/pkg/path/filepath/#IsAbs https://golang.org/pkg/path/filepath/#IsAbs

quotemstr 11y ago

> Windows uses drive letters and backslashes, unix has a root filesystem with forward slashes. Windows also has mountpoints. When I use a Windows system, I pretend that C: is the only drive, and mount external volumes under the root, Unix-style. I then install Cygwin in the C:\ root, creating the illusion of a fairly Unix-ish filesystem layout.

JoBrad 11y ago
And you can also use forward slashes. I've used them in applications (Java apps, NodeJS apps, etc) and in the command line since at least Windows 7.
- jcrawfordor 11y ago
  The forward slash (/) as a directory separator has always been acceptable to NT. One source says that it's been acceptable to DOS since DOS 2.0, although I don't have a lot of backing on that. The only place that / as a directory separator doesn't work is when interpreted by cmd, as it's then ambiguous with DOS-style switches. And even then, modern cmd tries to understand / properly when possible. Most of the time it works as long as it is not the first character of a path (which you would not frequently see in Windows anyway), as there's then no way to tell it apart from a switch with a long name. The canonical representation of paths in Windows uses a backslash, but 95% of the time the slashes are interchangeable.
  - quotemstr 11y ago
    IIRC (it's been a while), the native NT API does _not_ accept backslashes; win32 converts forward slashes to backslashes when passing along filenames to NT. The distinction doesn't matter very much, though, since very few people use the NT native API.
  - bboreham 11y ago
    > One source says that it's been acceptable to DOS since DOS 2.0 I can confirm that from personal memory. At the time, the biggest vendor of UNIX-like systems was Microsoft (Xenix), so compatibility was a benefit. They soon sold that product off, though.

BrandonLive 11y ago

Regarding the AppX thing... Actually AppX packaged apps (even from the Store in AppContainer) have always had direct filesystem access, but only from their install directory and to/from their AppData directories. It's only access outside those areas which requires use of the picker and/or library capabilities (which both go through a broker process). They can also of course load DLLs, and there are APIs for moving your app Windows around (with some limitations). That said, I do not know exactly what they're doing here, and it may be using some new capabilities being added to the system in the rs1 release.

rsync 11y ago

"The hardest part of running bash and other posix things under windows is filesystem access. Windows uses drive letters and backslashes, unix has a root filesystem with forward slashes. It seems they are taking the same route as cygwin by "mounting" windows drives in /mnt/c (or /cygdrive/c)." How do these ubuntu tools interact with the rest of the windows system that they are running on ? Can I 'kill -9 explorer.exe' ? Can I touch a file in /mnt/c/(whatever)/Desktop ... and have that file actually show up on my windows desktop ?

timgws 11y ago
PIDs from Windows are not exposed when looking at /proc, so I am guessing you will not be able to `kill -9 explorer.exe` unless there is some real voodoo going on (and then it would not be POSIX compatible, anyways). I think the command that you meant to say was killall :) `kill` will only kill pids, not process names.
- JdeBP 11y ago
  This was one of the selling points of the old Windows NT POSIX subsystem (the Interix-derived SFU/SFUA one) over its predecessor. Programs such as "ps" and "kill" can "see" Win32 processes, with POSIX process IDs. * https://technet.microsoft.com/en-gb/library/bb463219.aspx#EHAA https://technet.microsoft.com/en-gb/library/bb463219.aspx#EH... * https://technet.microsoft.com/en-gb/magazine/2005.05.interop.aspx https://technet.microsoft.com/en-gb/magazine/2005.05.interop...