4 ms·

Seems weird to upstream this support all as one shared kernel interface, modelled as only “NT emulation.” I’m surprised the kernel would be okay with this impl

by derefr 3y ago

Seems weird to upstream this support all as one shared kernel interface, modelled as only “NT emulation.” I’m surprised the kernel would be okay with this impl. It reminds me of when OpenVZ tried to upstream their big blob of container logic. The kernel maintainers forced them to part it out into individual patches so it could be reworked (and we ended up with cgroups and namespaces.) If Linux is missing certain kernel-provided sync primitives, why not just implement them all as regular Linux syscalls for every userland app to use (similar to how e.g. futexes were introduced) rather then crowding them all into some compatibility chardev with an ABI arcane enough that it would only ever be talked to by one library?

rrdharan 3y ago
cgroups was built by Google https://en.wikipedia.org/wiki/Cgroups https://en.wikipedia.org/wiki/Cgroups
arka2147483647 3y ago
I imagine the point is not to implement some species of sync primitives currently not supported by Linux, but to implement exactly what windows does, so one does not need to emulate them in userspace.
- derefr 3y ago
  The point for the contributor is that, yes. The point that would convince the Linux kernel maintainers to actually accept the patchset, though, would likely be the introduction of generally-useful "species of sync primitives." In such a way that those primitives can be used to solve the "hard part" of WINE's virtualization of NT's versions of those primitives; but which wouldn't be constrained to guarantee an efficient zero-impedance call path between those implementations.
the8472 3y ago
The slides[0] (page 17) mention alternative solutions. One of them is faster userspace switching. That sounds a bit like something google has proposed before with switch_to/FUTEX_SWAP[1] [0] https://lpc.events/event/17/contributions/1517/attachments/1257/2549/lpc2023.pdf https://lpc.events/event/17/contributions/1517/attachments/1... [1] https://lwn.net/Articles/824409/ https://lwn.net/Articles/824409/
- pcwalton 3y ago
  I really want the switchto patch for high performance userspace threading. It'd potentially allow 1:1 threading to better compete with M:N, which would be very nice for systems languages like C++ and Rust.
cwillu 3y ago
This _is_ broken up into individual patches, with documentation and intent discussed (briefly) in each patch. It's not a big blob of undocumented and opaque container logic being dumped in one giant patch.
- derefr 3y ago
  Sure, but all of these patches still have a common implicit "framework" they're introducing ABI surface to — the ntsync char device. That "framework" is axiomatic to the patches. And that's the part that made OpenVZ unacceptable to integrate into the kernel — the axiomatic framework being introduced by the patch series. No matter how you split and refactor and document and drip-feed in the OpenVZ patches, they'd still just be a set of patches each proposing one component of an already-externally-designed higher-level feature, that can only exist if all the patches are accepted. (See the relevance?) My understanding of how things should go in Linux kernel development, is that when submitting a single patch to LKML, 1. the motivating user-story should first be explained, and 2. the patch that follows should then be the most idiomatic way to solve that single problem within the existing architecture the Linux kernel. This will then 3. start a discussion, where any "specialization" of the design from its normative "obvious" form may be proposed, and justifications for this argued. When submitting a patch series, each patch must go through this process independently. If there is any common framework to the patches, it can only be because the "obvious way" (by the consensus of all participants) to solve patch 2's user-story, with patch 1 already in place in a dev branch, is to refactor the logic of patch 1 and patch 2 together. In other words, it should be up to the core kernel maintainers — not the contributor — to propose an in-kernel architecture like "this set of operations introduced over the last few development patch series, would make a lot of sense to expose as messages passed into a single character device." And such an architecture should be proposed because it's what's best for the kernel, rather than because having it that way would make the contributor's life easier in implementing their library. (The only exception to this being if the aim is to support direct interoperation with some existing portable software, by exposing a syscall or device or filesystem that other platforms already have, and that existing software already depends upon on these other platforms. Which doesn't apply here — NT itself doesn't have an ntsync device!) I would expect, if this process were followed for NT's sync primitives, that: • some of the resulting changes would introduce ioctls; • some of the resulting changes would add flags for the io_uring or epoll syscall families; • and some of the resulting changes — for the legacy NT sync-primitive objects specifically — would probably become their own syscalls (probably in such a way that they only exist with a certain kernel module loaded, so that the attack surface they represent can be disabled in hardened systems.) These changes would likely all solve the problems as stated (i.e. making it possible to execute NT software that uses sync-object X, with only a userland translation library between the kernel and the software, no RPC daemon needed) ... but these changes likely wouldn't solve the problems in the most efficient way for WINE. Not if there's any other uses — in greenfield Linux software, in virtualization of other OSes, in accelerated kernel-mode hypervision, etc — for exposing such primitives. And also, obviously, these primitives would all end up fairly different from one-another, and from what NT expects — and so WINE would still need a fairly complex translation layer to target these syscalls. Just like it does today. Just, one without inefficiant and apparently-untenable userland arbitration.
wtallis 3y ago
Wine needs a high-performance implementation of sync primitives that even Microsoft has deprecated and discouraged, because there are real programs out there using those Windows APIs that Linux users want to run. Most of what Wine wants added to the kernel should not have a scope of use beyond Wine, which is why the Wine devs have put so much effort into trying to solve these problems in userspace.
- zeusk 3y ago
  Which sync primitive has Microsoft deprecated and discouraged? WaitForMultipleObjects and Nt(Set|Reset|Pulse)Event are used throughout Windows source code
  - dblohm7 3y ago
    It's still used, but PulseEvent has been discouraged for a long time. [1] https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-pulseevent#remarks https://learn.microsoft.com/en-us/windows/win32/api/winbase/... [2] https://devblogs.microsoft.com/oldnewthing/20050105-00/?p=36803 https://devblogs.microsoft.com/oldnewthing/20050105-00/?p=36...
    - sumtechguy 3y ago
      I had a wince proj that someone had used that thing and did not understand the 'pulse' part of that. Where it could go off but if nothing was waiting it would not signal. So they were really confused why sometimes their home grown event queue would be delayed until another event went off and it would drain their queue.
zaphirplane 3y ago
I though Cgroup originated from google employed engineers It’s a shame openVz pioneered container, yet docker ate their lunch
- pjmlp 3y ago
  On GNU/Linux, I was already using containers in HP-UX via Vaults in 1999.
- dusted 3y ago
  I used OpenVZ for a long time before Docker came along, it was certainly nicer than what I did previously (run everything right on the host) but, of the two, I'd pick Docker every time. Dockerfiles, images, layers, thinking of containers as "application instances", the networking, the cli, the speed, there's nothign I miss from OpenVZ, even if it was good for what it was, the way Docker improved on it made a very big difference, in a positive way.
- antod 3y ago
  Docker didn't bring container running features or kernel features for running containers (they just used LXC at first didn't they?), Docker brought an image layering and distribution set of features for single process containers. OpenVZ (like LXC) was more of an OS level VM alternative type of container. Docker was more of a developer oriented distribution/isolation system than OpenVZ/LXC/etc which were more of a sysadmin oriented hosting system. In the very early days Docker didn't make much sense to me until I realised that.
jonhohle 3y ago
Perhaps the operations could have all been added to `ioctl`.

by derefr 3y ago

rrdharan 3y ago

cgroups was built by Google https://en.wikipedia.org/wiki/Cgroups https://en.wikipedia.org/wiki/Cgroups

arka2147483647 3y ago

I imagine the point is not to implement some species of sync primitives currently not supported by Linux, but to implement exactly what windows does, so one does not need to emulate them in userspace.

derefr 3y ago
The point for the contributor is that, yes. The point that would convince the Linux kernel maintainers to actually accept the patchset, though, would likely be the introduction of generally-useful "species of sync primitives." In such a way that those primitives can be used to solve the "hard part" of WINE's virtualization of NT's versions of those primitives; but which wouldn't be constrained to guarantee an efficient zero-impedance call path between those implementations.

the8472 3y ago

The slides[0] (page 17) mention alternative solutions. One of them is faster userspace switching. That sounds a bit like something google has proposed before with switch_to/FUTEX_SWAP[1] [0] https://lpc.events/event/17/contributions/1517/attachments/1257/2549/lpc2023.pdf https://lpc.events/event/17/contributions/1517/attachments/1... [1] https://lwn.net/Articles/824409/ https://lwn.net/Articles/824409/

pcwalton 3y ago
I really want the switchto patch for high performance userspace threading. It'd potentially allow 1:1 threading to better compete with M:N, which would be very nice for systems languages like C++ and Rust.

cwillu 3y ago

This _is_ broken up into individual patches, with documentation and intent discussed (briefly) in each patch. It's not a big blob of undocumented and opaque container logic being dumped in one giant patch.

derefr 3y ago
Sure, but all of these patches still have a common implicit "framework" they're introducing ABI surface to — the ntsync char device. That "framework" is axiomatic to the patches. And that's the part that made OpenVZ unacceptable to integrate into the kernel — the axiomatic framework being introduced by the patch series. No matter how you split and refactor and document and drip-feed in the OpenVZ patches, they'd still just be a set of patches each proposing one component of an already-externally-designed higher-level feature, that can only exist if all the patches are accepted. (See the relevance?) My understanding of how things should go in Linux kernel development, is that when submitting a single patch to LKML, 1. the motivating user-story should first be explained, and 2. the patch that follows should then be the most idiomatic way to solve that single problem within the existing architecture the Linux kernel. This will then 3. start a discussion, where any "specialization" of the design from its normative "obvious" form may be proposed, and justifications for this argued. When submitting a patch series, each patch must go through this process independently. If there is any common framework to the patches, it can only be because the "obvious way" (by the consensus of all participants) to solve patch 2's user-story, with patch 1 already in place in a dev branch, is to refactor the logic of patch 1 and patch 2 together. In other words, it should be up to the core kernel maintainers — not the contributor — to propose an in-kernel architecture like "this set of operations introduced over the last few development patch series, would make a lot of sense to expose as messages passed into a single character device." And such an architecture should be proposed because it's what's best for the kernel, rather than because having it that way would make the contributor's life easier in implementing their library. (The only exception to this being if the aim is to support direct interoperation with some existing portable software, by exposing a syscall or device or filesystem that other platforms already have, and that existing software already depends upon on these other platforms. Which doesn't apply here — NT itself doesn't have an ntsync device!) I would expect, if this process were followed for NT's sync primitives, that: • some of the resulting changes would introduce ioctls; • some of the resulting changes would add flags for the io_uring or epoll syscall families; • and some of the resulting changes — for the legacy NT sync-primitive objects specifically — would probably become their own syscalls (probably in such a way that they only exist with a certain kernel module loaded, so that the attack surface they represent can be disabled in hardened systems.) These changes would likely all solve the problems as stated (i.e. making it possible to execute NT software that uses sync-object X, with only a userland translation library between the kernel and the software, no RPC daemon needed) ... but these changes likely wouldn't solve the problems in the most efficient way for WINE. Not if there's any other uses — in greenfield Linux software, in virtualization of other OSes, in accelerated kernel-mode hypervision, etc — for exposing such primitives. And also, obviously, these primitives would all end up fairly different from one-another, and from what NT expects — and so WINE would still need a fairly complex translation layer to target these syscalls. Just like it does today. Just, one without inefficiant and apparently-untenable userland arbitration.

wtallis 3y ago

Wine needs a high-performance implementation of sync primitives that even Microsoft has deprecated and discouraged, because there are real programs out there using those Windows APIs that Linux users want to run. Most of what Wine wants added to the kernel should not have a scope of use beyond Wine, which is why the Wine devs have put so much effort into trying to solve these problems in userspace.

zeusk 3y ago
Which sync primitive has Microsoft deprecated and discouraged? WaitForMultipleObjects and Nt(Set|Reset|Pulse)Event are used throughout Windows source code
- dblohm7 3y ago
  It's still used, but PulseEvent has been discouraged for a long time. [1] https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-pulseevent#remarks https://learn.microsoft.com/en-us/windows/win32/api/winbase/... [2] https://devblogs.microsoft.com/oldnewthing/20050105-00/?p=36803 https://devblogs.microsoft.com/oldnewthing/20050105-00/?p=36...
  - sumtechguy 3y ago
    I had a wince proj that someone had used that thing and did not understand the 'pulse' part of that. Where it could go off but if nothing was waiting it would not signal. So they were really confused why sometimes their home grown event queue would be delayed until another event went off and it would drain their queue.

zaphirplane 3y ago

I though Cgroup originated from google employed engineers It’s a shame openVz pioneered container, yet docker ate their lunch

pjmlp 3y ago
On GNU/Linux, I was already using containers in HP-UX via Vaults in 1999.
dusted 3y ago
I used OpenVZ for a long time before Docker came along, it was certainly nicer than what I did previously (run everything right on the host) but, of the two, I'd pick Docker every time. Dockerfiles, images, layers, thinking of containers as "application instances", the networking, the cli, the speed, there's nothign I miss from OpenVZ, even if it was good for what it was, the way Docker improved on it made a very big difference, in a positive way.
antod 3y ago
Docker didn't bring container running features or kernel features for running containers (they just used LXC at first didn't they?), Docker brought an image layering and distribution set of features for single process containers. OpenVZ (like LXC) was more of an OS level VM alternative type of container. Docker was more of a developer oriented distribution/isolation system than OpenVZ/LXC/etc which were more of a sysadmin oriented hosting system. In the very early days Docker didn't make much sense to me until I realised that.

jonhohle 3y ago

Perhaps the operations could have all been added to `ioctl`.