3 ms·

> In fact in the first few releases of Fedora Workstation where we shipped PipeWire we solely enabled it as a tool to handle screen sharing for Wayland and Flat

by HMH 5y ago

> In fact in the first few releases of Fedora Workstation where we shipped PipeWire we solely enabled it as a tool to handle screen sharing for Wayland and Flatpaks. PipeWire is truly a godsend for screen recording on Wayland, not very long ago, if at all, every desktop environment / compositor had their own home brewed protocol and developing anything related to screen recording on Wayland was a massive pain [1]. I still can not fathom how Wayland could even exist this long without any decent way to do screen recording and I also do not get why there is no official protocol or at least extension for this! But thanks to PipeWire some first projects finally support screen capturing on Wayland. Both Firefox (via webrtc) [2] and OBS [3] directly interface with PipeWire. For my personal project Weylus [4] I opted for the gstreamer plugin as at the time the documentation for PipeWire itself was very lacking and I could not figure out how to get it to work. Fortunately gstreamer is better documented and it turned out to be rather easy to use. But it shows that all this is still in an early stage as I hit quite some bugs, probably the most egregious (maybe even somewhat funny) one is KDE's kwin_wayland crashing if you hover the cursor over any window close button while screen recording [5], sadly it doesn't seem to be fixed yet. For some more bugs I've hit, see [6]. Something I am a little worried about is that flatpak with their xdg-desktop-portal [7] is now responsible for the de facto standard how screen recording is negotiated on Wayland. I fear that this means if flatpak does not need a feature, even if it makes sense in another context, it probably won't ever be implemented. For example something I require for my project are proper window names and their position as well as size, but apparently this is nothing essential for flatpak and the issues I have opened have largely been ignored [8]. This is by no means meant as an accusation, just an observations that things are not all good. [1]: https://github.com/H-M-H/Weylus/issues/3#issuecomment-652670480 https://github.com/H-M-H/Weylus/issues/3#issuecomment-652670... [2]: https://webrtc.googlesource.com/src/+/refs/heads/main/modules/desktop_capture/linux/base_capturer_pipewire.cc https://webrtc.googlesource.com/src/+/refs/heads/main/module... [3]: https://github.com/obsproject/obs-studio/blob/master/plugins/linux-capture/pipewire.c https://github.com/obsproject/obs-studio/blob/master/plugins... [4]: https://github.com/H-M-H/Weylus https://github.com/H-M-H/Weylus [5]: https://bugs.kde.org/show_bug.cgi?id=435042 https://bugs.kde.org/show_bug.cgi?id=435042 [6]: https://github.com/H-M-H/Weylus/issues/3#issuecomment-808940593 https://github.com/H-M-H/Weylus/issues/3#issuecomment-808940... [7]: https://github.com/flatpak/xdg-desktop-portal https://github.com/flatpak/xdg-desktop-portal [8]: https://github.com/flatpak/xdg-desktop-portal/issues/created_by/H-M-H https://github.com/flatpak/xdg-desktop-portal/issues/created...

speeddemon 5y ago
>I still can not fathom how Wayland could even exist this long without any decent way to do screen recording and I also do not get why there is no official protocol or at least extension for this! It likely won't ever get an official Wayland extension, for a number of reasons. Screen capturing is inherently insecure and so it shouldn't really be happening over the Wayland socket. It also doesn't need to happen over the Wayland socket or be related to the windowing protocol at all, attempting to do it that way is just re-accumulating technical debt left over from X11. >I fear that this means if flatpak does not need a feature, even if it makes sense in another context, it probably won't ever be implemented. I wouldn't think of it so much as just flatpak, but the API needs to be usable from within a sandbox, and it needs to be done in such a way that it can be reasonably presented to the user. >For example something I require for my project are proper window names and their position This is a really bad idea, don't do this. This is guaranteed to mess with the window manager and run into any number of syncing issues with window sizes being reported incorrectly. It's also a misuse of the screencasting API. I checked the issues you posted and I don't think you're going to get much farther trying to hack around the API. If you want to do this right now then you'll have to integrate with the specifics of the window manager or shell. There is no way around it, I really doubt you will get much interest in doing it another way.
- HMH 5y ago
  > attempting to do it that way is just re-accumulating technical debt left over from X11 I see, I do not know much about the internal workings of Wayland. The problem I do see though is that screen recording is an essential feature that has been neglected for years due to the lack of a standard. > This is a really bad idea, don't do this. While I agree that getting window positions may be questionable, providing a human readable title for the streams that have been selected seems obvious, yet is missing. > syncing issues with window sizes being reported incorrectly This seems rather theoretical to me, while of course true, I doubt it is going to matter in practice. > It's also a misuse of the screencasting API. Hmm, do you think there is a better solution, that isn't giving up and having each desktop environment do its own thing? > If you want to do this right now then you'll have to integrate with the specifics of the window manager or shell. There is no way around it, I really doubt you will get much interest in doing it another way. This is also my conclusion, alas it's not happening and one of the reasons why I think Wayland is not quite there yet.

by HMH 5y ago

speeddemon 5y ago

>I still can not fathom how Wayland could even exist this long without any decent way to do screen recording and I also do not get why there is no official protocol or at least extension for this! It likely won't ever get an official Wayland extension, for a number of reasons. Screen capturing is inherently insecure and so it shouldn't really be happening over the Wayland socket. It also doesn't need to happen over the Wayland socket or be related to the windowing protocol at all, attempting to do it that way is just re-accumulating technical debt left over from X11. >I fear that this means if flatpak does not need a feature, even if it makes sense in another context, it probably won't ever be implemented. I wouldn't think of it so much as just flatpak, but the API needs to be usable from within a sandbox, and it needs to be done in such a way that it can be reasonably presented to the user. >For example something I require for my project are proper window names and their position This is a really bad idea, don't do this. This is guaranteed to mess with the window manager and run into any number of syncing issues with window sizes being reported incorrectly. It's also a misuse of the screencasting API. I checked the issues you posted and I don't think you're going to get much farther trying to hack around the API. If you want to do this right now then you'll have to integrate with the specifics of the window manager or shell. There is no way around it, I really doubt you will get much interest in doing it another way.

HMH 5y ago
> attempting to do it that way is just re-accumulating technical debt left over from X11 I see, I do not know much about the internal workings of Wayland. The problem I do see though is that screen recording is an essential feature that has been neglected for years due to the lack of a standard. > This is a really bad idea, don't do this. While I agree that getting window positions may be questionable, providing a human readable title for the streams that have been selected seems obvious, yet is missing. > syncing issues with window sizes being reported incorrectly This seems rather theoretical to me, while of course true, I doubt it is going to matter in practice. > It's also a misuse of the screencasting API. Hmm, do you think there is a better solution, that isn't giving up and having each desktop environment do its own thing? > If you want to do this right now then you'll have to integrate with the specifics of the window manager or shell. There is no way around it, I really doubt you will get much interest in doing it another way. This is also my conclusion, alas it's not happening and one of the reasons why I think Wayland is not quite there yet.