3 ms·
The basic problem with X11 is it wraps a whole bunch of things into one software package. So in theory you have to know about fonts, OpenGL, graphics drivers &
by roenxi 7y ago
The basic problem with X11 is it wraps a whole bunch of things into one software package. So in theory you have to know about fonts, OpenGL, graphics drivers & memory management, the window model and networking simultaneously to configure and run the thing.
Wayland is a protocol for a well designed future where all that functionality is broken up into separate pieces of software. Wayland handles the window model and most of the other parts already live in their own software projects. The problem is that Wayland makes some changes to the basic window model on the way through that are a big deal and so far there is no great software written that uses it.
Adoption is very slow because the best case scenario is there a Wayland compositor that has feature parity with an X11 window manage and a bunch of weird problems cropping up because screen recording is a to-be-solved problem.
My position from the armchair is that Wayland really needed a complimentary official protocol for how to request another application's graphical output. Not coordinating that means that anything that supports Wayland is going to be practically inferior to an X11 solution. It'll get fixed sooner or later I expect, it isn't a technically hard problem.
- shmerl 7y agoI think this is intended for screen capturing with Wayland: * https://wiki.archlinux.org/index.php/PipeWire https://wiki.archlinux.org/index.php/PipeWire * https://pipewire.org https://pipewire.org
- roenxi 7y agoQuoth the wiki 'PipeWire is a rather new multimedia framework by GNOME'. There will be people who don't want to use GNOME frameworks and don't want to use the GNOME window manager. Wayland is enforcing a policy ('don't access other devices graphics output') where it should be providing capability ('an applications graphics output is private by default but that can be configured'). However it looks like the design of Wayland purposefully omitted that sort of configuration from the protocol. So instead of having a security model that is sufficiently complicated to allow screen recording, the security model is 'No'. To get a framework independent solution all the compositors would need to implement an identical API for accessing graphics output. That needs to be coordinated in the same fashion as the Wayland spec. As the spec stands, it is just asking for every framework, compositor or application to go implement its own different solution. The Wayland designers may even have tried to avoid explicitly creating a security model, but in practice it comes with an implicit security model that is inflexible, unconfigurable and difficult to coordinate around.
- shmerl 7y agoFrom what I gather, Pipewire isn't part of Gnome, i.e. it's generic and can be used with any DE. For example, all compositors are using libinput, which replaces legacy ways of X handling input. It's not mandated by the protocol either. So I don't see why they can't use Pipewire for audio and video needs. If anyone wants to make a better alternative to standardize on - they are free to propose it.
- roenxi 7y agoBut libinput is a freedesktop.org project, ie, supported by the same people who are working on Wayland. It was specifically built to support Wayland. Pipewire looks like some audio-visual framework for GNOME and GTK apps and while it might be fantastic it is both new and not very comprehensibly advertised. And it will probably be very complicated and involve some rather interesting design issues compared to building something in when the Wayland protocol was created. We're coming out of a world where the standard used to be "run graphics drivers in userspace!" so any weird compromise is workable (including an audio framework deciding on graphics security policy). But ultimately the idea that a GNOME-sponsored effort is going to be what sorts out the security model for desktop apps is weird. I can't find anything to suggest that Pipewire is intended to be the definitive way to do screen recording, it looks to me like that might be a happy side effect. It wouldn't be surprising if it turns into two or three competing ideas.
- boudin 7y agoYou're mistaken with pipewire, it's a daemon intending to replace pulseaudio at some point, it's not a gnome component. Screen recording is also available in wlroot based window managers. The wayland environment is maturing quite fast now.
- shmerl 7y agoSome details here: https://tanukaskinen.wordpress.com/2018/11/12/pipewire-hackfest-2018-in-edinburgh/ https://tanukaskinen.wordpress.com/2018/11/12/pipewire-hackf... So far video applications on Linux have accessed the hardware directly, meaning that two applications can’t access e.g. a webcam simultaneously. Solving that problem, and screen sharing with Wayland, were to my understanding the main motivations for creating a video daemon. So screen sharing on Wayland (that covers capture use case) was one of the main motivations behind it. If both Gnome and KDE (the most widely used DEs) are behind it, I'd say it's already a major consensus on where things are going. Also, it looks like even wlroots has some plans to support it: https://github.com/emersion/xdg-desktop-portal-wlr https://github.com/emersion/xdg-desktop-portal-wlr And if anyone wants to propose a better alternative, like you said, they can do it.
- simion314 7y agoThe issue is I need the powerfull features, like I have a script that OCR a region of the screen when I press a button (the coordinates depend of the app that runs) so I do not want a GUI provided by the DE to let me capture the screen. Anyway the issue with Wayland is that it is not ready, the important screen recording feature is yet not here,I seen also some complaints from gamers and their issues are a won't fix, probably when Wayland will be actually ready we will need something to replace it or a Wayland 2 that learned from first attempt mistakes.
- aloer 7y agoCould you tell us more about the use case here? What do you use the OCR for?
- simion314 7y agoI suffer of macular degeneration https://en.wikipedia.org/wiki/Macular_degeneration https://en.wikipedia.org/wiki/Macular_degeneration , so I am having issues reading texts , I am using TTS and Zoom features for most applications but the big issue I have is with video games. For open source game engines I can patch them to have the text pushed to my TTS scripts but for closed source game engine OCR or zoom is the only solution but it has a lot of issues. 1 my zoom follows the mouse but this is broken is nome games where the trying to move the zoom area to the edge where the text is would move the camera in game , pausing the game is a solution but some games would put a Pause screen on top of te text I want to read. The OCR has issues when games use fancy fonts, old games use some pixelated fonts that are impossible to OCR, other games use more artistic fonts that are also hard to OCR, other games use text over semi transp[arent background and it makes hard to separate the text from the background. For DirectX I am thinking if I could maybe intercept the draw text calls, I am considering making a Stack OVerflow post but I am a bit anxious that I am not sure what the correct title and tags should be and I could get the off topic and the question closed.
- pedrocr 7y ago>like I have a script that OCR a region of the screen when I press a button (the coordinates depend of the app that runs) This is easy to do. Something like grim to take the screenshot of whatever region you want: https://github.com/emersion/grim https://github.com/emersion/grim And calculate the coordinates you need by querying the window manager. In sway (i3 compatible tiling WM) that would be: $ swaymsg -t get_tree To get a JSON of all the windows currently open and do whatever math is needed. You can also do a hand selection using slurp: https://github.com/emersion/slurp https://github.com/emersion/slurp It works together with grim like this: $ grim -g "$(slurp)" screenshot.png Having recently moved from X to Wayland there are still some things missing sometimes but they're not many. If you want a DE at least GNOME and I think KDE as well are making the switch reasonably seamless. And there are some features that Wayland solves much more thoroughly (e.g., tear free output, fractional scaling). The network transparency part, that is often brought up but rarely used in X is also being delivered: https://lists.freedesktop.org/archives/wayland-devel/2019-August/040819.html https://lists.freedesktop.org/archives/wayland-devel/2019-Au...