9 ms·

Zoom has a popular feature which allows for screen sharing a single application window instead of the entire desktop. I assume this is how its discovering runni

Zoom has a popular feature which allows for screen sharing a single application window instead of the entire desktop. I assume this is how its discovering running applications with GUI windows open in a cross-platform manner. Perhaps there is a better API for this? Its not a common use-case that an app would need to know all of the other open windows, but it seems like a perfectly valid use case, and frankly handy for sharing a PPT without fear of an embarrassing email showing up in during a meeting.

sails 5y ago
Does Teams do this in the same way? They offer the same feature
- sniperjzp 5y ago
  Not sure about Teams, but Google Meet allow you to share a specific browser tab, I don't know how to implement it without reading all the opened tabs.
  - meibo 5y ago
    There is a JavaScript API that abstracts this, so Google Meet/whatever asked can only read the "final" video stream of what you selected and not everything that's going on, which stays in your local browser.
  - Flame 5y ago
    Teams lets you share an application window or your display(s).
    - saikan 5y ago
      and "Window sharing isn't available for Linux users." https://support.microsoft.com/en-us/office/share-content-in-a-meeting-in-teams-fcc2bf59-aecd-4481-8f99-ce55dd836ce8 https://support.microsoft.com/en-us/office/share-content-in-...
cranekam 5y ago
There’s nothing in /proc/PID/cmdline etc that would definitely confirm an app has a GUI. The two concepts are at very different layers of the stack — to the OS the GUI is just another app on top of the kernel. Mayyyyyybe Zoom could be doing some crude pattern matching on process name or environment variables of a given process (though it’d only be able to read data of processes running as the same user) but it’s a long shot. Personally I’d guess it is either some other library Zoom uses or some kind of debug info capturing system. But I don’t know work at Zoom so who knows.
- woodruffw 5y ago
  This is an unlikely stretch, but it looks like they're also attempting to read `/proc/PID/exe` -- they could be doing something (extremely) cursed like attempting to parse the program linkages and filtering by programs that link to `libX11`. But that's me straining to explain this in the context of the GP's hypothesis.
  - thrower123 5y ago
    Never attribute to malice what is adequately explained by laziness. I can easily believe somebody just wrote a chunk of naive code that grabbed all the running processes, and it worked, and they moved on.
    - propogandist 5y ago
      or it’s spyware, maybe not as bad as a keylogger, but it can be mining your active usage behavior
      - reilly3000 5y ago
        This could be verified by inspecting their analytics requests. If I have time I may take a peek at those later.
        propogandist 5y ago
woodruffw 5y ago
Based on OP's snippet they're probably doing this on Linux. Assuming the presence of X11 and an EWMH-compatible window manager, they could query X11 directly and ask the root window for its `_NET_CLIENT_LIST` attribute. That, in turn, would list every "semantic" (i.e., non-decorative) window currently open on the current display. But that doesn't work for non-X11 or if the WM is non-EWMH compliant. Presumably Wayland has a similar API, and non-EWMH is probably a minuscule group that considers this a desirable feature.
- sp1rit 5y ago
  One of the selling points of Wayland is, that they have no such API. An Application is supposed to be unable to access windows (and their corresponding keyboard/mouse input). One of the downsides this has is the described issue of "screensharing beeing impossible on wayland". This is solved by the XDG Desktop Portal, which provides a unified dbus interface across the different compositor implementations for requesting a pipewire file descriptor (which can be used with gstreamer to get a live video stream of the deskop, in a way far superior to x11 framegrab). However the implementation differs for each compositor, GNOME for example asks you if you what to share the whole screen or just a specific application but wlroots (swaywm, wayfire, etc.) AFAIK automatically accepts and shares the whole screen. I don't know what KDE Plasma does.
  - dathinab 5y ago
    I initially miss read you post, you formulations can be a bit misleading. E.g. "Wayland is, that they have no such API.", it has screen sharing APIs but they are different and require you do go through other programs like XDG Desktop Portal and Pipewire to allow the user to control such access. Similar this also means Wayland supports screen sharing just in different ways. Anyway the important parts are: - the implementation might differ, but the API doesn't (or at least not by a relevant degree) - wayland requires you to go through specific APIs for screen sharing, scanning processes has little to do with screen sharing on either wayland or X.
  - account42 5y ago
    > which can be used with gstreamer to get a live video stream of the deskop, in a way far superior to x11 framegrab I don't think I'd agree that having to go through dbus and pipewire just to get the contents of the screen or a window is far superior to requesting that data from the display server. It certainly adds a lot of complexity with subpar documentation spread across multiple projects. Does this provide you a compressed stream or can you get the raw pixels that are displayed?
twic 5y ago
I run Zoom from flatpak, which runs it in a container, and sandboxes it to some extent [1] This probably explains why, when i try to screenshare a single application window, not every application shows up! I can share my browser, file manager, and various other things, but not windows for games started by Steam. [1] I followed these instructions https://www.mayrhofer.eu.org/post/zoom-flatpak-sandboxing/ https://www.mayrhofer.eu.org/post/zoom-flatpak-sandboxing/
- heavyset_go 5y ago
  Are you using Wayland? That might be an artifact of Steam and games running via XWayland.
  - twic 5y ago
    As far as i know, no, it's all still pure X11. I would like to try Wayland, but i'm using Cinnamon as a desktop, and i'm not sure they work together.
- ineedasername 5y ago
  Games always seem impacted in odd ways by remote desktop protocols. One game might run fine, while another is just a black screen, or an uncontrollable mouse. It makes me wonder how Things like Steam streaming and Paperspace get around the issue.
kelnos 5y ago
> I assume this is how its discovering running applications with GUI windows open in a cross-platform manner. That's definitely not a cross-platform way of doing it (and I doubt there is one, even). On Linux you'd use libX11 and just enumerate all windows (using XQueryTree()). Walking the contents of /proc is not only unnecessary, but is more difficult to do, as looking at executable names won't tell you if a program has a GUI, or if it has any open windows. It won't give you window titles, or how many windows are open, or how to grab their contents. Pretty sure Zoom is snooping on us and is gathering telemetry.
- mwcampbell 5y ago
  > Pretty sure Zoom is snooping on us and is gathering telemetry. Don't forget Hanlon's razor, as someone else in the thread pointed out.
  - dathinab 5y ago
    But I can't come up with a "stupidity" which would explain this behaviour...
- user5994461 5y ago
  Windows has a function EnumWindows() to do just that. If you say that there's an similar API on Linux for X11, then that's the same methodology across platforms.
  - dathinab 5y ago
    I'm not sure there is such a API on wayland, BUT there are standardized protocols/APIs for screen sharing including screen sharing of just a window. And you won't get far without using them so also no reason to scan processes.
yakubin 5y ago
> handy for sharing a PPT without fear of an embarrassing email showing up in during a meeting. When you share a single window in Zoom, notifications are still visible to others in the meeting when they overlap with the window you're sharing. That's the case for e.g. Slack notifications.
- TameAntelope 5y ago
  I literally quit all messaging apps when presenting to people for any period of time more than ~10s.
  - dylan604 5y ago
    You could also disable notifications if your OS has that kind of feature. Mine does, so I took advantage of that after the first incident of being interrupted. It's a nice feature.
    - cygned 5y ago
      Let me introduce you to Microsoft Teams which comes with custom notifications ignoring do-not-disturb settings.
- aetherspawn 5y ago
  They are grayed out for everyone else, so they can’t actually read the contents. Caveat being if you move the window around really fast sometimes it’s possible to catch a glimpse.
  - yakubin 5y ago
    Maybe that's changed, but a couple months ago we tested it on vanilla Ubuntu in the team at $dayjob and we could see everything. Nothing was grayed out.
    - 123jay7 5y ago
      $dayJob
dathinab 5y ago
> I assume this is how its discovering running applications with GUI windows open in a cross-platform manner. The screen sharing functionality is handled by a mix of protocols of the windows manger and service providers announced over dbus. Even if you want to map GUI windows to processes you would do so by getting a list of windows from the window manager and getting the pid property of the windows, but if you have a list of windows you don't need to scan processes anymore... There might (I'm not sure) be valid use-cases for this behaviour but I'm pretty confident screen sharing of specific windows isn't part of it.
ineedasername 5y ago
Reminds me a bit of how platforms like Citrix will publish single applications. Considering you can give control to another user with Zoom I suppose it's a lot like that, although I know the underlying tech is very different.
birksherty 5y ago
Google meet running in a browser can share any application specific windows. Not sure if it also does the same thing though.
barbs 5y ago
I'm pretty sure Jitsi allows for screen-sharing a particular application so I'm curious to know how that works. Might check the source later.

sails 5y ago

Does Teams do this in the same way? They offer the same feature

sniperjzp 5y ago
Not sure about Teams, but Google Meet allow you to share a specific browser tab, I don't know how to implement it without reading all the opened tabs.
- meibo 5y ago
  There is a JavaScript API that abstracts this, so Google Meet/whatever asked can only read the "final" video stream of what you selected and not everything that's going on, which stays in your local browser.
- Flame 5y ago
  Teams lets you share an application window or your display(s).
  - saikan 5y ago
    and "Window sharing isn't available for Linux users." https://support.microsoft.com/en-us/office/share-content-in-a-meeting-in-teams-fcc2bf59-aecd-4481-8f99-ce55dd836ce8 https://support.microsoft.com/en-us/office/share-content-in-...

cranekam 5y ago

There’s nothing in /proc/PID/cmdline etc that would definitely confirm an app has a GUI. The two concepts are at very different layers of the stack — to the OS the GUI is just another app on top of the kernel. Mayyyyyybe Zoom could be doing some crude pattern matching on process name or environment variables of a given process (though it’d only be able to read data of processes running as the same user) but it’s a long shot. Personally I’d guess it is either some other library Zoom uses or some kind of debug info capturing system. But I don’t know work at Zoom so who knows.

woodruffw 5y ago
This is an unlikely stretch, but it looks like they're also attempting to read `/proc/PID/exe` -- they could be doing something (extremely) cursed like attempting to parse the program linkages and filtering by programs that link to `libX11`. But that's me straining to explain this in the context of the GP's hypothesis.
- thrower123 5y ago
  Never attribute to malice what is adequately explained by laziness. I can easily believe somebody just wrote a chunk of naive code that grabbed all the running processes, and it worked, and they moved on.
  - propogandist 5y ago
    or it’s spyware, maybe not as bad as a keylogger, but it can be mining your active usage behavior
    - reilly3000 5y ago
      This could be verified by inspecting their analytics requests. If I have time I may take a peek at those later.
      - propogandist 5y ago

woodruffw 5y ago

Based on OP's snippet they're probably doing this on Linux. Assuming the presence of X11 and an EWMH-compatible window manager, they could query X11 directly and ask the root window for its `_NET_CLIENT_LIST` attribute. That, in turn, would list every "semantic" (i.e., non-decorative) window currently open on the current display. But that doesn't work for non-X11 or if the WM is non-EWMH compliant. Presumably Wayland has a similar API, and non-EWMH is probably a minuscule group that considers this a desirable feature.

sp1rit 5y ago
One of the selling points of Wayland is, that they have no such API. An Application is supposed to be unable to access windows (and their corresponding keyboard/mouse input). One of the downsides this has is the described issue of "screensharing beeing impossible on wayland". This is solved by the XDG Desktop Portal, which provides a unified dbus interface across the different compositor implementations for requesting a pipewire file descriptor (which can be used with gstreamer to get a live video stream of the deskop, in a way far superior to x11 framegrab). However the implementation differs for each compositor, GNOME for example asks you if you what to share the whole screen or just a specific application but wlroots (swaywm, wayfire, etc.) AFAIK automatically accepts and shares the whole screen. I don't know what KDE Plasma does.
- dathinab 5y ago
  I initially miss read you post, you formulations can be a bit misleading. E.g. "Wayland is, that they have no such API.", it has screen sharing APIs but they are different and require you do go through other programs like XDG Desktop Portal and Pipewire to allow the user to control such access. Similar this also means Wayland supports screen sharing just in different ways. Anyway the important parts are: - the implementation might differ, but the API doesn't (or at least not by a relevant degree) - wayland requires you to go through specific APIs for screen sharing, scanning processes has little to do with screen sharing on either wayland or X.
- account42 5y ago
  > which can be used with gstreamer to get a live video stream of the deskop, in a way far superior to x11 framegrab I don't think I'd agree that having to go through dbus and pipewire just to get the contents of the screen or a window is far superior to requesting that data from the display server. It certainly adds a lot of complexity with subpar documentation spread across multiple projects. Does this provide you a compressed stream or can you get the raw pixels that are displayed?

twic 5y ago

I run Zoom from flatpak, which runs it in a container, and sandboxes it to some extent [1] This probably explains why, when i try to screenshare a single application window, not every application shows up! I can share my browser, file manager, and various other things, but not windows for games started by Steam. [1] I followed these instructions https://www.mayrhofer.eu.org/post/zoom-flatpak-sandboxing/ https://www.mayrhofer.eu.org/post/zoom-flatpak-sandboxing/

heavyset_go 5y ago
Are you using Wayland? That might be an artifact of Steam and games running via XWayland.
- twic 5y ago
  As far as i know, no, it's all still pure X11. I would like to try Wayland, but i'm using Cinnamon as a desktop, and i'm not sure they work together.
ineedasername 5y ago
Games always seem impacted in odd ways by remote desktop protocols. One game might run fine, while another is just a black screen, or an uncontrollable mouse. It makes me wonder how Things like Steam streaming and Paperspace get around the issue.

kelnos 5y ago

> I assume this is how its discovering running applications with GUI windows open in a cross-platform manner. That's definitely not a cross-platform way of doing it (and I doubt there is one, even). On Linux you'd use libX11 and just enumerate all windows (using XQueryTree()). Walking the contents of /proc is not only unnecessary, but is more difficult to do, as looking at executable names won't tell you if a program has a GUI, or if it has any open windows. It won't give you window titles, or how many windows are open, or how to grab their contents. Pretty sure Zoom is snooping on us and is gathering telemetry.

mwcampbell 5y ago
> Pretty sure Zoom is snooping on us and is gathering telemetry. Don't forget Hanlon's razor, as someone else in the thread pointed out.
- dathinab 5y ago
  But I can't come up with a "stupidity" which would explain this behaviour...
user5994461 5y ago
Windows has a function EnumWindows() to do just that. If you say that there's an similar API on Linux for X11, then that's the same methodology across platforms.
- dathinab 5y ago
  I'm not sure there is such a API on wayland, BUT there are standardized protocols/APIs for screen sharing including screen sharing of just a window. And you won't get far without using them so also no reason to scan processes.

yakubin 5y ago

> handy for sharing a PPT without fear of an embarrassing email showing up in during a meeting. When you share a single window in Zoom, notifications are still visible to others in the meeting when they overlap with the window you're sharing. That's the case for e.g. Slack notifications.

TameAntelope 5y ago
I literally quit all messaging apps when presenting to people for any period of time more than ~10s.
- dylan604 5y ago
  You could also disable notifications if your OS has that kind of feature. Mine does, so I took advantage of that after the first incident of being interrupted. It's a nice feature.
  - cygned 5y ago
    Let me introduce you to Microsoft Teams which comes with custom notifications ignoring do-not-disturb settings.
aetherspawn 5y ago
They are grayed out for everyone else, so they can’t actually read the contents. Caveat being if you move the window around really fast sometimes it’s possible to catch a glimpse.
- yakubin 5y ago
  Maybe that's changed, but a couple months ago we tested it on vanilla Ubuntu in the team at $dayjob and we could see everything. Nothing was grayed out.
  - 123jay7 5y ago
    $dayJob

dathinab 5y ago

> I assume this is how its discovering running applications with GUI windows open in a cross-platform manner. The screen sharing functionality is handled by a mix of protocols of the windows manger and service providers announced over dbus. Even if you want to map GUI windows to processes you would do so by getting a list of windows from the window manager and getting the pid property of the windows, but if you have a list of windows you don't need to scan processes anymore... There might (I'm not sure) be valid use-cases for this behaviour but I'm pretty confident screen sharing of specific windows isn't part of it.

ineedasername 5y ago

Reminds me a bit of how platforms like Citrix will publish single applications. Considering you can give control to another user with Zoom I suppose it's a lot like that, although I know the underlying tech is very different.

birksherty 5y ago

Google meet running in a browser can share any application specific windows. Not sure if it also does the same thing though.

barbs 5y ago

I'm pretty sure Jitsi allows for screen-sharing a particular application so I'm curious to know how that works. Might check the source later.