3 ms·

Would a tool like Auto hotkey if it were implemented on Linux work with Wayland?

by FloatArtifact 4y ago

mmphosis 4y ago
https://github.com/autokey/autokey/issues/87 https://github.com/autokey/autokey/issues/87
matthewn 4y ago
https://github.com/espanso/espanso/discussions/1178 https://github.com/espanso/espanso/discussions/1178
phil294 4y ago
AHK_X11 is AutoHotkey for Linux and it does not yet support Wayland, but it's definitely possible https://github.com/phil294/AHK_X11/issues/2 https://github.com/phil294/AHK_X11/issues/2 (see also: ydotool)
Arnavion 4y ago
Some parts will work. Some parts won't. AHK lets you run arbitrary code in response to global hotkeys. There is no wayland protocol for applications to register global hotkeys for themselves. Compositors can choose to provide their own hotkey configurations, and what you can do in response to those hotkeys being pressed depends on the compositor. For example in Sway you can register hotkeys to run arbitrary executables. Alternatively, XDP apparently has a portal API for applications to register global hotkeys, but of course it's up to the individual compositors' portal implementations if they implement the API. For example xdpw, used for all wlroots-based compositors like sway, does not. AHK lets you read and set the mouse position. There is currently no wayland protocol, stable or unstable, for programs to read or set the position of a pointer (if the pointer is not over the program's surface). AHK lets you introspect arbitrary windows to read the text of a focused textbox, etc. There is currently no protocol, stable or unstable, for programs to implicitly grab the content of surfaces under the pointer, let alone read text of textboxes. At best you can use the XDP Screencast protocol but that only gives you images (AHK integrates with the standard Windows graphics UI API so it can detect textboxes etc), and even then it might pop up a permissions dialog for the user to approve like you would expect for a screencasting program. AHK lets you insert arbitrary keyboard sequences into the focused window. There is no Wayland protocol for this, but you can just use a new input device through the uinput kernel driver instead of anything Wayland-specific. (ydotool does this.) AHK lets you read and write to the clipboard. There is an (unstable) wayland protocol for clipboard managers that can be used for this.
- phil294 4y ago
  Thanks for the helpful summary. Capturing key presses could also work by talking to the kernel directly. > There is currently no protocol, stable or unstable, for programs to implicitly grab the content of surfaces under the pointer, let alone read text of textboxes. Yes there is, AtSpi, the accessibility protocol. But a normal window-independent `Click, X, Y` would indeed be quite important to have... I'm wondering if there is a way to somehow get active window position, move windows around etc. using root privileges, without relying on a custom compositor extension. Could not find anything.
  - Arnavion 4y ago
    >Yes there is, AtSpi, the accessibility protocol. Right, "protocol" in that sentence meant "Wayland protocol". I'd forgotten about AT-SPI, and it's true that it solves that problem, though it seems that it also includes API for synthesizing input which would require the compositor to implement something. >I'm wondering if there is a way to somehow get active window position, move windows around etc. using root privileges Root doesn't matter, because the compositor isn't running as root either. ie being root won't give you any more privileges over it that just being the same user doesn't. Things like "active window position", "move windows around", and even the entire concept of "windows" are internal to the compositor. Wayland only talks about surfaces; what the compositor does with those surfaces is up to the compositor. It might display them as windows, or as sides of a 3-D rotating cube, or print them through a printer. So you're not going to get that information from anywhere other than the compositor.
- AnIdiotOnTheNet 4y ago
  The solution seems obvious: build something like AHK into the compositor itself, since that's where all this is happening anyway. Do any compositors do that?
- FloatArtifact 4y ago
  Wow! So this really is a blocking issue for accessibility and automation framework.
Fnoord 4y ago
There are a lot of tools available which allow to generate/automate mouse and keyboard events within Wayland. Here's an example (I use another ydotool and another one, on another machine; don't even remember what it is called) [1]. It describes how they require a udev rule and use /dev/uinput Examples I use: 3 finger swipe to left/right switches desktop in Sway, rebinding caps to esc. [1] https://crates.io/crates/tfc https://crates.io/crates/tfc

Would a tool like Auto hotkey if it were implemented on Linux work with Wayland?

mmphosis 4y ago

https://github.com/autokey/autokey/issues/87 https://github.com/autokey/autokey/issues/87

matthewn 4y ago

https://github.com/espanso/espanso/discussions/1178 https://github.com/espanso/espanso/discussions/1178

phil294 4y ago

AHK_X11 is AutoHotkey for Linux and it does not yet support Wayland, but it's definitely possible https://github.com/phil294/AHK_X11/issues/2 https://github.com/phil294/AHK_X11/issues/2 (see also: ydotool)

Arnavion 4y ago

Some parts will work. Some parts won't. AHK lets you run arbitrary code in response to global hotkeys. There is no wayland protocol for applications to register global hotkeys for themselves. Compositors can choose to provide their own hotkey configurations, and what you can do in response to those hotkeys being pressed depends on the compositor. For example in Sway you can register hotkeys to run arbitrary executables. Alternatively, XDP apparently has a portal API for applications to register global hotkeys, but of course it's up to the individual compositors' portal implementations if they implement the API. For example xdpw, used for all wlroots-based compositors like sway, does not. AHK lets you read and set the mouse position. There is currently no wayland protocol, stable or unstable, for programs to read or set the position of a pointer (if the pointer is not over the program's surface). AHK lets you introspect arbitrary windows to read the text of a focused textbox, etc. There is currently no protocol, stable or unstable, for programs to implicitly grab the content of surfaces under the pointer, let alone read text of textboxes. At best you can use the XDP Screencast protocol but that only gives you images (AHK integrates with the standard Windows graphics UI API so it can detect textboxes etc), and even then it might pop up a permissions dialog for the user to approve like you would expect for a screencasting program. AHK lets you insert arbitrary keyboard sequences into the focused window. There is no Wayland protocol for this, but you can just use a new input device through the uinput kernel driver instead of anything Wayland-specific. (ydotool does this.) AHK lets you read and write to the clipboard. There is an (unstable) wayland protocol for clipboard managers that can be used for this.

phil294 4y ago
Thanks for the helpful summary. Capturing key presses could also work by talking to the kernel directly. > There is currently no protocol, stable or unstable, for programs to implicitly grab the content of surfaces under the pointer, let alone read text of textboxes. Yes there is, AtSpi, the accessibility protocol. But a normal window-independent `Click, X, Y` would indeed be quite important to have... I'm wondering if there is a way to somehow get active window position, move windows around etc. using root privileges, without relying on a custom compositor extension. Could not find anything.
- Arnavion 4y ago
  >Yes there is, AtSpi, the accessibility protocol. Right, "protocol" in that sentence meant "Wayland protocol". I'd forgotten about AT-SPI, and it's true that it solves that problem, though it seems that it also includes API for synthesizing input which would require the compositor to implement something. >I'm wondering if there is a way to somehow get active window position, move windows around etc. using root privileges Root doesn't matter, because the compositor isn't running as root either. ie being root won't give you any more privileges over it that just being the same user doesn't. Things like "active window position", "move windows around", and even the entire concept of "windows" are internal to the compositor. Wayland only talks about surfaces; what the compositor does with those surfaces is up to the compositor. It might display them as windows, or as sides of a 3-D rotating cube, or print them through a printer. So you're not going to get that information from anywhere other than the compositor.
AnIdiotOnTheNet 4y ago
The solution seems obvious: build something like AHK into the compositor itself, since that's where all this is happening anyway. Do any compositors do that?
FloatArtifact 4y ago
Wow! So this really is a blocking issue for accessibility and automation framework.

Fnoord 4y ago

There are a lot of tools available which allow to generate/automate mouse and keyboard events within Wayland. Here's an example (I use another ydotool and another one, on another machine; don't even remember what it is called) [1]. It describes how they require a udev rule and use /dev/uinput Examples I use: 3 finger swipe to left/right switches desktop in Sway, rebinding caps to esc. [1] https://crates.io/crates/tfc https://crates.io/crates/tfc