14 ms·
Why Is It So Hard to Detect Keyup Event on Linux?
- akmittal 5y agoAll the applications getting keyup events seems like big security issue. Is this same case with Wayland
- yw3410 5y agoIs what the same as Wayland? Maybe you've misunderstood the article? The process is running as root and reading directly from /proc; the article acknowledges that it's essentially a keylogger.
- chaorace 5y agoI am now curious, though. How does Wayland handle key events? I have to imagine it only lets the active window listen in on key events, right? Maybe there's an API for subsribing to specific key combinations?
- chrisseaton 5y ago> Maybe there's an API for subsribing to specific key combinations? Why would you ever let a non-focused application subscribe to any key combinations?
- pjc50 5y agohttps://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setwindowshookexa https://docs.microsoft.com/en-us/windows/win32/api/winuser/n... : for global hotkeys.
- deleted 5y ago[deleted]
- chrisseaton 5y agoThat doesn't seem like something that an application should be getting involved with. An application should only be able to manage its own windows and key presses to its own window.
- R0b0t1 5y agoProblematic. Means you are relying on the DE for all UI development. With Windows and X11 the DE is reconfigurable to some extent by applications. There is also more to this, showing the Wayland security model is incoherent. If debugging is enabled on your system I can simply debug your root terminal and start injecting commands into memory.
- zxzax 5y agoWayland obviously cannot solve all security issues and privilege escalations in the underlying OS -- the solution to that problem would be to just disable debugging within that security context. In Wayland the situation is also not so much different from Windows, you would just reconfigure and extend the DE itself. You are already relying on them to provide most of the UI to an extent.
- R0b0t1 5y agoIn regards to your first point, I am saying that Wayland does not do what it sets out to do and the proper way of doing it is a much larger problem than what Wayland can address. It is possibly better to avoid breaking everyone's stuff if you're just going to need to shortly do it again to fix the things Wayland does not and can not fix. For your latter point, Wayland is much different. In Windows there are ways to inject behavior into other programs or the UI elements. This exists so you can change the UI. Wayland's idea of security is very much against that, at least in the way it can be done on Windows/X11.
- zxzax 5y agoI don't know what you mean you'll need to break it again, this one particular hole is already plugged in Wayland. That won't cause additional breakage, and you'll still want to be plugging those additional holes like ptrace anyway. Input injection is being worked on in a different library, that brings a standard API that's supposed to work the same across Wayland, X11, and with sandboxed contexts: https://gitlab.freedesktop.org/libinput/libei https://gitlab.freedesktop.org/libinput/libei
- yjftsjthsd-h 5y agoIn order to implement https://www.semicomplete.com/projects/keynav/ https://www.semicomplete.com/projects/keynav/ (or anything else that listens for a specific key combo to activate)
- chrisseaton 5y agoIsn't that kind of thing the responsibility of the desktop environment service? Not an application?
- yjftsjthsd-h 5y agoI suppose the proper answer is that that very much depends on your philosophy of desktop environments; I happen to prefer modular desktops built out of separate programs from separate authors which may or may not even know about each other. This has the advantage of making it trivial to retrofit new features without needing to ask your DE upstream. Or to be more pithy: Okay, fine; what desktop environments implement keynav? With X11, I can make the answer be "(probably) all of them".
- riskable 5y agoAs long as the desktop environment provides a mechanism for applications to register their global shortcuts then all is well. However, without such a feature the application has to just listen for all keystrokes and check if it matches something the user's configured. Example: I've bound several scenes in OBS to keyboard shortcuts so that I can switch scenes no matter what application I'm in. I'm pretty sure OBS does this by listening to all keyboard events on the root window. It's not very efficient, but it works. Now imagine having to support a global shortcut daemon per DE that exists. I suppose someone would write an abstraction but really, it's something that FreeDesktop.org should provide (if it already doesn't).
- jeroenhd 5y agoWorkRave can't function well without it. Currently, it's working through a compatibility layer I think, but if that breaks, the application can't be made to work on Wayland AFAIK. Alternative shortcut managers might also run into trouble (say, clipboard managers with customizable shortcuts) although those could work with a simple permission prompt for each new shortcut. Very annoying and obtrusive to the user, but the security principles remain. There are also tools that extend some window managers with i3-like shortcuts and configurations; the config files are parsed on the fly and they need to work in the background. Then there's tools like AutoHotkey that can be great tools for productivity through scripting and custom shortcuts. A lot of AH's functionality can be replicated using the standard hotkey API, but not everything. There's also diagnostic tools (see Windows Steps Recorder) that record all keystrokes and generate a step by step report about what happened with screenshots as a guide. IMO global key capture should still be possible with the right capabilities set because there are valid use cases for it. Requiring additional permissions is fine IMO, but completely removing the option to do this is a pain.
- joaonmatos 5y agoGlobal Hotkeys for something like Quake terminal, or voice chat push-to-talk. Obviously I don't see why you would encumber the core protocol for it, but a subscription model for it as a wlroots or FreeDesktop portal could be interesting.
- reificator 5y ago> Why would you ever let a non-focused application subscribe to any key combinations? * Media playback controls * Hotkeys to take screnshots, or record videos * Quake style terminals * Ingame chat and game invites * Color pickers * Screen controls like F.lux * Window management hotkeys * Push-to-Talk in group voice calls And that's just what I personally use on my own machine. It's a massive security issue that any installed software can listen to any input activity or view/affect other windows, and that does need to be reigned in. But to claim that there's no legitimate utility to global hotkeys is absurd. We need robust permissions, not a completely crippled experience.
- topaz0 5y agoIn my experience most of these use cases are handled on linux by the window manager calling a script that sends a message to the relevant application, e.g. over dbus.
- reificator 5y agoSure, and that works fine. Just let my application tell the window manager: * What key events I care about. (I.E. what the keybind should be named in the settings UI) * What script or message to pass back to me when that event is activated. * What the default keycombo should be, if any.
- michaelmrose 5y agoSome applications prefer to provide a ui for the user to configure a hotkey. This is naturally discoverable within the scope of the other settings in its preferences screen. Providing a cli interface for automation or binding a hotkey is imo more powerful and useful but its not discoverable. Doubly so for a controlling it over dbus. The logical thing is that since there is a substantial use case and a desire to limit applications access to global info is that a permission system ought to have been built into Wayland such that applications could request not only global access to the keyboard but permission to get a notification when a particular key press happened. Since this feature was a staple of desktop operating systems for decades it ought to have been part of the plan from the start.
- pdkl95 5y agoI am currently writing a utility to spy on Super Hexagon's KeyPress/KeyRelease XEvents with the goal of recording very precise timestamps for each event. Since the walls in the game arrive at a constant rate, I want to graph my keypress/keyrelease events modulo the time between walls. My success rate for finishing Hyper Hexagonest[1] is falling; being able to inspect recent attempts might reveal where my timing is drifting. (it's only ~25%-ish finished, unfortunately) [1] https://www.youtube.com/watch?v=JJ96olZr8DE https://www.youtube.com/watch?v=JJ96olZr8DE
- tjalfi 5y agoPureText[0] is a Windows application I use regularly. It adds a keyboard shortcut for pasting as plain text. [0] https://stevemiller.net/puretext/ https://stevemiller.net/puretext/
- ashafer 5y agoiirc it's pretty much completely up to the compositor. The spec is worded to allow "focus" to enter a surface and deliver events to it, but it never specifies under what conditions the compositor should do so. So the compositor chooses what surface to deliver events to based on its own desires (like letting pointer focus enter background apps) and the users input. I think there is a protocol (used by Xwayland?) to allow a client to get events from any window if the compositor/user allows it
- Cloudef 5y agoThat's how it is. It's up to compositor to decide what events to send to what clients. Usually this means compositor only forwards input events to the focused surface.
- pjc50 5y ago> A tty doesn't know about the concept of key press or key release events. It only knows about 'data stream in' and 'data stream out'. This is basically the long and the short of it; it's a very old API, and retrofitting key-up detection reliably is a serious problem.
- CoastalCoder 5y agoPerhaps this is nitpicking, but I don't think the issue here is "data stream" vs. not-"data stream". Perhaps a better distinction is "data stream of characters" vs. "data stream that can express key events (and perhaps other stuff as well)".
- swiley 5y agoThis is correct, plan9 continues using a data stream but puts events in the data stream rather than characters. Even on Linux the low level kernel API for getting keypress events works this way.
- icedchai 5y agoIt's really a lower level protocol issue. The serial terminals upon which the "tty" model is based had no event stream outside of characters being sent or received. There are events provided by the serial port itself and delivered out-of-band though (example: modem DCD line drops, SIGHUP is generated.)
- drran 5y agoTo fix this low level problem, someone will need to modify xterm, or libvte, or any other terminal, to send escape sequences for key release events, when such mode is requested by an application.
- patrec 5y agoAlready done. Kitty and other modern terminals have an extended keyboard interface which allows to detect keypress, keyrelase and arbitrary key combinations (unlike legacy vt100-style terminal emulators, where e.g. Tab and Ctrl-I are indistinguishable). For backwards compatibility you need to switch to this mode by sending an escape sequence. https://sw.kovidgoyal.net/kitty/keyboard-protocol.html https://sw.kovidgoyal.net/kitty/keyboard-protocol.html
- ajross 5y agotl;dr: It's not about Linux, it's the terminal. Linux exposes your keyboards as /dev/inputN devices and (with the exception of oddball devices or drivers without the hardware capability to detect them) every single one of them presents clean down/up pairs on every press, every time. If you want to read the device state, read it from the kernel. The problem is that the author doesn't want to detect the device state. The author wants to write a program to run at the command line. And the command line is an environment descended from a long line of tty environments going back to the original teletypes of the 1960's. And those devices were never designed to expose a "keyboard" as a "device". They were a source of "characters" only. Now, over time countless people writing countless terminal emulators have tried to address this shortcoming, in countless not-quite compatible ways. And that's been about as successful as you'd expect. You can do it, but... But again, it's not about device support. Your command line program isn't and has never been connected to a "keyboard" device via any useful abstraction. The same feature that allows you to pipe input into it instead of typing at it makes this problem hard.
- agumonkey 5y agoWhat is the correct architecture or design to make TUI on a terminal then ? Honest question
- TeMPOraL 5y agoI think the other problem mentioned in the article is that it has to work over SSH. TUI on a local machine has no problem hooking into input devices; display-side, the difference between GUI and TUI is just what API calls you use to render stuff. But if you want to drive things remotely, you're constrained by the remoting protocol - which, in TFAs case, is SSH without X forwarding.
- alkonaut 5y agoDetecting keyup only in a window env seems like a perfectly good place to be. The problem there might be that there is no such thing as a "desktop linux" so writing a nice cross platform desktop app for any Linux regardless of window manager and desktop env I assume is a bit of a black art.
- josefx 5y agoThe X11 example only has a basic event loop with a check if the if the key up event was repeated. Everything else is setting up the window and tearing it down on close. That should work in any window manager. Most of the examples are either global listeners or outside of a window environment.
- lfmunoz4 5y agoCan't someone just modify the keyboard linux driver to detect key press events?
- marcodiego 5y agoI don't think the article is fair. I'm not used to such lower level on linux, but it is certainly doable without root. SDL has been doing it for ages. Doing it from the terminal is another thing entirely since it was never designed for such kinds of events.
- thebeardisred 5y agoI'm right there with you, but as someone who is used to the "lower levels" of Linux (to use your phrasing) this seems like the exploration of someone beginning to understand they were on the question side of an "X / Y problem" (https://xyproblem.info/ https://xyproblem.info/). Put differently, they're making the transition from Linux as a commodity used to build the thing they're dreaming of to Linux as a critical design component. For example, he calls out in the beginning: > They are interested in performing some real-time based task that is controlled using keyboard presses. In my case, the goal was to remotely navigate a robot over an SSH connection using the 'w', 'a', 's', 'd' keys. As the author discovers, that's fundamentally not how SSH works. This sort of behavior could be achieved using other mechanisms, but it's not even really an issue of the tty/pty. They're just trying to map a functional model from a different operating system to Linux. As to the discussion of keyboard handling moves into Python code I would have again, gone a different path. When the author took a turn into Python my first thought was this is trivial for evdev to handle (https://en.m.wikipedia.org/wiki/Evdev https://en.m.wikipedia.org/wiki/Evdev / https://python-evdev.readthedocs.io/en/latest/ https://python-evdev.readthedocs.io/en/latest/). Next thing I know they touch on the kernel mechanisms managed by evdev.... And pivot across to X11 (which would seem to make sense until one realizes that the transition to Wayland from X11 is far further along than a layperson might imagine). In the end a good write up which shows a lot of "raw power" on the part of the author. With some additional tutilage, exploration, or guidance they to really take their understanding to the next level. For the folks who are in the weeds (like me) it is a good guided tour. Seeing into the "beginners mind" (and taking it to heart) can provide perspective as to how to make software more intuitive. (Typed and butchered from my phone).
- riskable 5y agoI could be wrong but I'm pretty sure SDL just reads the keydown/keyup events it gets from the X server (that's why you don't need to choose a keyboard device when you use it). So without X running SDL wouldn't be able to grab keyboard events. It can grab joystick/gamepad events though since it reads those directly. In that case you have to enumerate then specify which gamepad/joystick device to use so it can open the correct /dev/ path.
- pengaru 5y agoUserspace doesn't implement debouncing in linux, that's driver territory; linux isn't an rtos. It's unclear why this is even mentioned at all. And you don't need root if your program's user is in the input group on most distros. You're opening a device node in /dev/input/XXX, the standard UNIX permissions model applies...
- zozbot234 5y ago> And you don't need root if your program's user is in the input group on most distros. It's complicated. Being in the input group is part of it, but to actually get the access you need to interface with the freedesktop-derived multiseat stack, which is as clunky as everything else that comes from the fd.o folks.
- klodolph 5y agoYes, I think the article's description of getch() as being inadequate due to a lack of debouncing is a bit weird. I would think that debouncing would be done even earlier, in the keyboard itself, since the debouncing you need is necessarily specific to the actual hardware. While USB HID keyboards are state-based (they transmit a packet listing which keys are pressed), the PS/2 protocol is event-based, and transmits "make" and "break" messages for key-press and key-release events, respectively. (That said, I wouldn't be surprised if debouncing were done in the driver. I just think it's the wrong place for it.)
- pengaru 5y agoI suspect the author may have some experience implementing ad-hoc buttons via GPIO polling from userspace, and maybe used getch() in doing so.
- klodolph 5y agoThe thing about getch() is that there's not really one standard getch(). There's a Windows version of getch() which is compatible with, notably, Borland's version for DOS (I think), and then various people have reimplemented getch() for other systems.
- dec0dedab0de 5y agoThis might be a dumb question, but why would I want to detect a Keyup event in the first place?
- dragontamer 5y agoLets say you're porting a fighting game, such as Blazblue, to Linux. Carl Clover (Blazblue character) attacks on keydown, while Carl's doll attacks on key-up. Expert players change the rhythms of their key-down / key-up to make combos possible. In general, these kinds of fighting game characters are called 'Negative Edge' characters. There's a large number of them in many different fighting games. I know its present in Street Fighter, Marvel vs Capcom, Mortal Kombat, and Injustice. Even SSB:U (Shield-release / perfect shields) is a negative-edge event, showing just how common negative-edges have become in modern games. I personally am only really good at Blazblue: and thus, I know that Carl Clover, and to a lesser extent Taokaka and Lichi are negative-edge characters. Its an entire "character design" philosophy, to make certain characters feel much different from others. But its all over the place. ---------- I HATE playing negative edge characters. If I realize a character has negative-edges, I run the heck away from them. Nonetheless, I accept that fighting games are fun because there's "always a character" that matches someone's personality. If someone else has fun playing negative edge characters, I want to welcome them into the community, and therefore want to support negative-edge gameplay into a game. Its not about "me", its about "the community of players".
- theon144 5y agoThe problem isn't Linux, but the terminal environment...
- dragontamer 5y agoThen lets say a hypothetical terminal video game that acts on negative-edges. My point: negative-edge gameplay is common in fighting games (and maybe other video games). Its reasonable to expect that a video game designer would use that technique. I've played terminal action games before. Lets say you're programming one and have some useful function mapped to the negative edge. How do you implement that in the terminal? -------- Megaman's charge shot (and most "charge shot" video games, including Samus / Metroid, Rocket Knight Adventures, etc. etc.) Maybe those platformers are more common? Either way, the negative-edge is all over video games, its a good control scheme.
- leoc 5y agoSeeing this near the top of HN is a bit like seeing a "Why Does Italian Have All These Screwy Verb Endings?" post near the top of a web forum for professional students of linguistics ...
- munchbunny 5y agoWhat's obvious to you might not be obvious to everyone. I've only heard reference to the issue previously but never looked at a detailed discussion, mostly out of not caring because I've never had to do something that depended on detecting keyup events in Linux in a non-browser context.
- ajkjk 5y agoReally? There is almost no information in this article that I have ever heard before, and I imagine that I'm a pretty typical reader of this forum. You might have have a very mis-calibrated model of HN readers.
- BoppreH 5y agoI'm the author of the `keyboard`[1] library mentioned in the article. - It reads all events because it's meant to interface with your keyboard directly, for global hotkeys, macros, disabling or remaping keys, etc. Interacting with windows and per-application hotkeys are explicitly not a goal of the library at the moment. - It reads /dev/input because the library was developed to work in as many environments as possible, including headless installations like raspberry pi's that may not have a graphical environment or even a monitor. There's an open change to _try_ to communicate with the X server first and fallback to /dev/input, but event suppression is not working reliably with this yet. - It could read /dev/input by just being in the `input` group, but then `dumpkeys` doesn't work and you are stuck typing numeric scan codes instead of key names. Due to a series of unfortunate personal circumstances I've been unable to give the proper maintenance the library requires, but the issues and thank yous have never stopped, and I'm slowly getting back to it again. --- Little known trick: if you run the library as a standalone module (`python -m keyboard`), it prints a JSON object for each detected event. You can save them to a file and pipe them back (`python -m keyboard < events.txt`) to replay them like a macro. Little known trick 2: I also created `mouse`[2], the companion library for my second favorite peripheral. It's the same thing, but for mouse events. [1]: https://github.com/boppreh/keyboard https://github.com/boppreh/keyboard [2]: https://github.com/boppreh/mouse https://github.com/boppreh/mouse
- RickHull 5y agoFor those interested, I wrote a similar library for Ruby: https://github.com/rickhull/device_input https://github.com/rickhull/device_input
- vanous 5y agoThank you very much for the library. Recently I have been learning touch typing and detecting system wise characters typed per day plus speed and acuracy is on my todo list. I already selected the keyboard module for this after some initial research.
- jorl17 5y agoHi! I've used your library in various projects and am very grateful for it! I figured it'd be worth a shot to ask you this here, since I just caught you: have you considered adding some support for "rehooking"/"restarting" the library, so that callees can restart your library whenever they detect that their environment has changed (e.g. input devices have been (dis)connected, etc)? I've contributed with some info to this issue https://github.com/boppreh/keyboard/issues/264 https://github.com/boppreh/keyboard/issues/264 as to how we're dealing with this in some of our projects. Funnily enough, I didn't know about `python -m keyboard`, and had I known about it, it would definitely have saved me some time! Once again, thank you very much for your hard work!
- teknopaul 5y agoThis is silly, you can't detect key up or keyb down in a tty, you detect what the terminal emulator gives you: chars and Esc sequences. It's as sane as asking why you can detect joystick movements or the kettle boiling. It bothers me how little people understand about the terminal and "the command line", despite using it everyday. If people read man bash from top to bottom they would probably get a feel for what is going on. Learn how changing the title of your tty works and there is not much left to _not_ understand.
- imtringued 5y agoYeah, the author is not running into a Linux problem. He is running into an SSH problem. >Something worth noting to avoid confusion is that if you run the example python keyboard example in the introductory paragraph of this article over an SSH connection, the code will still work and run, but when you type characters, nothing will happen. >What gives?!? That's because it will be detecting 'local' keyboard events from the machine you just SSHed into! If it's a cloud server like EC2 or something, there probably aren't any keyboards attached to it! "What gives?!?"? Is this supposed to be a joke? What else did you expect? Expecting your local keyboard to connect to the remote machine is as insane as expecting your keyboard to read the keys you want to press from your mind. The problem he is running into is that SSH only forwards a character stream to the remote machine and that is entirely an SSH problem and the solution to forwarding the keyboard has been X forwarding. What he is asking is that SSH should be able to forward the keyboard without X. He even mentioned a workaround that involves compensating the lack of this feature by forwarding the keyboard himself. >Well, you have to build your own client/server application where a client/server listens locally on the machine with the keyboard, and then forwards these events to the remote machine where you're running the applications that needs to respond to these events. My suggestion is that he should send an email to the openssh developers to add keyboard forwarding without X.
- memoblobi 5y agoBecause they are serial character devices?
- GuB-42 5y agoSmall nitpick, if you use WASD for control, it may be better to use the keycodes and not the characters. People with non-QWERTY layouts probably want to use whatever keys they have on this location (ex: ZQSD for AZERTY layout).
- pcwalton 5y ago> So, what do you do when you want to send 'key release' events over SSH when the target machine doesn't have an X server? Well, you have to build your own client/server application where a client/server listens locally on the machine with the keyboard, and then forwards these events to the remote machine where you're running the applications that needs to respond to these events. That sounds like a lot of work because you have to set up all the sockets and custom messaging protocols, but there's no way around it because the key release events simply aren't forwarded over your terminal-based SSH session when there is no X server forwarding configured. This isn't that much work if you write it as a trivial Web app using WebSockets. Then you can leverage the browser's OS abstraction layer instead of having to write your own.
- deleted 5y ago[deleted]
- account42 5y ago> Are you interested in detecting local 'key up' events over an SSH connection without involving an X server? Oh boy, are you in for a disappointment! It turns out, that it's impossible. I don't mean the "I tried really hard and couldn't figure out how to do it, so it must therefore be impossible" kind of impossible, but the real kind of impossible where the keyup event doesn't get communicated at all during a non-X forwarded SSH session. Here is an experiment you can do to prove this to yourself [...] This isn't really a good experiment as it assumes that everything that can be sent over SSH will always be sent. This not the case in the terminal world where by default you get a processed character stream and have to explicitly enable anything else. For example kitty's keyboard protocol [0] mentioned elsewhere in this thread needs to be activated by sending the escape sequence CSI > 1 u to the terminal on the other side of the SSH connection. [0] https://sw.kovidgoyal.net/kitty/keyboard-protocol.html https://sw.kovidgoyal.net/kitty/keyboard-protocol.html