4 ms·
Right ... the output isn't really "keyboard & mouse operations" as stated in the abstract. The output is a sequence of atomic commands ("shoot", "select_weapon
by finnh 2y ago
Right ... the output isn't really "keyboard & mouse operations" as stated in the abstract. The output is a sequence of atomic commands ("shoot", "select_weapon") or composite commands ("follow") that are then translated into mouse actions by a human-written layer.
Similarly, when I'm typing this post my output isn't "keyboard interrupts" nor "TCP frames". Those things do happen, but only by the grace of others' work.
Still I like the basic framework idea & I suppose future work looks at pushing the provided layer lower & lower - so the system can learn those atomic commands, rather than having humans provide them - and pushing the logic higher & higher via AI-generated composite commands.
- fordacious 2y agoMany games do happen to be set up this way wrt their input systems if they happen to leverage Steam input. Player actions in game are defined as abstract verbs with concrete input device binding being left to the player / underlying system. Valve is trying to nudge games towards being platonic objects isolated from specific systems (so they'll be readily compatible steam deck, vr, pc etc...). Translating these verbs in an automated way with an agent is an interesting side effect.
- miohtama 2y agoThe problem is not triggering the action, the problem is being at the right place at the right time and looking into the right direction.