2 ms·

for lightweight automation outside the browser: https://github.com/elyase/screenium https://github.com/elyase/screenium

• okso 2y ago
  macOS only (uses Apple Vision framework)
  • erikcw 2y ago
    I've used SikuliX[0] in the past for similar purposes. Unfortunately the author hasn't had much time to maintain it recently. [0] https://github.com/RaiMan/SikuliX1 https://github.com/RaiMan/SikuliX1
• oulipo 2y ago
  Very cool! Could be a kind of open-source, text-based (eg recipes are .md with instructions) version of KeyboardMaestro! I'd love to see such an "open automation" format (could even be more general than pure software, could also automate your IoT or whatever, through extensions) eg you could have a file "Type my bank login password" for bank websites which doesn't let you use keyboard input but force you to click on stuff, like a self-documented script using .md with code # Type my bank login password ## Trigger ```trigger:hotkey key: cmd+l filter: frontmost-app=Chrome and chrome.tab.url=~mybank.com/login ``` ## Deps ```ensure-deps shell-runner>=1.* screen-ocr>=1.* python-runner>=1.* ``` Ensure that my system has the proper extensions for the framework, to run all tasks ## What it does This automation lets me input my password in a "click-only" input for my lousy bank UI ```run:shell /bin/sh:capture-output=password echo $(op --vault personal --site mybank) ``` (the above runs the shell script and captures the output as a "password" variable I can use in other scripts below) ```run:screen-ocr:capture-output=ocr-result window:chrome ``` ...go on scripting using typescript/python to locate the numbers in the ocr-result