5 ms·
SikuliX – Automate what you see on a computer monitor
- jnwatson 5y agoI had great success with SikuliX about a decade ago in automating a native UI. It worked quite well, though it was processor intensive. The UI for scripting was well done too. The nice thing about using CV is that the UI can change somewhat and the automation still works.
- eastendguy 5y agoYes, using CV/OCR for automation is the future.
- azeirah 5y agoUnable to connect ;( Down right when on HN frontpage?
- guiambros 5y agoHN hug of death. https://web.archive.org/web/20220212231525/http://sikulix.com/ https://web.archive.org/web/20220212231525/http://sikulix.co...
- derac 5y agoI've used this with Jython and I find it very cool and straightforward to use. I think it has potential for automated testing implemented by non-programmers. A potential downside is that I imagine running OpenCL takes a decent bit of GPU power. It is also more flexible and quicker way to write automation scrips than most of the alternatives. It could be a good alternative to AHK, for instance.
- senectus1 5y agoLink seems to be dead... but i found: https://sikulix.github.io/ https://sikulix.github.io/
- ladberg 5y agoYears ago I used to use SikuliX to automate grinding in MMOs! My first introduction to Python/scripting in general.
- trinovantes 5y agoSikuli is quite nostalgic. I remember using it to automate multiple accounts to play those FB games
- terrycody 5y agoso do we need to know code to use this? Say, how to automate the grinding of a MMOs if I only go to a fixed place and farm fixed mobs?
- garou 5y agoI've used it to automate a desktop app in the past, a ruby script in the server side running in multiple windows instances doing GUI things… It was like having robots working for us…
- sidpatil 5y ago> It was like having robots working for us… I suppose that's why they call it "robotic process automation". https://en.m.wikipedia.org/wiki/Robotic_process_automation https://en.m.wikipedia.org/wiki/Robotic_process_automation
- MonaroVXR 5y agoSounds interesting, can you tell more about it?
- imachine1980_ 5y agohttps://web.archive.org/web/20220130104843/http://www.sikulix.com/ https://web.archive.org/web/20220130104843/http://www.sikuli...
- syntaxing 5y agoI used to work with SAP a lot and my employer decided to disable macros. I had some extremely tedious work (making and maintaining PN, woes of the MechE life) so I just wrote a script with SikuliX to automate the work by importing an excel sheet. It easily saved me hundreds of hours of work. And of course I followed the golden rule of automating things, never tell your manager.
- zulvkr 5y agoThe golden rule indeed. My previous job as pharmacist in pharma company has a lot of grunt job that I tried to automate using only word and excel sheets. It possibly saves a lot of man hour. Why only excel and word? Pretty much because I almost can't install anything in company laptop. excel Macros also prohibited. I was thinking to create a simple js automation app in browser if it gets attention. I showed my little automation tools to my superior but it failed to get appreciation nor support. In the end my colleagues asks for help to run the automation but they refuse to learn how to use it themselves. Bah.
- MaxikCZ 5y ago> In the end my colleagues asks for help to run the automation but they refuse to learn how to use it themselves. Bah. Sounds to me like "please do my work for me" kinda ask.
- DoingIsLearning 5y ago> Why only excel and word? Pretty much because I almost can't install anything in company laptop. excel Macros also prohibited. I was thinking to create a simple js automation app in browser if it gets attention FYI assuming you're on windows from the office suite, python msi installer allows you to install without admin escalation (install just for this user). Definitely a lot easier to automate jobs with python.
- hackerfromthefu 5y agoI used this years ago to setup a UI test automation framework for iOS apps in the very first days of iOS, before any commercial products were available. iOS was run in the emulator with Sikuli driving it. It was very effective, Sikuli was a pleasure to work with and the ability to record the tests and then edit the scripts and make abstractions was invaluable. It was very effective.
- okaram 5y agoIf anybody else is seeing the server dying, the project is on github https://sikulix.github.io/ https://sikulix.github.io/
- kleer001 5y agoniiice, could-a used this in my last job which I just left yesterday
- throwamon 5y agoWow, I've been looking for something like this for a long time but all I could find was proprietary stuff or not-that-good free alternatives. I'm not being dramatic when I say I feel this is going to change my life, so thank you!
- dandanua 5y agoThis is a pretty old project, started long before AI hype. It's still very cool, but I wonder why no one wrote something similar using all the advancements in CV that we have now.
- eastendguy 5y agoNewer: UI.Vision is newer and I find it easier to use. A key difference is that UI.Vision supports text reading (via OCR) out of the box: https://ui.vision/rpa/x/desktop-automation https://ui.vision/rpa/x/desktop-automation
- elitepleb 5y agoSo does sikulix, via tesseract, out of the box.
- eastendguy 5y agoMy key point was that there are newer alternatives (the OP asked for that). PyAutoGUI would be another newer alternative. So are Applitools and UIPath, but they are not open-source. The challenge with Tesseract is that its OCR quality is (too) low for screenshots because screenshots are usually text on colored background. Tesseract is optimized for classical text document OCR. Of course, you can switch OCR engines with Sikuli (I did that before), but this requires touching the source code.
- monkeydust 5y agoLooks good. Plenty of RPA companies flogging this type of technology around at the moment under the guise of 'Intelligent Automation' or 'AI powered Automation'.
- bryanrasmussen 5y agoI worked on a government project in which we put a lot of work into making it difficult for automated attacks, after which our automated ui tests were not able to run, I left before a solution was found (if it was) but was arguing that SikuliX was the solution before I left.
- Shalle135 5y agoReminds me of a program called SCAR (Shit Compared to AutoRune). It was designed for runescape but was also frequently used for other things. https://scar-divi.com/oldmanual/ https://scar-divi.com/oldmanual/
- fake-name 5y agoGithub for the project: https://github.com/RaiMan/SikuliX1 https://github.com/RaiMan/SikuliX1 It seems to have most of the content as the main site (which is down ATM).
- dorianmariefr 5y agoI've used AutoHotKey on Windows a while ago and the boss was impressed Basically copying from Excel into their own system via an interface
- Veen 5y agoI use Keyboard Maestro for similar grunt work on the Mac. It's quite powerful.
- mensetmanusman 5y agoSince my gaming days, I have always wondered whether there is software that could exactly track the bit stream of mouse moments at the frequency they are generated. I was trying to automate complex jump movements in quake 2. Never did figure out how (it was not possible with Sikuli or other popular macro software at the time)
- c6401 5y agoI know and like this application for a long time, but I wish it worked just in python instead of jython.
- milleramp 5y agoI have used Sukuli off and on for 9 years, it really is great for quick automation. Thanks to RaiMan for taking it over and improving/supporting it!