7 ms·

Building a Bejeweled Bot

22 points by RC1140 15y ago

patrickk 15y ago
Very interesting. I'm also very interested in making bots to automate various web games, in my case casino games. When I was in college, I came across a gambling strategy for online roulette. I tested it for a few weeks but I gave up on it as it was extremely boring and tedious, even though it seems legitimately profitable. A few years later I heard about bots that automate poker and so on, so I looked at tools that I could use to code up the roulette strategy to play it automatically. I looked into AutoIt and wrote some scripts, but the code ended up being beyond my ability to write, as my knowledge of AutoIt starting out was zero. Even a relatively simple game like roulette (only 37 supposedly random possible outcomes on a European roulette table) ended up having bot logic that was quite complex. However a few weeks ago I came across this excellent article: http://active.tutsplus.com/tutorials/workflow/how-to-build-a-python-bot-that-can-play-web-games/ http://active.tutsplus.com/tutorials/workflow/how-to-build-a... So I looked at re-writing the bot in Python instead. Turns out that writing it in Python is FAR simpler than AutoIt, and the code far simpler to figure out. Even though I had close to zero knowledge of Python starting out (learning Python really quickly by using it to build a real project, rather than learning Python theory first.) Now my bot is about 90% complete, just a couple of steps away from having a working (potentially profitable) bot. Really cool what you can do in a few weeks in Python from a standing start.
- RC1140 15y ago
  Thanks for that link , I really did want to use python for my bot (i got reasonably far) , but after digging through the autoit docs it turned out that what I was trying to do in python could be done alot simpler in AutoIt. Specifically I did not have to worry about grabbing an image of the screen and then searching the image for a pixel (to determine the start location) , I could instead just search for the start pixel using a built function in autoit.
- goostavos 15y ago
  Hey! That's me! I wrote that there tutorial. Ignore my shameful, newbie-ish coding practices in that tutorial..
  - patrickk 15y ago
    There's nothing to be ashamed of! I learned an awful lot from it. The part where you "read" values from the screen was particularly interesting. I had coded up the equivalent functionality in AutoIt, it involved iterating through an array of all possible images (stored locally)....ugh. Utter mess. Your method was much cleaner. How did you get interested in automating web games? What is your background? Any other interesting projects you are cooking up? I find the whole area of automating tedious human-driven actions fascinating. If the roulette bot works out, my next project is going to be automating spread betting/forex trading or perhaps some sort of sports betting arbitrage. Loads of possibilities out there in my opinion. ps. If I end up 'making bank' as a result, let me know where I send the cheque as a thank you. My email is in my profile ;-)
    - goostavos 15y ago
      Ha, these Bejeweled Bots are actually what got me into this stuff. Not this one specifically, but someone linked to video of a Bejeweled Blitz bot on /r/programming awhile ago. I was instantly fascinated. No cool projects yet. There is a bot that I've been really wanting to make, just because I can't get past the stupid 4th level, but it requires a bit more brain power than I currently have.. Speaking of which, if you're interesting in the computer vision-y stuff check this out: http://www.udacity.com/ http://www.udacity.com/. They're having a class on programming a robotic car. It looks like it will touch on some pretty neat vision stuff.
      - patrickk 15y ago
        It's funny, I was just discussing udacity with a co-worker today. I might try some of the easier courses before I tackle the more hardcore topics like a self-driving car! On a slightly tangential note, I'm blown away when I look at the learning resources on the web in the last year or two. I largely learned the (admittedly rudimentary) iOS knowledge I have from the Stanford U course on iTunes[1]. Then you look at things like Udacity, Coursera and Khan Academy. In the case of the self-driving car course, you are getting a world-class computer-science education, free, from one of the foremost experts in AI, all from the comfort of your own bedroom. We might not have flying cars yet, but you can get a world-class education for free! The times we live in. I'd also encourage you to do more write ups on your bot experiences! You have a knack for explaining things well. I'm toying with the idea of doing maybe a series of YouTube videos or blog posts myself once my Python skills get above the level of being totally embarrassing ;-) BTW, here's a link for a series of blog posts on how this guy builds working poker bots: http://web.archive.org/web/20090608115755/http://www.codingthewheel.com/archives/how-i-built-a-working-poker-bot http://web.archive.org/web/20090608115755/http://www.codingt... http://web.archive.org/web/20100130074034/http://www.codingthewheel.com/tags/poker http://web.archive.org/web/20100130074034/http://www.codingt... (Had to put it through the wayback machine as some current employer of his persuaded him to take the content offline!) Not sure if it's something you would be interested in, but have a look at some of the content. He goes through using Monte Carlo simulations and everything to aid decision making. Very C++ and Windows based, but have a look nonetheless, there may be transferable concepts to your own projects. [1] http://itunes.apple.com/us/itunes-u/developing-apps-for-ios-hd/id395605774 http://itunes.apple.com/us/itunes-u/developing-apps-for-ios-...
karpathy 15y ago
I would recommend using AutoHotkey instead of AutoIt. I wrote some bots for fun for a few similar games but skipped Bejeweled in favor of (a simpler) alternative called Diamond Dash. The bot could hold its own but it couldn't beat some of my better friends. To squeeze out the extra performance one has to start doing much more complicated things. For example, the blocks in these games rarely appear in fixed coordinates but slide around, explode (and therefore partially occlude other tiles), etc. Recently, this has been turning into a bit of a cat and mouse game with game developers. For example, they started adding noise on top of sprites to throw off very simple methods. But there is something very fun in developing scripts that moves your mouse around and play for you. I like that game much better :)
- RC1140 15y ago
  Thanks for the feedback , I will look into autohotkey but in the end autoit was just there to get me started, I want to look at a other languages just to see how it would be done differently.
  - patrickk 15y ago
    I looked into AHK for my roulette bot originally. AutoIt is quite similar to AHK but it can do more complex stuff. If you are just looking for basic GUI manipulation then AHK is your tool for the job. http://stackoverflow.com/questions/1686975/choosing-a-windows-automation-script-language-autoit-vs-autohotkey http://stackoverflow.com/questions/1686975/choosing-a-window...
Lozzer 15y ago
I had a go at something similar for the Facebook version a couple of years ago. My main goal was to stop playing the game, but it was also an excuse to try Clojure. At the time it absolutely thrashed my best scores, though I'm sure it won't work with the updates the developers have made since then. I think it was the first thing I wrote that drove home the power of having a REPL for exploratory programming: https://gist.github.com/490689 https://gist.github.com/490689
tldnr 15y ago
Interesting post, a few friends and I did something similar in Java/C# a while back. It's surprisingly easy to automate, and very satisfying to see working.

22 points by RC1140 15y ago

Very interesting. I'm also very interested in making bots to automate various web games, in my case casino games. When I was in college, I came across a gambling strategy for online roulette. I tested it for a few weeks but I gave up on it as it was extremely boring and tedious, even though it seems legitimately profitable. A few years later I heard about bots that automate poker and so on, so I looked at tools that I could use to code up the roulette strategy to play it automatically. I looked into AutoIt and wrote some scripts, but the code ended up being beyond my ability to write, as my knowledge of AutoIt starting out was zero. Even a relatively simple game like roulette (only 37 supposedly random possible outcomes on a European roulette table) ended up having bot logic that was quite complex. However a few weeks ago I came across this excellent article: http://active.tutsplus.com/tutorials/workflow/how-to-build-a-python-bot-that-can-play-web-games/ http://active.tutsplus.com/tutorials/workflow/how-to-build-a... So I looked at re-writing the bot in Python instead. Turns out that writing it in Python is FAR simpler than AutoIt, and the code far simpler to figure out. Even though I had close to zero knowledge of Python starting out (learning Python really quickly by using it to build a real project, rather than learning Python theory first.) Now my bot is about 90% complete, just a couple of steps away from having a working (potentially profitable) bot. Really cool what you can do in a few weeks in Python from a standing start.

RC1140 15y ago
Thanks for that link , I really did want to use python for my bot (i got reasonably far) , but after digging through the autoit docs it turned out that what I was trying to do in python could be done alot simpler in AutoIt. Specifically I did not have to worry about grabbing an image of the screen and then searching the image for a pixel (to determine the start location) , I could instead just search for the start pixel using a built function in autoit.
goostavos 15y ago
Hey! That's me! I wrote that there tutorial. Ignore my shameful, newbie-ish coding practices in that tutorial..
- patrickk 15y ago
  There's nothing to be ashamed of! I learned an awful lot from it. The part where you "read" values from the screen was particularly interesting. I had coded up the equivalent functionality in AutoIt, it involved iterating through an array of all possible images (stored locally)....ugh. Utter mess. Your method was much cleaner. How did you get interested in automating web games? What is your background? Any other interesting projects you are cooking up? I find the whole area of automating tedious human-driven actions fascinating. If the roulette bot works out, my next project is going to be automating spread betting/forex trading or perhaps some sort of sports betting arbitrage. Loads of possibilities out there in my opinion. ps. If I end up 'making bank' as a result, let me know where I send the cheque as a thank you. My email is in my profile ;-)
  - goostavos 15y ago
    Ha, these Bejeweled Bots are actually what got me into this stuff. Not this one specifically, but someone linked to video of a Bejeweled Blitz bot on /r/programming awhile ago. I was instantly fascinated. No cool projects yet. There is a bot that I've been really wanting to make, just because I can't get past the stupid 4th level, but it requires a bit more brain power than I currently have.. Speaking of which, if you're interesting in the computer vision-y stuff check this out: http://www.udacity.com/ http://www.udacity.com/. They're having a class on programming a robotic car. It looks like it will touch on some pretty neat vision stuff.
    - patrickk 15y ago
      It's funny, I was just discussing udacity with a co-worker today. I might try some of the easier courses before I tackle the more hardcore topics like a self-driving car! On a slightly tangential note, I'm blown away when I look at the learning resources on the web in the last year or two. I largely learned the (admittedly rudimentary) iOS knowledge I have from the Stanford U course on iTunes[1]. Then you look at things like Udacity, Coursera and Khan Academy. In the case of the self-driving car course, you are getting a world-class computer-science education, free, from one of the foremost experts in AI, all from the comfort of your own bedroom. We might not have flying cars yet, but you can get a world-class education for free! The times we live in. I'd also encourage you to do more write ups on your bot experiences! You have a knack for explaining things well. I'm toying with the idea of doing maybe a series of YouTube videos or blog posts myself once my Python skills get above the level of being totally embarrassing ;-) BTW, here's a link for a series of blog posts on how this guy builds working poker bots: http://web.archive.org/web/20090608115755/http://www.codingthewheel.com/archives/how-i-built-a-working-poker-bot http://web.archive.org/web/20090608115755/http://www.codingt... http://web.archive.org/web/20100130074034/http://www.codingthewheel.com/tags/poker http://web.archive.org/web/20100130074034/http://www.codingt... (Had to put it through the wayback machine as some current employer of his persuaded him to take the content offline!) Not sure if it's something you would be interested in, but have a look at some of the content. He goes through using Monte Carlo simulations and everything to aid decision making. Very C++ and Windows based, but have a look nonetheless, there may be transferable concepts to your own projects. [1] http://itunes.apple.com/us/itunes-u/developing-apps-for-ios-hd/id395605774 http://itunes.apple.com/us/itunes-u/developing-apps-for-ios-...

karpathy 15y ago

I would recommend using AutoHotkey instead of AutoIt. I wrote some bots for fun for a few similar games but skipped Bejeweled in favor of (a simpler) alternative called Diamond Dash. The bot could hold its own but it couldn't beat some of my better friends. To squeeze out the extra performance one has to start doing much more complicated things. For example, the blocks in these games rarely appear in fixed coordinates but slide around, explode (and therefore partially occlude other tiles), etc. Recently, this has been turning into a bit of a cat and mouse game with game developers. For example, they started adding noise on top of sprites to throw off very simple methods. But there is something very fun in developing scripts that moves your mouse around and play for you. I like that game much better :)

RC1140 15y ago
Thanks for the feedback , I will look into autohotkey but in the end autoit was just there to get me started, I want to look at a other languages just to see how it would be done differently.
- patrickk 15y ago
  I looked into AHK for my roulette bot originally. AutoIt is quite similar to AHK but it can do more complex stuff. If you are just looking for basic GUI manipulation then AHK is your tool for the job. http://stackoverflow.com/questions/1686975/choosing-a-windows-automation-script-language-autoit-vs-autohotkey http://stackoverflow.com/questions/1686975/choosing-a-window...

Lozzer 15y ago

I had a go at something similar for the Facebook version a couple of years ago. My main goal was to stop playing the game, but it was also an excuse to try Clojure. At the time it absolutely thrashed my best scores, though I'm sure it won't work with the updates the developers have made since then. I think it was the first thing I wrote that drove home the power of having a REPL for exploratory programming: https://gist.github.com/490689 https://gist.github.com/490689

tldnr 15y ago

Interesting post, a few friends and I did something similar in Java/C# a while back. It's surprisingly easy to automate, and very satisfying to see working.