6 ms·

Gremlin.js: Throw a thousand monkeys at your page

132 points by sam152 13y ago

sergiotapia 13y ago
Netflix also has a famous division of monkeys that they unleash to take down servers in the hive, and generally muck about. They want to break as many things as possible while keeping uptime and that sounds freaking awesome! :D
- midas007 13y ago
  Chaos monkeys meet entropy gremlins. (Watch out for water.)
- madeofpalk 13y ago
  More info (the open sourced it): http://techblog.netflix.com/2012/07/chaos-monkey-released-into-wild.html http://techblog.netflix.com/2012/07/chaos-monkey-released-in...
  - bhhaskin 13y ago
    That is pretty darn cool!
ianbicking 13y ago
A similar library that I wrote: https://github.com/ianb/walkabout.js https://github.com/ianb/walkabout.js
imkevinxu 13y ago
lol I'm gonna call this "Twitch Plays the Internet"
- wildpeaks 13y ago
  You read my mind, that totally looks like anarchy mode applied to to the web :)
espeed 13y ago
Not to be confused with the Gremlin graph traversal language (https://github.com/tinkerpop/gremlin/wiki https://github.com/tinkerpop/gremlin/wiki) or the JavaScript version of Gremlin.
mitchellh 13y ago
For those who don't know: this is PROBABLY named after Palm OS Gremlins. Palm OS had a debugging tool called "gremlins" that would randomly tap and type and gesture all over your Palm OS app. This debugging tool would often find edge cases that would end up crashing your app. As someone who used to program for Palm OS a lot, I was pretty happy to see this name. :)
- Semaphor 13y ago
  Or even more probable, they are both simply named after Gremlins [0] [0]: http://www.etymonline.com/index.php?search=gremlin http://www.etymonline.com/index.php?search=gremlin
- fzaninotto 13y ago
  As the author of this lib, I can confirm: it was named after the movie.
- apinstein 13y ago
  Back in the very early days of Palm OS, Palm had a "Quality Certification" you could apply for, and if your app passed, you'd get a bunch of marketing benefits. So we went for the highest-level certification on our products, which required surviving 1,000,000 gremlins. At first I was very skeptical that it would find any meaningful bugs, but in fact it was very good at finding memory leaks and edge cases. In the end it took us several weeks to finally be able to survive the 1M gremlins without a crash. A single run of 1M gremlins would take about 24 hours. In later years they added the capability to supply your own dictionary of values to apply to certain inputs so that gremlins could get past validation, and it would also save the entire state of the device every 10K events so that you could easily reproduce errors that Gremlins found. I could definitely see this as a good way to stress-test certain parts of your app, though it takes a ton of time, and frankly without surrounding tooling (like resource monitoring, saving "state") it wouldn't be practical.
presty 13y ago
For Android: http://developer.android.com/tools/help/monkey.html http://developer.android.com/tools/help/monkey.html
quesera 13y ago
The 1984 Macintosh had a similar testing tool called The Monkey. Andy Hertzfeld: http://www.folklore.org/StoryView.py?story=Monkey_Lives.txt http://www.folklore.org/StoryView.py?story=Monkey_Lives.txt
xhrpost 13y ago
I think this is essentially "Fuzz testing"[1]. Something I read about in a book years ago but never saw done in real-life. The term may be antiquated at this point. [1] http://en.wikipedia.org/wiki/Fuzz_testing http://en.wikipedia.org/wiki/Fuzz_testing
- dleen 13y ago
  Chromium uses fuzzing extensively: http://blog.chromium.org/2012/04/fuzzing-for-security.html http://blog.chromium.org/2012/04/fuzzing-for-security.html
- billyhoffman 13y ago
  I wouldn't call this fuzz test because it focuses on simulating legitimate actions. Do random clicking. Do random scrolling. Do random mouseovers. Do random [any DOM event]. Yes, they are doing this at a massive scale, but the key here is that germlin.js can only do things a user can do. In short, Gremlin is just automating actions; basic legal, legitimate actions. Still cool, and great for stress testing or looking for race conditions in your UI code, but not fuzz testing. Fuzz testing, on the other hand, is about modifying the data, to interacting with the UI. To use an analogy, fuzz testing Microsoft word might involve corrupting various structures in the DOC or DOCX file or in an OLE embed with malformed data and seeing how the parser/program reacts. The Gremlin.js equivlent would be just clicking on a bunch of buttons in the Word UI really really fast. Both are helpful, but they are testing different things
  - yeukhon 13y ago
    Not entirely true. Fuzzing is not about modifying data. I don't think that's the right way to describe fuzzing. It's about testing the durability of a program by trying all kinds of data, as greedy as possible. This includes known problematic inputs and random inputs to match some expectations. By random it can either be totally random (any length, any pattern) or protocol-aware. Random clicking is a form of random data testing because you are trying random input stream to a program. Your argument is not entirely wrong either. By fuzzing his UI he may trigger the browser to crash. He may trigger his monkey to crash. Fuzzing is a very general technique. I can write a fuzzer that fuzz Firefox's UI Australis. What will I look for? Maybe after opening 100 tabs and closing the 45th tab the titlebar disappeared. Or resize the browser from range s to range w I will find some range will cause the UI to look ugly (style overflow, etc). Or after clicking on the scrollbar several times consecutively the browser crashed. Barton Miller, the "Father of Fuzzing" did UI fuzzing by simulating actual keystrokes and mouse clicks. See ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-nt.pdf In his Forwards page, he even mentioned "Monkey" http://pages.cs.wisc.edu/~bart/fuzz/Foreword1.html http://pages.cs.wisc.edu/~bart/fuzz/Foreword1.html
toolslive 13y ago
So the person who named this is old enough to have seen (and liked) http://en.wikipedia.org/wiki/Gremlins http://en.wikipedia.org/wiki/Gremlins I guess 40-something.
- minikomi 13y ago
  Or older perhaps.. https://www.youtube.com/watch?v=D1xqrdtJs8w https://www.youtube.com/watch?v=D1xqrdtJs8w
- reubenmorais 13y ago
  The gremlin is older (and broader) than that movie: https://en.wikipedia.org/wiki/Gremlin https://en.wikipedia.org/wiki/Gremlin
fzaninotto 13y ago
Hi! I'm one of the authors of Gremlins.js. Glad to see that some of you find it interesting. I wrote a blog post a while ago about the purpose of the lib, it's called "Completing the JavaScript Test Stack". Worth reading if you wonder what this is all about. http://redotheweb.com/2014/01/07/completing-the-js-test-stack-introducing-greminsjs.html http://redotheweb.com/2014/01/07/completing-the-js-test-stac...
metaedge 13y ago
I just give my laptop or iPad to 2 year old son and it does the same trick!
Houshalter 13y ago
That looks really cool, though I think it would have a hard time navigating past the first few menus, and could hit features that are only rarely meant to be touched (e.g. "delete everything".) Perhaps a searching algorithm that intentionally tries to explore every menu and feature, navigate to them and then test them. Or intentionally tries things it knows are more likely to find bugs.
- fzaninotto 13y ago
  All the gremlins are configurable, you can specify which elements they can interact on (and therefore exclude "dangerous" elements). Relevant doc: https://github.com/marmelab/gremlins.js#configuring-gremlins https://github.com/marmelab/gremlins.js#configuring-gremlins
atom-morgan 13y ago
The GIF in Github makes me think TwitchPlaysYourApplication.js would be a better name :)
taybin 13y ago
Now have this play the 2048 game.

132 points by sam152 13y ago

Netflix also has a famous division of monkeys that they unleash to take down servers in the hive, and generally muck about. They want to break as many things as possible while keeping uptime and that sounds freaking awesome! :D

midas007 13y ago
Chaos monkeys meet entropy gremlins. (Watch out for water.)
madeofpalk 13y ago
More info (the open sourced it): http://techblog.netflix.com/2012/07/chaos-monkey-released-into-wild.html http://techblog.netflix.com/2012/07/chaos-monkey-released-in...
- bhhaskin 13y ago
  That is pretty darn cool!

ianbicking 13y ago

A similar library that I wrote: https://github.com/ianb/walkabout.js https://github.com/ianb/walkabout.js

imkevinxu 13y ago

lol I'm gonna call this "Twitch Plays the Internet"

wildpeaks 13y ago
You read my mind, that totally looks like anarchy mode applied to to the web :)

espeed 13y ago

Not to be confused with the Gremlin graph traversal language (https://github.com/tinkerpop/gremlin/wiki https://github.com/tinkerpop/gremlin/wiki) or the JavaScript version of Gremlin.

mitchellh 13y ago

For those who don't know: this is PROBABLY named after Palm OS Gremlins. Palm OS had a debugging tool called "gremlins" that would randomly tap and type and gesture all over your Palm OS app. This debugging tool would often find edge cases that would end up crashing your app. As someone who used to program for Palm OS a lot, I was pretty happy to see this name. :)

Semaphor 13y ago
Or even more probable, they are both simply named after Gremlins [0] [0]: http://www.etymonline.com/index.php?search=gremlin http://www.etymonline.com/index.php?search=gremlin
fzaninotto 13y ago
As the author of this lib, I can confirm: it was named after the movie.
apinstein 13y ago
Back in the very early days of Palm OS, Palm had a "Quality Certification" you could apply for, and if your app passed, you'd get a bunch of marketing benefits. So we went for the highest-level certification on our products, which required surviving 1,000,000 gremlins. At first I was very skeptical that it would find any meaningful bugs, but in fact it was very good at finding memory leaks and edge cases. In the end it took us several weeks to finally be able to survive the 1M gremlins without a crash. A single run of 1M gremlins would take about 24 hours. In later years they added the capability to supply your own dictionary of values to apply to certain inputs so that gremlins could get past validation, and it would also save the entire state of the device every 10K events so that you could easily reproduce errors that Gremlins found. I could definitely see this as a good way to stress-test certain parts of your app, though it takes a ton of time, and frankly without surrounding tooling (like resource monitoring, saving "state") it wouldn't be practical.

presty 13y ago

For Android: http://developer.android.com/tools/help/monkey.html http://developer.android.com/tools/help/monkey.html

quesera 13y ago

The 1984 Macintosh had a similar testing tool called The Monkey. Andy Hertzfeld: http://www.folklore.org/StoryView.py?story=Monkey_Lives.txt http://www.folklore.org/StoryView.py?story=Monkey_Lives.txt

xhrpost 13y ago

I think this is essentially "Fuzz testing"[1]. Something I read about in a book years ago but never saw done in real-life. The term may be antiquated at this point. [1] http://en.wikipedia.org/wiki/Fuzz_testing http://en.wikipedia.org/wiki/Fuzz_testing

dleen 13y ago
Chromium uses fuzzing extensively: http://blog.chromium.org/2012/04/fuzzing-for-security.html http://blog.chromium.org/2012/04/fuzzing-for-security.html
billyhoffman 13y ago
I wouldn't call this fuzz test because it focuses on simulating legitimate actions. Do random clicking. Do random scrolling. Do random mouseovers. Do random [any DOM event]. Yes, they are doing this at a massive scale, but the key here is that germlin.js can only do things a user can do. In short, Gremlin is just automating actions; basic legal, legitimate actions. Still cool, and great for stress testing or looking for race conditions in your UI code, but not fuzz testing. Fuzz testing, on the other hand, is about modifying the data, to interacting with the UI. To use an analogy, fuzz testing Microsoft word might involve corrupting various structures in the DOC or DOCX file or in an OLE embed with malformed data and seeing how the parser/program reacts. The Gremlin.js equivlent would be just clicking on a bunch of buttons in the Word UI really really fast. Both are helpful, but they are testing different things
- yeukhon 13y ago
  Not entirely true. Fuzzing is not about modifying data. I don't think that's the right way to describe fuzzing. It's about testing the durability of a program by trying all kinds of data, as greedy as possible. This includes known problematic inputs and random inputs to match some expectations. By random it can either be totally random (any length, any pattern) or protocol-aware. Random clicking is a form of random data testing because you are trying random input stream to a program. Your argument is not entirely wrong either. By fuzzing his UI he may trigger the browser to crash. He may trigger his monkey to crash. Fuzzing is a very general technique. I can write a fuzzer that fuzz Firefox's UI Australis. What will I look for? Maybe after opening 100 tabs and closing the 45th tab the titlebar disappeared. Or resize the browser from range s to range w I will find some range will cause the UI to look ugly (style overflow, etc). Or after clicking on the scrollbar several times consecutively the browser crashed. Barton Miller, the "Father of Fuzzing" did UI fuzzing by simulating actual keystrokes and mouse clicks. See ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-nt.pdf In his Forwards page, he even mentioned "Monkey" http://pages.cs.wisc.edu/~bart/fuzz/Foreword1.html http://pages.cs.wisc.edu/~bart/fuzz/Foreword1.html

toolslive 13y ago

So the person who named this is old enough to have seen (and liked) http://en.wikipedia.org/wiki/Gremlins http://en.wikipedia.org/wiki/Gremlins I guess 40-something.

minikomi 13y ago
Or older perhaps.. https://www.youtube.com/watch?v=D1xqrdtJs8w https://www.youtube.com/watch?v=D1xqrdtJs8w
reubenmorais 13y ago
The gremlin is older (and broader) than that movie: https://en.wikipedia.org/wiki/Gremlin https://en.wikipedia.org/wiki/Gremlin

fzaninotto 13y ago

Hi! I'm one of the authors of Gremlins.js. Glad to see that some of you find it interesting. I wrote a blog post a while ago about the purpose of the lib, it's called "Completing the JavaScript Test Stack". Worth reading if you wonder what this is all about. http://redotheweb.com/2014/01/07/completing-the-js-test-stack-introducing-greminsjs.html http://redotheweb.com/2014/01/07/completing-the-js-test-stac...

metaedge 13y ago

I just give my laptop or iPad to 2 year old son and it does the same trick!

Houshalter 13y ago

That looks really cool, though I think it would have a hard time navigating past the first few menus, and could hit features that are only rarely meant to be touched (e.g. "delete everything".) Perhaps a searching algorithm that intentionally tries to explore every menu and feature, navigate to them and then test them. Or intentionally tries things it knows are more likely to find bugs.

fzaninotto 13y ago
All the gremlins are configurable, you can specify which elements they can interact on (and therefore exclude "dangerous" elements). Relevant doc: https://github.com/marmelab/gremlins.js#configuring-gremlins https://github.com/marmelab/gremlins.js#configuring-gremlins

atom-morgan 13y ago

The GIF in Github makes me think TwitchPlaysYourApplication.js would be a better name :)

taybin 13y ago

Now have this play the 2048 game.