8 ms·
>While I’m not a QA or security professional, I have developed a knack for doing “stupid” things with software which causes it to malfunction. A person after m
by pixl97 6y ago
>While I’m not a QA or security professional, I have developed a knack for doing “stupid” things with software which causes it to malfunction.
A person after my own heart.
I've had many a dev go "why would you do that"
In which I answer "it doesn't matter, but if you accept my input it's your job to ensure the app doesn't crash"
- josephg 6y agoMine too. I write a lot of javascript and the string "null" is pretty harmless in most code. But there's all sorts of fun bugs (and often security vulnerabilities) you can find if you make an identifier "__proto__". (If code ever uses that as the key in an object, you're off to the races!)
- steinuil 6y agoNice, I never thought of testing with "__proto__"! "constructor" is also a fun one.
- bhrgunatha 6y agoA favourite technique of one of my colleague's was just to mash the keyboard randomly to see if the app breaks. It's very crude and not at all foolproof. For the lack of sophistication it's shockingly effective at highlighting a huge amount of assumptions we make about how software is / can be used.
- jsrcout 6y agoOne of my professors back in college loved to do that. "Shockingly effective" is exactly the right term. Learned a lot from that little exercise.
- jannes 6y agoThis makes me wonder if there's a tool that can auto-generate random inputs and send them to the application. (maybe excluding Alt-Tab to avoid switching away from the application) Fuzzing is a pretty popular testing technique for libraries, but GUI software has not seen the same attention.
- dgellow 6y agoGUI fuzzing is a thing, yes. I've seen it mostly on mobile apps but I'm sure you have it for other type of GUIs. Example of tools: https://www.fuzzingbook.org/html/GUIFuzzer.html https://www.fuzzingbook.org/html/GUIFuzzer.html
- taneq 6y agoGUI fuzzing was a thing way back on the original Macintosh: https://www.folklore.org/StoryView.py?project=Macintosh&story=Monkey_Lives.txt https://www.folklore.org/StoryView.py?project=Macintosh&stor... Something similar should be reasonably easy to build these days using AutoHotKey or the like. I bet it's been done.
- capableweb 6y agoThis even has a name, "monkey testing". Basically some software that pretends to be a monkey in front of a computer and mashes random buttons and keys to make the application behave badly. Usually you only care about the application not breaking in monkey tests. Can be used similarly to "fuzzing" but for UIs as well, see Gremlins.js: https://marmelab.com/blog/2020/06/02/gremlins-2.html https://marmelab.com/blog/2020/06/02/gremlins-2.html
- chowells 6y agoYou could use a fish, too: https://www.dexerto.com/pokemon/pokemon-fans-pet-fish-discovers-game-breaking-sapphire-bug-years-later-1429506/ https://www.dexerto.com/pokemon/pokemon-fans-pet-fish-discov...
- sethammons 6y agoThis is how, at age 12 or so, I discovered in the old Windows game “Chip’s Challenge” that you could cheat and unlock (nearly all) levels. If you tried ctrl+n (I think), you could advance to the next level, but only if you had beaten the current level or had the pass code for the level. In a bout of frustration, I rando-mashed the keyboard and advanced to the next level, and could then ctrl+n to the last level! I could reproduce the effect, but never worked out the actual combination that unlocked it. Good times :)
- msh 6y agoWhy do you require software to be more resilient than other things? If I pour water in the gastank of my car, it will also fail to drive. Or gas in the sprinkler tank. So the car should somehow prevent the enduser putting the wrong thing in the tank?
- jl6 6y ago“Using it wrong” is usually quite obvious with real-life machinery. Software has many more ways of using it wrong that are very unobvious, that the user has no reason to suspect could be dangerous, and that have never been seen before.
- iforgotpassword 6y agoHolding down the button for your hazard lights should not break your car. Holding down a key should not crash an app.
- mcosta 6y agoHolding down a pedal probably is going to crash your car.
- 3pt14159 6y ago
- rkangel 6y agoWe have an excellent (and big) QA department, but 13 years ago when I started at this company we were only just beginning to hire dedicated testers. We had a mature product which was a communication handset and it worked well and was stable. Our software engineers had pressed every button they could think of in every menu and there weren't any problems. Then we hired Kevin. Kevin had the handset for 40 minutes before piping up "crashed it". The lead comes over to have the sequence explained to her, and says "huh, nice edge case". Half an hour later "crashed it again" (in a completely different way). Explains the sequence to the lead again. An hour later this happens again and he explains the sequence and she finally bursts out "Why would you even do that?! How did you think of pressing those buttons like that with that timing?!!". Good testers just think differently than software engineers.
- thih9 6y ago> Good testers just think differently than software engineers. I'm sure in some cases it's useful to detect all possible crashes, e.g. to make an app as secure as possible. In other cases I'd watch out for diminishing returns; perhaps instead of "think differently than software engineers" it would be enough to "think in the similar way as product users".
- rkangel 6y agoThere's a consideration about number of users. If 100k users are using your product a lot, in a similar way to the 'million monkeys' thing they're accidentally going to find bugs. Software engineers tend to use products in a consistent way (based on how they know it's meant to be used) whereas good testers explore the space of possible inputs in a much more 'creative' way.
- thih9 6y ago> good testers explore the space of possible inputs in a much more 'creative' way. My point is that the extent of testing should depend on the actual product; eliminating all bugs is not the primary goal of every team. Some companies might decide on other goals and prioritize e.g.: just paying users (giving them better support to resolve issues), or acquiring new users, or something else.
- ygra 6y agoFor a long time I've had my full name as user name on my machine, which meant that my user profile path contains both a space (evil) and a non-ASCII character (even more evil, although it's in Latin 1 at least). A lot of things break on one or both of those things and at times it's a bit annoying to deal with. Some bug reports have also been closed as "Won't fix, just don't do that. Who needs spaces in paths, anyway?". I haven't tried to use non-Latin Unicode in my user name and profile directory, which would break everything that uses the old ANSI APIs on Windows instead of the Unicode ones, and that's probably way too much breakage. But broken nonetheless, and that includes a lot of new, recent, and still-maintained software :-/ In a similar vein, I've also used U+2212 as minus sign in my regional settings. There's a lot of software that refuses to parse numbers it previously happily emitted. I've given up on that too, by now, though. The only thing I still do is using English as UI language (so I don't have to deal with bad translations of software), but German as my regional settings (with ISO 8601 dates). There's a lot of software out there (I think GNU gettext is broken in that way on Windows) that assumes that the way I want my dates and numbers formatted has any bearing on the language I want to see in an application. Many others don't care about the regional settings and use the UI language to also format dates, times, and numbers. That's annoying, but at least nothing breaks, so that's the only deviation from the standard user I still use, to still be able to work.
- vsareto 6y ago>In which I answer "it doesn't matter, but if you accept my input it's your job to ensure the app doesn't crash" This is how try/catch alls get added :(