3 ms·

It depends on the cost to the end user and the cost of failure. At one end of the spectrum you have free-to-play games. You'd have to be insane to implement a

by mantap 6y ago

It depends on the cost to the end user and the cost of failure. At one end of the spectrum you have free-to-play games. You'd have to be insane to implement a high test coverage on that. Nobody cares if Battletoads Royale crashes. On the other end of the spectrum you have safety critical software, financial software, etc, where the costs of failure are very high. Ask yourself "what's the worst that can happen if this fails?". Asking this can also inform where exactly the tests need to be added first.

Shacklz 6y ago
Agreeing with your sentiment, but want to point out that there are free-to-play-games out there where I'd be seriously surprised if they didn't do some very serious testing; they make just too much money and put out too many new features too fast in order to forgo automated testing. Just to name a few I know: Dota2, Path of Exile, League of Legends. Granted, they're probably not the cash-grab-f2p-mobile-game you might have had in mind, but still technically free-to-play.
SamuelAdams 6y ago
Yep. I'm reading a book right now called "Click Here to Kill Everybody [1] and it discusses exactly this. It also considers the financial cost involved. What is the cost of establishing and maintaining a CI/CD system that OP is proposing? And what is the cost of a bug / deployment error? In the last decade (assuming the company has been around that long), how often has a deployment error occurred? How much time saved will occur due to automating deployments? Unless the errors are substantial, or the time savings for staff are significant, it's unlikely that organizations will invest significantly in non-business / non-consumer facing technology. Unless software is their business, then having mature CI/CD systems helps the company attract strong candidates, like OP. [1]: https://www.schneier.com/books/click_here/ https://www.schneier.com/books/click_here/
Cthulhu_ 6y ago
You mention f2p games, but those definitely need testing because they can bring in money that most startups and other companies can only dream of. If their servers go down / if the game becomes unplayable they stand to lose millions. But I guess there's plenty of games that don't have that kind of criticality. Just mentioning a counter-example. (Personally I'm still salty that GTA Online was released for free but the servers couldn't handle it, I mean come on, it's a decade old, you'd think they'd have sorted out scalability and stability by now. That game still earns its publisher hundreds of millions a year).

by mantap 6y ago

Shacklz 6y ago

Agreeing with your sentiment, but want to point out that there are free-to-play-games out there where I'd be seriously surprised if they didn't do some very serious testing; they make just too much money and put out too many new features too fast in order to forgo automated testing. Just to name a few I know: Dota2, Path of Exile, League of Legends. Granted, they're probably not the cash-grab-f2p-mobile-game you might have had in mind, but still technically free-to-play.

SamuelAdams 6y ago

Yep. I'm reading a book right now called "Click Here to Kill Everybody [1] and it discusses exactly this. It also considers the financial cost involved. What is the cost of establishing and maintaining a CI/CD system that OP is proposing? And what is the cost of a bug / deployment error? In the last decade (assuming the company has been around that long), how often has a deployment error occurred? How much time saved will occur due to automating deployments? Unless the errors are substantial, or the time savings for staff are significant, it's unlikely that organizations will invest significantly in non-business / non-consumer facing technology. Unless software is their business, then having mature CI/CD systems helps the company attract strong candidates, like OP. [1]: https://www.schneier.com/books/click_here/ https://www.schneier.com/books/click_here/

Cthulhu_ 6y ago

You mention f2p games, but those definitely need testing because they can bring in money that most startups and other companies can only dream of. If their servers go down / if the game becomes unplayable they stand to lose millions. But I guess there's plenty of games that don't have that kind of criticality. Just mentioning a counter-example. (Personally I'm still salty that GTA Online was released for free but the servers couldn't handle it, I mean come on, it's a decade old, you'd think they'd have sorted out scalability and stability by now. That game still earns its publisher hundreds of millions a year).