8 ms·

Since I am mentioned in the article I have a correction to make that I think is pretty important. The article says that I like to find ideas by “playtesting” t

by jblow 7y ago

Since I am mentioned in the article I have a correction to make that I think is pretty important. The article says that I like to find ideas by “playtesting” then goes on to describe playtesting as a form of user testing. Whereas there is some definition of the word playtesting that might be applicable to the way I design puzzles, it is not remotely the one in the article. In fact I think too much “playtesting” as described here will make your game boring, and will degrade your skill as a designer, and over time make you a boring designer. It is important to avoid this. If you playtest too much it means you don’t really know what you are making or don’t have confidence to determine what playing it can be like. Even if you don’t have these skills or this confidence yet, you should be working on building them. Certain high skill levels of design will not be accessible otherwise. The amount of playtesting we did on The Witness was very small — about 5 or 6 days over the course of 6.5 years, on a game that takes 50-100 hours to play through. Whereas I do sometimes get puzzle ideas from bottom-up exploration of a space by myself (which is the definition of “playtesting” that would make sense here), in fact the vast majority of my puzzle ideas just spring to mind after I have built a sufficient understanding of the possibility space. So these are more like two distinct phases of design, and you might switch between them when designing one game. But I think most of the actual puzzle ideas have to come at least semi-intentionally, or the game will not feel very strong. You have to know where you are trying to drive the truck.

doctorpangloss 7y ago
This is a really insightful point, design wouldn't be learnable if it always relied on discovery through testing. Compare to giant companies like Twitter, where I have heard engineering & product managers say "Every idea has been thought of, it's only a matter of testing them one after another." What an intellectually bankrupt idea. It is the most experienced product designers who can just look at something and tell you if it is good or bad, before trying it, that big corp product undermines the most.
- chii 7y ago
  > It is the most experienced product designers who can just look at something and tell you if it is good or bad, before trying it, that big corp product undermines the most. that's because for every product designer that can tell good from bad, there are 10's of pretenders who dont know what they're talking about. The stakeholders don't have the time or expertise to vet (otherwise, they'd be product designers themselves!). Thus, they need objective verification. This is what turns into the 'matter of testing the idea'.
- mntmoss 7y ago
  I call it the method of creating self-feedback. Everyone starts off by using hunches, "taste", and the opinion of the crowd or authorities. But to get beyond that - and getting beyond playtesting, data-collection and other brute force feedback mechanisms entails doing so - one has to systematically devise original feedback signals - pass-fail checklists, writers' rubrics, dependency graphs, workout logs and so on. Once you have the framework in hand and have validated its philosophical premise to some degree, you can feed your work and ideas through it and it automatically corrects you and guides the ideas away from major fallacies. Of course, some amount of experimental data helps, but the huge amounts we have grown accustomed to have a habit of producing a lot of busywork - a product tuned and tweaked and prodded to improve this or that short-term KPI or audience appeal metric - without striking at the critical issues that one can probe with a bit of followthrough on self-feedback.
- lonelappde 7y ago
  It maybe a worldwide communications ecosystem is a different problems space from a throwaway videogame.
codesushi42 7y ago
I had to do a double take on your name. You are a genius and a god among us. Thank you for making the deepest and most beautiful games I have ever enjoyed.
- codesushi42 7y ago
  I don't know why I am being downvoted. Braid and The Witness are two of the best games I have ever played. And I've been playing games for 30 years now. Your tech talks are also awesome.
  - jessermeyer 7y ago
    Flattery, in these circles, is not seen as continuing the conversation forward. I also imagine that those who are predisposed to dislike Jon will downvote you out of spite.
aidos 7y ago
Thanks for The Witness. I don’t play games at all anymore, and haven’t for years but I’m really glad I took the time to work through The Witness. It’s an absolute work of art. (as was Braid - literally the only other game I’ve played in years)
- chii 7y ago
  i've been told that 'Outer Wilds' is also a good game. (not to be confused with 'Outer Worlds', which is not a great one).
  - baud147258 7y ago
    Outer Worlds isn't yet released, it will be available on October 25.
chriskok 7y ago
Hi Jonathan, I'm the author of the article and I definitely agree with you. At the time of my writing of this article I had just begun creating puzzle games myself and I just tried to pull together all I could find on the topic. I stepped away from all this for awhile now to actually create instead of just doing research and found that many of my initial ideas were flawed. What you mentioned is incredibly insightful and I hope you'd allow me to directly quote you and edit my post so that others who visit will be able to better understand the whole scary but wonderful process of puzzle game design. Goes without saying, but I am a huge fan of your work and you as a person so I'm incredibly honored by your response. That said, I hope that I have not mistakenly misdirected the audience by putting words into your mouth in the form of advice. Please let me know if there are other portions of this that you find incorrect. Either way, thank you for all you've done!
- Liquidor 7y ago
  Hey Chris. Love the posts. Lots of insight. A bit offtopic, but could you remove the fixed widths of your #content (.baby-container) at screen width below 767px ? 300 pixels is just too narrow for the content. Maybe set width to auto and set margin to 15px for a better responsive experience? :)
  - chriskok 7y ago
    Yes! I am pretty inexperienced with web dev so I just changed the theme entirely but I really hope its better for anyone on their mobile phones now. Thanks for bringing it to my attention!
- theNJR 7y ago
  This thread is why Hacker News is one of the best sites online today.
greggman2 7y ago
And on the opposite side you have Nintendo, known for some of the most beloved games in the world. They have the "Mario Club" and playtest extensively. They test often and continuously. They can test every day if they want. Adjust the jump height 10%, ask the Mario Club if the game got better. In this way they iterate on game play far more than just about any other game company I know of. The biggest benefit is they get feedback early. Most game devs wait until their game is "presentable" which basically means they've already spent too much time making what they've made that they can't change anything except minor tweaks. Nintendo gets feedback early and often.
- OnlineGladiator 7y ago
  Nintendo's games are intentionally meant to address as large as audience as possible though. While I'm sure Jonathan wants to sell a lot of games, I think he cares more about making it great for a smaller subset than making it palatable to a larger subset. To put it another way, Nintendo's games are incredibly bland compared to something like The Witness. I think Nintendo makes great games, but they're not nearly as unique an experience as you'll find from some indie developers.
  - greggman2 7y ago
    Me thinks maybe you haven't actually played Nintendo games if you believe they are bland compared to a game with 650 repetitive puzzles that would have been just as interesting (or non-interesting) as a 2D game on a phone. Yes, I get there is a context in the game and a big reveal (and I enjoyed the game) but I don't agree that 650 puzzles is less bland than my time spent in Breath of the Wild. I also don't agree that "Indie" = "better" or "unique". AAA companies, especially Sony and Nintendo have released amazingly creative games. Just a few from Nintendo include Pikman, Animal Crossing, 1-2 Switch (cow milking), Cubivore, and there are several others. Meanwhile 99% of "indie" games are skinned versions of old games who's best examples are often Nintendo's. How many "Metroid"vanias are there? How many Smash Bros clones? As one more final point, the games made at big companies are made by actual people who poor their creativity into the games they make. Indies don't have a monopoly on creativity. Being AAA != can't be creative nor does it mean can't take risks on new ideas. There are plenty of examples of new ideas from these companies.
totemandtoken 7y ago
I started to read this and thought to myself "This sounds like Jonathan Blow" and then you mentioned the Witness and I finally tore my eyes away from the text to the username, and lo and behold! Quick question - how do you gain this understanding of the possibility space without bottom-up exploration? Is it a matter of having good influences (and who might those be)? Or some other reason?
- jblow 7y ago
  I think it’s just, after you have designed enough things, you learn to think like a system and you know what a lot of the possibilities are before you play those specific corners of the system. That said, you can always be surprised, and it’s delightful when this happens, so you don’t want to get on an egotistical high horse and decide you really know the system without playing it. The actual experience of the actual system is your reality check. But, the more experience you have with that system, the more you can see where things might go (unless it’s intensely chaotic, thus unplayable).
pengstrom 7y ago
Very true. The Witness is in my top one PC games, precisely because everything seems to be there for a reason. It's extremely well done!
deleted 7y ago
[deleted]
Elyot 7y ago
> I think most of the actual puzzle ideas have to come at least semi-intentionally, or the game will not feel very strong. This right here is a really underrated statement. It's why I think the designers behind WPC-style paper puzzles (Toketa, gmpuzzles, etc.) consistently produce works of higher quality than most levels in most puzzle video games. Working on paper forces you to design with intention rather than design by playing around with objects on a grid. Intentionality is one of the most important aspects of good puzzle design. Reminds me of a 2001 quote from Nikoli editor Nobuhiko Kanamoto: Computer-generated Sudoku puzzles are lacking a vital ingredient that makes puzzles enjoyable - the sense of communication between solver and author.
yyyk 7y ago
Zach Barth from Zachrtonics famously never solved the final SpaceChem level. IIRC the interviews, it (and some other levels in that game) was done by inspiration, and playtesting was 'check out if this is reasonably solvable'.
- AstralStorm 7y ago
  Indeed, while I found Witness difficulty very uneven, with lack of episodic structure like many other games. It lacks guidance so you stumble around until you find things - but it's not actually truly open world as there are many knowledge gates. Which you will regularly bounce off. That does not make it worse than other puzzle games, just annoying in parts. Especially its huge reliance on perception as opposed to actually thinking and figuring things out, but then it's not quite a puzzle game. It's more of a big escape room game which does have puzzles in it. There are others on the market, few are well known. (Even something like Antichamber is more of a puzzle game than Witness, yet it is still an escape room rather than a pure puzzle game.) A relevant contrasting comparison would be to something like The Talos Principle which was heavily playtested including some puzzle redesigns, reordering, some having been dropped. Puzzles were also automatically checked to be doable in desired and a few atypical ways. The other design principles apply though the same - minimalism, tutorial principles and difficulty scaling. Antichamber has the guns and blocks and physics, Talos has the few tools and pseudophysics, Witness has varying puzzle mechanics but one input with tons of output. The difference is the degree of things being hidden. Puzzles with hidden pieces tend to become adventure games. It does not matter if you have to carry the pieces or if they're already in place.
  - jessermeyer 7y ago
    > just annoying in parts. Especially its huge reliance on perception I don't understand this attitude for a game that is called The Witness.

by jblow 7y ago

doctorpangloss 7y ago

This is a really insightful point, design wouldn't be learnable if it always relied on discovery through testing. Compare to giant companies like Twitter, where I have heard engineering & product managers say "Every idea has been thought of, it's only a matter of testing them one after another." What an intellectually bankrupt idea. It is the most experienced product designers who can just look at something and tell you if it is good or bad, before trying it, that big corp product undermines the most.

chii 7y ago
> It is the most experienced product designers who can just look at something and tell you if it is good or bad, before trying it, that big corp product undermines the most. that's because for every product designer that can tell good from bad, there are 10's of pretenders who dont know what they're talking about. The stakeholders don't have the time or expertise to vet (otherwise, they'd be product designers themselves!). Thus, they need objective verification. This is what turns into the 'matter of testing the idea'.
mntmoss 7y ago
I call it the method of creating self-feedback. Everyone starts off by using hunches, "taste", and the opinion of the crowd or authorities. But to get beyond that - and getting beyond playtesting, data-collection and other brute force feedback mechanisms entails doing so - one has to systematically devise original feedback signals - pass-fail checklists, writers' rubrics, dependency graphs, workout logs and so on. Once you have the framework in hand and have validated its philosophical premise to some degree, you can feed your work and ideas through it and it automatically corrects you and guides the ideas away from major fallacies. Of course, some amount of experimental data helps, but the huge amounts we have grown accustomed to have a habit of producing a lot of busywork - a product tuned and tweaked and prodded to improve this or that short-term KPI or audience appeal metric - without striking at the critical issues that one can probe with a bit of followthrough on self-feedback.
lonelappde 7y ago
It maybe a worldwide communications ecosystem is a different problems space from a throwaway videogame.

codesushi42 7y ago

I had to do a double take on your name. You are a genius and a god among us. Thank you for making the deepest and most beautiful games I have ever enjoyed.

codesushi42 7y ago
I don't know why I am being downvoted. Braid and The Witness are two of the best games I have ever played. And I've been playing games for 30 years now. Your tech talks are also awesome.
- jessermeyer 7y ago
  Flattery, in these circles, is not seen as continuing the conversation forward. I also imagine that those who are predisposed to dislike Jon will downvote you out of spite.

aidos 7y ago

Thanks for The Witness. I don’t play games at all anymore, and haven’t for years but I’m really glad I took the time to work through The Witness. It’s an absolute work of art. (as was Braid - literally the only other game I’ve played in years)

chii 7y ago
i've been told that 'Outer Wilds' is also a good game. (not to be confused with 'Outer Worlds', which is not a great one).
- baud147258 7y ago
  Outer Worlds isn't yet released, it will be available on October 25.

chriskok 7y ago

Hi Jonathan, I'm the author of the article and I definitely agree with you. At the time of my writing of this article I had just begun creating puzzle games myself and I just tried to pull together all I could find on the topic. I stepped away from all this for awhile now to actually create instead of just doing research and found that many of my initial ideas were flawed. What you mentioned is incredibly insightful and I hope you'd allow me to directly quote you and edit my post so that others who visit will be able to better understand the whole scary but wonderful process of puzzle game design. Goes without saying, but I am a huge fan of your work and you as a person so I'm incredibly honored by your response. That said, I hope that I have not mistakenly misdirected the audience by putting words into your mouth in the form of advice. Please let me know if there are other portions of this that you find incorrect. Either way, thank you for all you've done!

Liquidor 7y ago
Hey Chris. Love the posts. Lots of insight. A bit offtopic, but could you remove the fixed widths of your #content (.baby-container) at screen width below 767px ? 300 pixels is just too narrow for the content. Maybe set width to auto and set margin to 15px for a better responsive experience? :)
- chriskok 7y ago
  Yes! I am pretty inexperienced with web dev so I just changed the theme entirely but I really hope its better for anyone on their mobile phones now. Thanks for bringing it to my attention!
theNJR 7y ago
This thread is why Hacker News is one of the best sites online today.

greggman2 7y ago

And on the opposite side you have Nintendo, known for some of the most beloved games in the world. They have the "Mario Club" and playtest extensively. They test often and continuously. They can test every day if they want. Adjust the jump height 10%, ask the Mario Club if the game got better. In this way they iterate on game play far more than just about any other game company I know of. The biggest benefit is they get feedback early. Most game devs wait until their game is "presentable" which basically means they've already spent too much time making what they've made that they can't change anything except minor tweaks. Nintendo gets feedback early and often.

OnlineGladiator 7y ago
Nintendo's games are intentionally meant to address as large as audience as possible though. While I'm sure Jonathan wants to sell a lot of games, I think he cares more about making it great for a smaller subset than making it palatable to a larger subset. To put it another way, Nintendo's games are incredibly bland compared to something like The Witness. I think Nintendo makes great games, but they're not nearly as unique an experience as you'll find from some indie developers.
- greggman2 7y ago
  Me thinks maybe you haven't actually played Nintendo games if you believe they are bland compared to a game with 650 repetitive puzzles that would have been just as interesting (or non-interesting) as a 2D game on a phone. Yes, I get there is a context in the game and a big reveal (and I enjoyed the game) but I don't agree that 650 puzzles is less bland than my time spent in Breath of the Wild. I also don't agree that "Indie" = "better" or "unique". AAA companies, especially Sony and Nintendo have released amazingly creative games. Just a few from Nintendo include Pikman, Animal Crossing, 1-2 Switch (cow milking), Cubivore, and there are several others. Meanwhile 99% of "indie" games are skinned versions of old games who's best examples are often Nintendo's. How many "Metroid"vanias are there? How many Smash Bros clones? As one more final point, the games made at big companies are made by actual people who poor their creativity into the games they make. Indies don't have a monopoly on creativity. Being AAA != can't be creative nor does it mean can't take risks on new ideas. There are plenty of examples of new ideas from these companies.

totemandtoken 7y ago

I started to read this and thought to myself "This sounds like Jonathan Blow" and then you mentioned the Witness and I finally tore my eyes away from the text to the username, and lo and behold! Quick question - how do you gain this understanding of the possibility space without bottom-up exploration? Is it a matter of having good influences (and who might those be)? Or some other reason?

jblow 7y ago
I think it’s just, after you have designed enough things, you learn to think like a system and you know what a lot of the possibilities are before you play those specific corners of the system. That said, you can always be surprised, and it’s delightful when this happens, so you don’t want to get on an egotistical high horse and decide you really know the system without playing it. The actual experience of the actual system is your reality check. But, the more experience you have with that system, the more you can see where things might go (unless it’s intensely chaotic, thus unplayable).

pengstrom 7y ago

Very true. The Witness is in my top one PC games, precisely because everything seems to be there for a reason. It's extremely well done!

deleted 7y ago

[deleted]

Elyot 7y ago

> I think most of the actual puzzle ideas have to come at least semi-intentionally, or the game will not feel very strong. This right here is a really underrated statement. It's why I think the designers behind WPC-style paper puzzles (Toketa, gmpuzzles, etc.) consistently produce works of higher quality than most levels in most puzzle video games. Working on paper forces you to design with intention rather than design by playing around with objects on a grid. Intentionality is one of the most important aspects of good puzzle design. Reminds me of a 2001 quote from Nikoli editor Nobuhiko Kanamoto: Computer-generated Sudoku puzzles are lacking a vital ingredient that makes puzzles enjoyable - the sense of communication between solver and author.

yyyk 7y ago

Zach Barth from Zachrtonics famously never solved the final SpaceChem level. IIRC the interviews, it (and some other levels in that game) was done by inspiration, and playtesting was 'check out if this is reasonably solvable'.

AstralStorm 7y ago
Indeed, while I found Witness difficulty very uneven, with lack of episodic structure like many other games. It lacks guidance so you stumble around until you find things - but it's not actually truly open world as there are many knowledge gates. Which you will regularly bounce off. That does not make it worse than other puzzle games, just annoying in parts. Especially its huge reliance on perception as opposed to actually thinking and figuring things out, but then it's not quite a puzzle game. It's more of a big escape room game which does have puzzles in it. There are others on the market, few are well known. (Even something like Antichamber is more of a puzzle game than Witness, yet it is still an escape room rather than a pure puzzle game.) A relevant contrasting comparison would be to something like The Talos Principle which was heavily playtested including some puzzle redesigns, reordering, some having been dropped. Puzzles were also automatically checked to be doable in desired and a few atypical ways. The other design principles apply though the same - minimalism, tutorial principles and difficulty scaling. Antichamber has the guns and blocks and physics, Talos has the few tools and pseudophysics, Witness has varying puzzle mechanics but one input with tons of output. The difference is the degree of things being hidden. Puzzles with hidden pieces tend to become adventure games. It does not matter if you have to carry the pieces or if they're already in place.
- jessermeyer 7y ago
  > just annoying in parts. Especially its huge reliance on perception I don't understand this attitude for a game that is called The Witness.