3 ms·

> At the moment the vast majority of negative feedback on HN has been "it sucked" without giving a hint of what actually went wrong. That's the nature of the

> At the moment the vast majority of negative feedback on HN has been "it sucked" without giving a hint of what actually went wrong. That's the nature of the HN beast: getting all the "it sucked" comments without STR and/or use-case descriptions is certainly demoralizing (reading HN comments about your own work is not for the unprepared) but asking folks who've had a bad time with your product to spend more time, for free, going through that same experience again so they can do a detailed report-out for you is probably not going to get a lot of takers (even if you might get a few high quality reports). If you're not already doing so, I would strongly recommend you invest some money in getting access to "people who've never used Matrix before but are familiar with messaging apps" to do user testing (there's a number of sites for doing that for you, where you hand over money and instructions, and they find folks to do unbiased testing for you) where you ask pairs of people to get started with your software, with the explicit goal of getting to the point where they can set up E2EE messaging, and reviewing the recordings to see what roadblocks they run into. And for bonus points, dev-blog about those experiences on the matrix/element site (where did folks get stuck? was this a surprise to you? why? how did you fix it? what were the user reactions to the fix? how much better is Matrix now, thanks to this work?).

ricw 6y ago
I couldn’t emphasise this more. Stick to the YC mantra: always talk to users. Ask them what they want or have problems with, quantitatively (questionnaire style) and qualitatively (talk to them in person, real user testing). Added a new feature? Don’t assume it’s well designed. Assume something is wrong with it and that it can and should be further optimised and sweat the details. Details matter. Unless you sweat the details, users will notice.
eredengrin 6y ago
> I would strongly recommend you invest some money in getting access to "people who've never used Matrix before but are familiar with messaging apps" to do user testing I think they already do quite a bit of this, which is probably what makes it puzzling when tech oriented users on hn keep reporting how terrible it is, thus the desire for more details to see whether it's just an older anecdote, something missed in testing, etc. Two matrix blog posts[1][2] from the past 2 months both mention new user testing/research, and there's definitely more if you go back in their blog history. [1] https://matrix.org/blog/2020/12/17/matrix-to-reloaded#whats-new https://matrix.org/blog/2020/12/17/matrix-to-reloaded#whats-... [2] https://matrix.org/blog/2020/11/27/this-week-in-matrix-2020-11-27#element https://matrix.org/blog/2020/11/27/this-week-in-matrix-2020-...

ricw 6y ago

I couldn’t emphasise this more. Stick to the YC mantra: always talk to users. Ask them what they want or have problems with, quantitatively (questionnaire style) and qualitatively (talk to them in person, real user testing). Added a new feature? Don’t assume it’s well designed. Assume something is wrong with it and that it can and should be further optimised and sweat the details. Details matter. Unless you sweat the details, users will notice.

eredengrin 6y ago

> I would strongly recommend you invest some money in getting access to "people who've never used Matrix before but are familiar with messaging apps" to do user testing I think they already do quite a bit of this, which is probably what makes it puzzling when tech oriented users on hn keep reporting how terrible it is, thus the desire for more details to see whether it's just an older anecdote, something missed in testing, etc. Two matrix blog posts[1][2] from the past 2 months both mention new user testing/research, and there's definitely more if you go back in their blog history. [1] https://matrix.org/blog/2020/12/17/matrix-to-reloaded#whats-new https://matrix.org/blog/2020/12/17/matrix-to-reloaded#whats-... [2] https://matrix.org/blog/2020/11/27/this-week-in-matrix-2020-11-27#element https://matrix.org/blog/2020/11/27/this-week-in-matrix-2020-...