5 ms·

Solo dev for an app with hundreds of thousands of users. These are the things I do that ensure smooth releases: • run a beta test group. These users consist mo

Solo dev for an app with hundreds of thousands of users. These are the things I do that ensure smooth releases: • run a beta test group. These users consist mostly of users that have emailed in with feature suggestions or bugs. I ask them if they’d like to join the beta (e.g. they can check their bug is fixed). I frequently get great feedback from a few of these users. • fiverr.com. Shell out $100 spread across a range of testers, and give them all slightly different briefings. Some feedback is terrible but there’s some gold there that’s really worth far more than $100. • learn how to test your own product. It’s hard to do well, but you can learn. It’s a valuable skill that can really set you apart in a regular engineering job too. • used phased releases. Pause the release at 1%, give it a few days for any minor issues to roll in and fix these before releasing to 100%.

WaltPurvis 8y ago
> used phased releases. Pause the release at 1% Could you expound on this? How do you do that? (Related: How are you distributing? Through an app store?)
- faizmokhtar 8y ago
  There's an option to do this on both Google Play and App Store. It's called staged rollouts[1] in Google Play and phased release[2] for App Store. [1]: https://support.google.com/googleplay/android-developer/answer/6346149?hl=en https://support.google.com/googleplay/android-developer/answ... [2]: https://stackoverflow.com/questions/45073264/what-happens-when-i-do-an-itunes-connect-phased-release https://stackoverflow.com/questions/45073264/what-happens-wh...
hnrodey 8y ago
This sounds like very actionable advice. Thanks for sharing the info, it's very helpful.
startupdiscuss 8y ago
Solo dev and 100ks of users? Wow. What is your tech stack? What is your code-deploy process?
- Lord_Zero 8y ago
  Probably a mobile app, and probably deployed via the platform tools. They have options for beta releases which users can opt-in themselves, and also phased releases are supported out of the box. Just my guess.
- travbrack 8y ago
  It's probably his mobile app called Class Timetable https://news.ycombinator.com/item?id=14542045 https://news.ycombinator.com/item?id=14542045
feelix 8y ago
Also a "Solo dev for an app with hundreds of thousands of users" I just test everything myself. On one OS (Mac). The worst part is having to run VMWare with older OS versions sometimes, but not usually. I find that bugs are almost never OS specific, and are rather just logic errors. I find that having other people test my code adds little value, as I can literally just test it myself in the OS that I write it in most the time. I know that this goes against all advice that you read, and all the "common knowledge" that is out there... but I've been doing this for 15 years and have got over 1 million users in total probably, and I almost never get support emails regarding bug reports. So, in my experience, I really find that everything I have heard is wrong. I realize that other people may have other experiences but found this point of reference interesting.
- j45 8y ago
  Experience building software can lower the testing requirements. I find the degree of testing needed can be relative to the difficulty of the problem, the unnecessary complexity of the solution, and the average skill of the developers working on all 3. Clever (simple) architecture will often beat clever programming in the long run.
  - newsbinator 8y ago
    > Experience building software can lower the testing requirements. This is a good point. The longer I've been at this, the less often, "this is likely to work" is a thought that I let go unchallenged.
    - retbull 8y ago
      "This is totally going to work" is only said when I am completely sure that it is going to fail catastrophically. If anything works I feel mild distrust with my code from that point forward.
      - j45 8y ago
        Knowing the difference between thinking less about yourself vs less of yourself is important when taking your work seriously but not taking yourself too seriously. I have a healthy doubt of my own work that seems to end up making it work well first by keeping it as simple as possible. Keeping the mindset of an innocent beginner is critical as a developer with experience. Everything is easy in the beginning, not the long term - I like to see what code bases people have worked on for many years instead of being able to neatly start from scratch.
chejazi 8y ago
> fiverr.com. Shell out $100 spread across a range of testers, and give them all slightly different briefings. Some feedback is terrible but there’s some gold there that’s really worth far more than $100. Also check out Cent (https://beta.cent.co https://beta.cent.co) - $20 in Ether can get you a pretty good set of responses. I am a cofounder - feel free to ama here or at cameron@cent.co
- theyinwhy 8y ago
  So you want most of the work to happen for free? I really hope your product fails. This is even worse than mechanical turk.
  - chejazi 8y ago
    The way Cent works is instead of having 100 people earn 20¢ (for a $20 bounty), $20 gets distributed to the top 10 responses. Respondents are encouraged to be thoughtful because the community votes on which 10 responses get the bounty. Many responses don't get paid as a result but making it competitive adds more value for the bounty-asker and makes the whole thing more fun in general. What about this do you find so terrible?
    - theyinwhy 8y ago
      Great gain for the bounty-asker, great loss for everyone > 10. Big win-lose gap. Just plain exploitation of the workforce. I don't see the fun in that.

WaltPurvis 8y ago

> used phased releases. Pause the release at 1% Could you expound on this? How do you do that? (Related: How are you distributing? Through an app store?)

faizmokhtar 8y ago
There's an option to do this on both Google Play and App Store. It's called staged rollouts[1] in Google Play and phased release[2] for App Store. [1]: https://support.google.com/googleplay/android-developer/answer/6346149?hl=en https://support.google.com/googleplay/android-developer/answ... [2]: https://stackoverflow.com/questions/45073264/what-happens-when-i-do-an-itunes-connect-phased-release https://stackoverflow.com/questions/45073264/what-happens-wh...

hnrodey 8y ago

This sounds like very actionable advice. Thanks for sharing the info, it's very helpful.

startupdiscuss 8y ago

Solo dev and 100ks of users? Wow. What is your tech stack? What is your code-deploy process?

Lord_Zero 8y ago
Probably a mobile app, and probably deployed via the platform tools. They have options for beta releases which users can opt-in themselves, and also phased releases are supported out of the box. Just my guess.
travbrack 8y ago
It's probably his mobile app called Class Timetable https://news.ycombinator.com/item?id=14542045 https://news.ycombinator.com/item?id=14542045

feelix 8y ago

Also a "Solo dev for an app with hundreds of thousands of users" I just test everything myself. On one OS (Mac). The worst part is having to run VMWare with older OS versions sometimes, but not usually. I find that bugs are almost never OS specific, and are rather just logic errors. I find that having other people test my code adds little value, as I can literally just test it myself in the OS that I write it in most the time. I know that this goes against all advice that you read, and all the "common knowledge" that is out there... but I've been doing this for 15 years and have got over 1 million users in total probably, and I almost never get support emails regarding bug reports. So, in my experience, I really find that everything I have heard is wrong. I realize that other people may have other experiences but found this point of reference interesting.

j45 8y ago
Experience building software can lower the testing requirements. I find the degree of testing needed can be relative to the difficulty of the problem, the unnecessary complexity of the solution, and the average skill of the developers working on all 3. Clever (simple) architecture will often beat clever programming in the long run.
- newsbinator 8y ago
  > Experience building software can lower the testing requirements. This is a good point. The longer I've been at this, the less often, "this is likely to work" is a thought that I let go unchallenged.
  - retbull 8y ago
    "This is totally going to work" is only said when I am completely sure that it is going to fail catastrophically. If anything works I feel mild distrust with my code from that point forward.
    - j45 8y ago
      Knowing the difference between thinking less about yourself vs less of yourself is important when taking your work seriously but not taking yourself too seriously. I have a healthy doubt of my own work that seems to end up making it work well first by keeping it as simple as possible. Keeping the mindset of an innocent beginner is critical as a developer with experience. Everything is easy in the beginning, not the long term - I like to see what code bases people have worked on for many years instead of being able to neatly start from scratch.

chejazi 8y ago

> fiverr.com. Shell out $100 spread across a range of testers, and give them all slightly different briefings. Some feedback is terrible but there’s some gold there that’s really worth far more than $100. Also check out Cent (https://beta.cent.co https://beta.cent.co) - $20 in Ether can get you a pretty good set of responses. I am a cofounder - feel free to ama here or at cameron@cent.co

theyinwhy 8y ago
So you want most of the work to happen for free? I really hope your product fails. This is even worse than mechanical turk.
- chejazi 8y ago
  The way Cent works is instead of having 100 people earn 20¢ (for a $20 bounty), $20 gets distributed to the top 10 responses. Respondents are encouraged to be thoughtful because the community votes on which 10 responses get the bounty. Many responses don't get paid as a result but making it competitive adds more value for the bounty-asker and makes the whole thing more fun in general. What about this do you find so terrible?
  - theyinwhy 8y ago
    Great gain for the bounty-asker, great loss for everyone > 10. Big win-lose gap. Just plain exploitation of the workforce. I don't see the fun in that.