3 ms·

In the past, I've done an invite-only beta to limit the exposure of issues with my code while in a QA-phase (minimize embarrassment) and to retain control over

by mjolk 14y ago

In the past, I've done an invite-only beta to limit the exposure of issues with my code while in a QA-phase (minimize embarrassment) and to retain control over the load/expense of hosting. I suppose that others could be going with the "keep a line to insinuate exclusivity" approach.

bmelton 14y ago
At the risk of making a very elaborate "this" post, I agree with all above points. I've mainly used invitation-only to minimize load or at least to get a better idea of where the pinch points were in where scaling is going to be tricky. It's nice to be able to say "We're operating just fine with 100 logged in users. What happens if we have 1,000 logged in users?" and be able to control that to some degree. Giving each user a couple of invites is going to grow the population (you hope) but not open up the firehose completely. If you don't know how you're going to scale to 1,000 users, you might not want a million users just yet.
- youngdev 14y ago
  How about using Open-Source tools such as Jmeter to run performance test and identify the bottle neck areas. We have done that with our site http://www.jackpotbuddy.com http://www.jackpotbuddy.com. Initially were were getting around 25 seconds page load time for 1000 users. We were able to reduce that to <5 seconds after identifying and fixing the issues
  - bmelton 14y ago
    That's probably a really good suggestion, I can't completely say. In my experience, load-generating tools give completely differing results than actual user patterns, but it's been awhile since I've used any with any real expectation. Obviously it's great to test more obvious things, but I would use something like that before real user testing, not in lieu of. How have you seen JMeter to compare in this? I haven't ever used it directly.

3 ms·

In the past, I've done an invite-only beta to limit the exposure of issues with my code while in a QA-phase (minimize embarrassment) and to retain control over

by mjolk 14y ago

bmelton 14y ago
At the risk of making a very elaborate "this" post, I agree with all above points. I've mainly used invitation-only to minimize load or at least to get a better idea of where the pinch points were in where scaling is going to be tricky. It's nice to be able to say "We're operating just fine with 100 logged in users. What happens if we have 1,000 logged in users?" and be able to control that to some degree. Giving each user a couple of invites is going to grow the population (you hope) but not open up the firehose completely. If you don't know how you're going to scale to 1,000 users, you might not want a million users just yet.
- youngdev 14y ago
  How about using Open-Source tools such as Jmeter to run performance test and identify the bottle neck areas. We have done that with our site http://www.jackpotbuddy.com http://www.jackpotbuddy.com. Initially were were getting around 25 seconds page load time for 1000 users. We were able to reduce that to <5 seconds after identifying and fixing the issues
  - bmelton 14y ago
    That's probably a really good suggestion, I can't completely say. In my experience, load-generating tools give completely differing results than actual user patterns, but it's been awhile since I've used any with any real expectation. Obviously it's great to test more obvious things, but I would use something like that before real user testing, not in lieu of. How have you seen JMeter to compare in this? I haven't ever used it directly.