4 ms·

Show HN: A/B testing for iOS and Android

119 points by ericflo 14y ago

daveambrose 14y ago
just registered for beta access! great stuff guys.
michaelmior 14y ago
My only concern here is that the delay in pushing things out to various app stores makes iterating on A/B tests fairly difficult. However, it seems like using data-driven tests, you can hopefully pick and serve the winner without having to push out an app update.
kizza 14y ago
I assume that if you're releasing this for Android, that Clutch for Android is not far away?
- ericflo 14y ago
  Stay tuned :)
ericflo 14y ago
We launched Clutch.io about a month ago, and one thing that really surprised us was how many of our customers asked us to support A/B testing. So that's what we've been building! You don't have to use our Clutch Framework in order to use this new A/B testing stuff, and this is all 100% native. (But if you do use our framework, it gets even more powerful because you can add new tests instantly.) Really looking forward to your feedback, HN!
- dave1619 14y ago
  Just some quick feedback about your pricing. I think you need to accommodate your pricing model better for the AppStore where many apps have millions of users, especially free apps.
  - ericflo 14y ago
    Thanks for that feedback. After our initial Clutch.io launch, we got tons of feedback about our pricing. We've since made every plan more affordable for everyone, and we'll continue to listen to feedback about our pricing, making changes where they're needed. If you have some any more feedback (or if you have an app that falls in that category) I'd love to talk more in depth, please e-mail us at founders@clutch.io.
    - dave1619 14y ago
      Awesome, thanks. Much appreciated.
pheelicks 14y ago
How do I go about testing each branch of the code when developing? Is there some nice way in the web console to manually set this for the test device?
- ericflo 14y ago
  Yes, there's a form on the web dashboard that lets you choose a certain version for your development device.
zashapiro 14y ago
I'm so excited to give this a spin. Something simple and turnkey was absolutely needed. Nice work guys!
aresant 14y ago
Slightly off-topic question - is there a way to A/B test App Store descriptions / pictures / etc?
- fourneau 14y ago
  Unfortunately not. You can only upload 5 images per device type (iPad / iPhone) and one description. In theory, you can play the language game and change your descriptions radically based on language, however that's not exactly a valid experiment.
nixarn 14y ago
Looks really nice. Couple questions though. 1. What happens if you shut down your service? Will the apps function properly? 2. Any idea how much this will cost? I wonder because it would be possible to implement A/B testing without a problem using Flurry, and that would be completely free.
- ericflo 14y ago
  1. Apps will function properly. It's been a design tenant from the beginning to never have an outage (or shut downage) by us negatively impact our customer's users. 2. We haven't 100% decided yet, but we're heavily leaning towards packaging this service in with our other offering, at the same price point, so you get access to both at once: https://clutch.io/pricing/ https://clutch.io/pricing/
  - nixarn 14y ago
    Thanks for the answer! The pricing on clutch.io's homepage looks good, I was worried it would get a pricing like parse.com that is in my opinion quite expensive per API call.
deleted 14y ago
[deleted]

119 points by ericflo 14y ago

just registered for beta access! great stuff guys.

My only concern here is that the delay in pushing things out to various app stores makes iterating on A/B tests fairly difficult. However, it seems like using data-driven tests, you can hopefully pick and serve the winner without having to push out an app update.

kizza 14y ago

I assume that if you're releasing this for Android, that Clutch for Android is not far away?

ericflo 14y ago
Stay tuned :)

ericflo 14y ago

We launched Clutch.io about a month ago, and one thing that really surprised us was how many of our customers asked us to support A/B testing. So that's what we've been building! You don't have to use our Clutch Framework in order to use this new A/B testing stuff, and this is all 100% native. (But if you do use our framework, it gets even more powerful because you can add new tests instantly.) Really looking forward to your feedback, HN!

dave1619 14y ago
Just some quick feedback about your pricing. I think you need to accommodate your pricing model better for the AppStore where many apps have millions of users, especially free apps.
- ericflo 14y ago
  Thanks for that feedback. After our initial Clutch.io launch, we got tons of feedback about our pricing. We've since made every plan more affordable for everyone, and we'll continue to listen to feedback about our pricing, making changes where they're needed. If you have some any more feedback (or if you have an app that falls in that category) I'd love to talk more in depth, please e-mail us at founders@clutch.io.
  - dave1619 14y ago
    Awesome, thanks. Much appreciated.

pheelicks 14y ago

How do I go about testing each branch of the code when developing? Is there some nice way in the web console to manually set this for the test device?

ericflo 14y ago
Yes, there's a form on the web dashboard that lets you choose a certain version for your development device.

zashapiro 14y ago

I'm so excited to give this a spin. Something simple and turnkey was absolutely needed. Nice work guys!

aresant 14y ago

Slightly off-topic question - is there a way to A/B test App Store descriptions / pictures / etc?

fourneau 14y ago
Unfortunately not. You can only upload 5 images per device type (iPad / iPhone) and one description. In theory, you can play the language game and change your descriptions radically based on language, however that's not exactly a valid experiment.

nixarn 14y ago

Looks really nice. Couple questions though. 1. What happens if you shut down your service? Will the apps function properly? 2. Any idea how much this will cost? I wonder because it would be possible to implement A/B testing without a problem using Flurry, and that would be completely free.

ericflo 14y ago
1. Apps will function properly. It's been a design tenant from the beginning to never have an outage (or shut downage) by us negatively impact our customer's users. 2. We haven't 100% decided yet, but we're heavily leaning towards packaging this service in with our other offering, at the same price point, so you get access to both at once: https://clutch.io/pricing/ https://clutch.io/pricing/
- nixarn 14y ago
  Thanks for the answer! The pricing on clutch.io's homepage looks good, I was worried it would get a pricing like parse.com that is in my opinion quite expensive per API call.

deleted 14y ago

[deleted]