4 ms·

Introducing mobile A/B testing

50 points by suhail 12y ago

Jculbertson 12y ago
Don't you think that a mobile a/b testing product should have a mobile-friendly website?
- suhail 12y ago
  Yes - we're working on making it more friendly right now but we wanted to release something rather than delay getting it into customer's hands.
- jug6ernaut 12y ago
  Really every website should have a mobile-friendly version, but no, i don't think that just because this is mobile testing oriented that it should have a mobile-friendly version any more then any other website. Being for mobile testing doesn't make it any more or less likely to be viewed or used from a mobile device.
orasis 12y ago
Note that this is under their People analytics, which is going to cost you a minimum of $1,800 per year for over 1000 users. Dear Mixpanel: When you say "mobile", understand that us "mobile app developers" don't make any money on less than 100,000 users. Your pricing is configured for enterprise Saas where the customer lifetime value is far higher than mobile. Please either change your pricing or stop calling yourselves a "mobile" analytics company.
- cdrux 12y ago
  I completely agree with this point that it is extremely challenging to make money on mobile. There are options, however, in the market if you are looking for a mobile A/B testing platform that understands this. To be completely open and honest, I am a co-founder of Taplytics. I just wanted to hop in here and point out that we have a Startup Plan that gives independent developers and mobile teams completely free use of our platform.
- deleted 12y ago
  [deleted]
- eli 12y ago
  Same issue. Our apps actually are for B2B, but they're ad supported so it's hard to make the numbers work on per-user People pricing. (We also don't really have "users" since pretty much everything is available without logging in, but that's a separate issue). But I'm willing to accept that that just isn't a niche Mixpanel wants to target.
- jedc 12y ago
  The Leanplum guys appear to be quite a bit cheaper: https://www.leanplum.com/pricing https://www.leanplum.com/pricing $79/month for up to 1.5k users.
- diziet 12y ago
  Yep, some companies I know have scaling issues with Mixpanel's pricing on the analytics side of things (because you end up paying basically 2-3c per user / year just on the people plan, and probably another couple of cents for events) - it might work OK if your LTV is pretty high but for anything consumer oriented it's tough to scale to lots of users. I'd seen pretty good pricing from Amplitude ( https://amplitude.com/pricing https://amplitude.com/pricing ) at scale.
- bduerst 12y ago
  Just checked it out, and you're right They're essentially charging 3.6 to 55.6 cents per user per year for the 1000-50000 user range. That's damn expensive for their low-end tier.
- hemantv 12y ago
  Optimizely has very reasonable pricing also https://www.optimizely.com/pricing https://www.optimizely.com/pricing
duartetb 12y ago
Is this for native development only? Or can it be used with frameworks like unity for games too?
- suhail 12y ago
  It's native only but we are thinking about making a wrapper to make it interoperable between iOS and Unity. You should be able to drop it in despite unity but it may require some small extra work. We can help if you email support@mixpanel.com
  - sandstrom 12y ago
    Support for JS-built apps [Phonegap, mobile-web] would be awesome too!
callmeed 12y ago
First, it's interesting that Mixpanel and Optimizely (both YC companies) are starting to compete in this space. Optimizely has mobile A/B testing as well[0]. Second, what seems to be an obvious need I haven't seen is A/B testing notifications. Notifications seem to be the "email drip campaigns" of the mobile world, so it seems like A/B testing both the frequency and content of those could bring big wins. [0] https://www.optimizely.com/mobile https://www.optimizely.com/mobile
- suhail 12y ago
  Hey there, we actually did that first before mobile ab testing: https://mixpanel.com/blog/2014/04/10/introducing-a-b-testing https://mixpanel.com/blog/2014/04/10/introducing-a-b-testing - it works for email, push, in-app, SMS, etc. We just didn't make a big stink about it.
aresant 12y ago
MixPanel's branding and the design consistency they achieve is best in class. I always know when I'm on a MixPanel product page / site - playful, colorful, professional. Just throwing some props out at Mason Yarnell who - I assume - is still responsible for executing the UX? https://dribbble.com/yarnell https://dribbble.com/yarnell
- suhail 12y ago
  Julien & Mason but it's a big team effort in the end.
luckydata 12y ago
Until their API doesn't improve, Mixpanel is a crippled product. Nothing they launch can cover that simple issue.
- suhail 12y ago
  What's the issue we have with our API? Email me: suhail@mixpanel.com - we'll try to fix it
- KennyCason 12y ago
  I'm interested in what is wrong as well, it has worked very well for us.
- brodil 12y ago
  what are the issues? i'd like to know before getting too much into it.
brodil 12y ago
has anyone tried it?

50 points by suhail 12y ago

Jculbertson 12y ago

Don't you think that a mobile a/b testing product should have a mobile-friendly website?

suhail 12y ago
Yes - we're working on making it more friendly right now but we wanted to release something rather than delay getting it into customer's hands.
jug6ernaut 12y ago
Really every website should have a mobile-friendly version, but no, i don't think that just because this is mobile testing oriented that it should have a mobile-friendly version any more then any other website. Being for mobile testing doesn't make it any more or less likely to be viewed or used from a mobile device.

orasis 12y ago

Note that this is under their People analytics, which is going to cost you a minimum of $1,800 per year for over 1000 users. Dear Mixpanel: When you say "mobile", understand that us "mobile app developers" don't make any money on less than 100,000 users. Your pricing is configured for enterprise Saas where the customer lifetime value is far higher than mobile. Please either change your pricing or stop calling yourselves a "mobile" analytics company.

cdrux 12y ago
I completely agree with this point that it is extremely challenging to make money on mobile. There are options, however, in the market if you are looking for a mobile A/B testing platform that understands this. To be completely open and honest, I am a co-founder of Taplytics. I just wanted to hop in here and point out that we have a Startup Plan that gives independent developers and mobile teams completely free use of our platform.
deleted 12y ago
[deleted]
eli 12y ago
Same issue. Our apps actually are for B2B, but they're ad supported so it's hard to make the numbers work on per-user People pricing. (We also don't really have "users" since pretty much everything is available without logging in, but that's a separate issue). But I'm willing to accept that that just isn't a niche Mixpanel wants to target.
jedc 12y ago
The Leanplum guys appear to be quite a bit cheaper: https://www.leanplum.com/pricing https://www.leanplum.com/pricing $79/month for up to 1.5k users.
diziet 12y ago
Yep, some companies I know have scaling issues with Mixpanel's pricing on the analytics side of things (because you end up paying basically 2-3c per user / year just on the people plan, and probably another couple of cents for events) - it might work OK if your LTV is pretty high but for anything consumer oriented it's tough to scale to lots of users. I'd seen pretty good pricing from Amplitude ( https://amplitude.com/pricing https://amplitude.com/pricing ) at scale.
bduerst 12y ago
Just checked it out, and you're right They're essentially charging 3.6 to 55.6 cents per user per year for the 1000-50000 user range. That's damn expensive for their low-end tier.
hemantv 12y ago
Optimizely has very reasonable pricing also https://www.optimizely.com/pricing https://www.optimizely.com/pricing

duartetb 12y ago

Is this for native development only? Or can it be used with frameworks like unity for games too?

suhail 12y ago
It's native only but we are thinking about making a wrapper to make it interoperable between iOS and Unity. You should be able to drop it in despite unity but it may require some small extra work. We can help if you email support@mixpanel.com
- sandstrom 12y ago
  Support for JS-built apps [Phonegap, mobile-web] would be awesome too!

callmeed 12y ago

First, it's interesting that Mixpanel and Optimizely (both YC companies) are starting to compete in this space. Optimizely has mobile A/B testing as well[0]. Second, what seems to be an obvious need I haven't seen is A/B testing notifications. Notifications seem to be the "email drip campaigns" of the mobile world, so it seems like A/B testing both the frequency and content of those could bring big wins. [0] https://www.optimizely.com/mobile https://www.optimizely.com/mobile

suhail 12y ago
Hey there, we actually did that first before mobile ab testing: https://mixpanel.com/blog/2014/04/10/introducing-a-b-testing https://mixpanel.com/blog/2014/04/10/introducing-a-b-testing - it works for email, push, in-app, SMS, etc. We just didn't make a big stink about it.

aresant 12y ago

MixPanel's branding and the design consistency they achieve is best in class. I always know when I'm on a MixPanel product page / site - playful, colorful, professional. Just throwing some props out at Mason Yarnell who - I assume - is still responsible for executing the UX? https://dribbble.com/yarnell https://dribbble.com/yarnell

suhail 12y ago
Julien & Mason but it's a big team effort in the end.

luckydata 12y ago

Until their API doesn't improve, Mixpanel is a crippled product. Nothing they launch can cover that simple issue.

suhail 12y ago
What's the issue we have with our API? Email me: suhail@mixpanel.com - we'll try to fix it
KennyCason 12y ago
I'm interested in what is wrong as well, it has worked very well for us.
brodil 12y ago
what are the issues? i'd like to know before getting too much into it.

brodil 12y ago

has anyone tried it?