4 ms·

I love mixpanel, and this looks really promising. There's still a bit too much magic to my liking here. Not entirely sure how accurate this thing is... I asked

I love mixpanel, and this looks really promising. There's still a bit too much magic to my liking here. Not entirely sure how accurate this thing is... I asked their support how their algorithm calculates the grades, and this is what they said: > "In essence, it analyzes users who have converted and not converted on the specified event in the past, and attempts to construct a few default profiles based on the characteristics of those users, both in their events behavior and their people properties. Then, it compares these default profiles to your actual people profiles, and places each profile in the bucket that that profile most resembles. So that's how we end up with the grades A, B, C, and D, and place users in those buckets." It sounds good in theory, but I'd be quite interested in false positive / false negative rates or something a bit more transparent. Still, this can potentially be a big differentiator for Mixpanel. I guess we'll try to target those 'high confidence' users and see how well they convert...

jonknee 11y ago
> It sounds good in theory, but I'd be quite interested in false positive / false negative rates or something a bit more transparent. I believe it's machine learning, difficult to make transparent. That said it should be easy to test if changing your marketing to different levels of likely customers changes your total conversion rate.
rdoar 11y ago
It might be a better test to target the B and C users, and leave the A users alone. According to the algorithm they should convert anyways without your intervention.
shostack 11y ago
Are there any details on data security/ownership here? If they are doing all of this lookalike modeling, are they using your audience data for other customers? Are they potentially selling it to Adobe's new audience exchange? When you give an analytics company the level of conversion/revenue detail about your customers that one gives Mixpanel, there's a lot of cool things they can do, but there's always the nagging question of how much of that data do you really own. When it gets into modeling audience profiles, that gets really murky and I'd question whether our 1st party data is being being used by and/or sold to other parties. Would love some official clarification on this.

jonknee 11y ago

> It sounds good in theory, but I'd be quite interested in false positive / false negative rates or something a bit more transparent. I believe it's machine learning, difficult to make transparent. That said it should be easy to test if changing your marketing to different levels of likely customers changes your total conversion rate.

rdoar 11y ago

It might be a better test to target the B and C users, and leave the A users alone. According to the algorithm they should convert anyways without your intervention.

shostack 11y ago

Are there any details on data security/ownership here? If they are doing all of this lookalike modeling, are they using your audience data for other customers? Are they potentially selling it to Adobe's new audience exchange? When you give an analytics company the level of conversion/revenue detail about your customers that one gives Mixpanel, there's a lot of cool things they can do, but there's always the nagging question of how much of that data do you really own. When it gets into modeling audience profiles, that gets really murky and I'd question whether our 1st party data is being being used by and/or sold to other parties. Would love some official clarification on this.