4 ms·
My MailChimp account actually got banned due to limitations in their AB testing framework. At the time at least, they had no proper way to segment cohorts, mea
by BrianEatWorld 12y ago
My MailChimp account actually got banned due to limitations in their AB testing framework.
At the time at least, they had no proper way to segment cohorts, meaning that in order to control for things like length or circumstances of subscription, we had to do multiple smaller sends to lists that were subsets of our subscribers and aggregate the testing data afterwards, rather than being able to do just one send. On top of this creating a lot of extra work setting up the sends, it skews the statistics on things like SPAM complaints.
What caused the account to be banned was that some of the smaller lists would occasionally see spikes in SPAM complaints. These spikes were well within statistical norms. However, because MailChimp will ban an account that breaks a threshold of complaints for a single list in a single sending, we triggered the ban, despite the list and send as a whole, being well within their limits. Had the AB testing framework been built properly, to account for cohorts, we would have had no issues.
I tried to speak with their CS to explain the issue, but it seems either they could not understand or weren't interested. All of this, after multiple communications with their tech team about the limitations of their frameworks and how they could improve their AB testing.
If you are looking to do anything beyond incredibly basic AB Testing, I would look elsewhere. We ended up at SendGrid, though my recent issues with their billing prevent me from recommending them either.
- Silhouette 12y agoI don't know whether this is still the case, but last time I looked into MailChimp they appeared extremely conservative/paranoid regarding spam. There seemed to be dire warnings all over the place about how anyone doing anything they didn't like would get banned. I wonder whether they once got stuck on a lot of blacklists after a bad customer abused their system, maybe some time in their early days when they were trying to establish credibility, and this (perhaps understandably) led to a policy of shooting first and not asking a lot of questions even later.