8 ms·

They had a working thing and then ruined it. I’m not sure how they managed to do that with a product change. Didn’t they have a way to release changes and measu

They had a working thing and then ruined it. I’m not sure how they managed to do that with a product change. Didn’t they have a way to release changes and measure impact before just forcing it on their entire user base? Once a company gets to this level, they need to recognize the role that luck played into their original product and design. They hit a resonance with people, and it was probably more luck and timing than anything else. Changes to that core product need to be treated as an experiment and conducted as such. Show some discipline. You can still innovate but those ideas should be made to stand on their own, grow on their own. Just because you have control over a platform with millions of users and can make changes doesn’t automatically mean you can do no wrong at this point. It just means you will have more time and resources before those mistakes take you out of the game completely. He should be punished for his mistakes by the board installing some experienced adult supervision for a while until he shows he can fix his current mess (reverse negative growth) and innovate in at least an additional major product category tangential to Snap. If he can’t do that in the next 24 months, he should step down for the good of the company.

ethbro 8y ago
I think it's unsurprising when founders of unicorns eschew A/B testing in favor of vision. Admitting part of their success was luck is tantamount to admitting "I went from nothing to a billionaire because the dice came up right." I don't know many people who would be comfortable truly accepting that. And even fewer who could do so AND be the type of driven person necessary. All while everyone around you is looking to you for answers.
- ljm 8y ago
  Purely anecdotally, the one place I worked at that embraced A/B testing went to such an extreme that it stifled the vision. It became so risk averse that nothing was really happening, yet at the same time the business was not in the place to sustain growth or even profit that way. This is because the introduction of such a data driven mentality meant the only people empowered to suggest decisions or experiments or approve the work were all in marketing. Because they were basically deciding what workload they wanted and of course you won’t put more on your plate if you don’t have to. This isn’t to say it’s about data vs. vision. The original success is usually created in large part due to a healthy collaboration between the visionaries, the engineers, and whoever else is in the mix. Once you take one part of the team out of the equation and tend too heavily towards a particular bias then you might well have lost the secret to the initial success.
  - jsonne 8y ago
    Your experience is similar to mine. I'm a marketing consultant so I get to work with a lot of different teams. People that either over-rely on Founder vision AND people that overly rely on data tend to not do well. It's why I've stopped saying data-driven and now say data informed. The best teams have a solid vision, but let the data guide the specific implementation of said vision. You need a healthy mix of both.
    - dwaltrip 8y ago
      I really like "data-informed" as an alternative to "data-driven". I'm going to copy that! It is important to honestly acknowledge the limitations of the data.
      - jiveturkey 8y ago
        me too
- sonnyblarney 8y ago
  This isn't quite fair. You can just as easily A/B your company to death. A/B usual represents an incremental comparison from one situation to another, and it helps 'walk down the feature path' ... but when you need to change paths, A/B testing is not going to help, this is where a bit of great PM talent is going to work. Moreover, the more subtle the change, or more the basis' are not very comparable (two different apps), the less A/B has value. For example: Snap was considered 'too complicated' for many users, they needed to make it simpler. But by doing so, they may alienate a bunch of current users, with the objective of making the app more amenable to the masses. This is a serious strategic issue. They might have gone about it somewhat differently but it's a difficult thing to do in any case, and they had to make changes. The existential issue for Snap's problems is not this app change, it's the fact that all of Facebook's products are trying to eat them. Instagram, WhatsApp, Messenger etc ... these are the real threats.
  - deleted 8y ago
    [deleted]
askafriend 8y ago
I think you hit the nail on the head here. There were reports that came out from several sources that Evan ignored all feedback from product and design including hard numbers from early tests. There were apparently several groups internally that tried to stop the massive changes from dropping without adequate testing and iteration. Those groups were overridden by Evan in favor of "vision" and that apparent lack of discipline led to the situation they're in today. If Evan can't show that he has the discipline to lead a company at this stage of it's life in the public market then the board should absolutely give him the boot in the next 24 months. I think the narrative of "product visionary" has been spun a bit too hard by the tech media and it likely has gotten to his head. He also took a $500mm bonus for successfully executing the IPO at a time when the company lost over $700mm in 2017. That's not a great look - but I digress.
- ReverseCold 8y ago
  > He also took a $500mm bonus for successfully executing the IPO at a time when the company lost over $700mm in 2017. So part of that $700mm loss was his bonus? Meaning if he hadn't taken a bonus the loss would "only" be $200mm?
  - DSingularity 8y ago
    No, wrong way to interpret that. He just received a massive bonus in $ value. It was massive because it was lots of Snapchat stock back when it was expensive. Although it’s quite normal to sell such stock award as soon as they vest, in this case it looks bad.
  - askafriend 8y ago
    I was a little off on the numbers. Snap posted a $2.2B loss in Q1 of 2017, it's first quarter as a public company. A lot of that was one-time stock payout to various employees, including a massive bonus to Spiegel which was 3% of Snap worth roughly ~$550mm at the time. If you take out all the one time stock compensation from that quarter, the loss comes out to $188mm. Here's a Recode article with some sources: https://www.recode.net/2017/5/10/15617740/snapchat-two-billion-loss-evan-spiegel-stock-compensation-q1-wall-street https://www.recode.net/2017/5/10/15617740/snapchat-two-billi... The $700mm loss figure I initially stated for 2017 is likely the loss figure with the one-time stock expenses deducted: https://investor.snap.com/news-releases/2018/02-06-2018-211639653 https://investor.snap.com/news-releases/2018/02-06-2018-2116... This stock award made Evan Spiegel the highest paid CEO in America in 2017, that's the kicker.
usrusr 8y ago
> Didn’t they have a way to release changes and measure impact before just forcing it on their entire user base? Many possible changes have positive effects short term, but negative or even disastrous effects long term. A/B tests won't ever run long enough to discover these long term costs. When A/B facts displace common sense instead of it complementing it, you can a/b yourself into your own grave.
- throwaway080383 8y ago
  If you run an experiment and time spent per day is cut in half, that's a pretty clear signal the experiment shouldn't ship, vision or not.
  - lukeschlather 8y ago
    What sort of web site do you run where customers pay by the minute? It's generally assumed that with advertising, more time == more money, but even that's somewhat questionable. In a B2B context, time spent per day cut in half sounds like a huge productivity boost. Of course, it could also be that you've made it so difficult to use now that people do a little bit, export to CSV and then use some other tool. But the data won't tell you that.
    - graeme 8y ago
      This isna thread about snapchat. People aren't on snapchat for efficiency. Those people moved to instagram instead.
_pmf_ 8y ago
> Once a company gets to this level, they need to recognize the role that luck played into their original product and design. They hit a resonance with people, and it was probably more luck and timing than anything else. Perfectly stated. "Pivoting from something shiny to an actual business plan" should be SV CEO's prime skill.
celticninja 8y ago
This is the lesson everyone should have taken from digg.com but it is still being played out again and again.
rl3 8y ago
To expand on what you're saying: when a user base forms habits around a product—especially something they use on a daily basis—changes no matter how good can be jarring as a result, and thus have a negative impact. Perhaps Snapchat as it exists now would've taken off just the same, but because the new version represents such a jarring departure from the old, it flops. A better example of this might be Microsoft keeping the Start Menu for over 20 years. While some cheered for its demise, most users decried the change at the time. That said, the problems with Snapchat as it exists today run far deeper than merely disrupting people's existing habits, so it's not the best example. Personally I think the app is now poorly-engineered trash. Nonetheless, the notion is an interesting one to at least consider, if only in an abstract sense.
akira2501 8y ago
> they need to recognize the role that luck played into their original product and design. That's a strong lesson I've learned in life. Successful people _rarely_ know the precise reasons that they are successful, almost exactly for this reason. You may be aware of many of the components of your success, but you won't be aware of all of them and it will be impossible for you to rank their importance correctly. Keep this in mind when successful people talk about their success and offer suggestions on how to replicate it.
- profalseidol 8y ago
  The hand that was dealt to you when you were born is probably the biggest luck factor of all.
  - Tempest1981 8y ago
    There is a good Warren Buffett documentary on HBO where he says the same thing. Or he’s just being humble. Or both.
    - profalseidol 8y ago
      If he is saying it because he simply wants to point this fact out (or even brag about it), and do not care doing anything about it. Then not so humble IMO. It is however still a merit that somebody raises it. It's probably the biggest Elephant in the room that there ever was. Especially if you are born in the third-world slums paying rent to the slumlord.
FooHentai 8y ago
>Just because you have control over a platform with millions of users and can make changes doesn’t automatically mean you can do no wrong at this point. Reminiscent of Digg v4.
simooooo 8y ago
They lost it as soon as they thought glasses were a good idea
agorabinary 8y ago
Did they ever reverse the product redesign back to the original design? Haven't really used Snapchat at all but curious to know, thanks
- dannyw 8y ago
  They completely reverted some small changes, and partially reverted more changes. But it wasn't a clean reverse; I still prefer the old snapchat to the new. I genuinely believe the best path forward for Snapchat is to completely reset back to pre-redesign as a way of making amends to all the users that left, and then re-designing again from there with their new focus.
calimac 8y ago
BINGO!

ethbro 8y ago

I think it's unsurprising when founders of unicorns eschew A/B testing in favor of vision. Admitting part of their success was luck is tantamount to admitting "I went from nothing to a billionaire because the dice came up right." I don't know many people who would be comfortable truly accepting that. And even fewer who could do so AND be the type of driven person necessary. All while everyone around you is looking to you for answers.

ljm 8y ago
Purely anecdotally, the one place I worked at that embraced A/B testing went to such an extreme that it stifled the vision. It became so risk averse that nothing was really happening, yet at the same time the business was not in the place to sustain growth or even profit that way. This is because the introduction of such a data driven mentality meant the only people empowered to suggest decisions or experiments or approve the work were all in marketing. Because they were basically deciding what workload they wanted and of course you won’t put more on your plate if you don’t have to. This isn’t to say it’s about data vs. vision. The original success is usually created in large part due to a healthy collaboration between the visionaries, the engineers, and whoever else is in the mix. Once you take one part of the team out of the equation and tend too heavily towards a particular bias then you might well have lost the secret to the initial success.
- jsonne 8y ago
  Your experience is similar to mine. I'm a marketing consultant so I get to work with a lot of different teams. People that either over-rely on Founder vision AND people that overly rely on data tend to not do well. It's why I've stopped saying data-driven and now say data informed. The best teams have a solid vision, but let the data guide the specific implementation of said vision. You need a healthy mix of both.
  - dwaltrip 8y ago
    I really like "data-informed" as an alternative to "data-driven". I'm going to copy that! It is important to honestly acknowledge the limitations of the data.
    - jiveturkey 8y ago
      me too
sonnyblarney 8y ago
This isn't quite fair. You can just as easily A/B your company to death. A/B usual represents an incremental comparison from one situation to another, and it helps 'walk down the feature path' ... but when you need to change paths, A/B testing is not going to help, this is where a bit of great PM talent is going to work. Moreover, the more subtle the change, or more the basis' are not very comparable (two different apps), the less A/B has value. For example: Snap was considered 'too complicated' for many users, they needed to make it simpler. But by doing so, they may alienate a bunch of current users, with the objective of making the app more amenable to the masses. This is a serious strategic issue. They might have gone about it somewhat differently but it's a difficult thing to do in any case, and they had to make changes. The existential issue for Snap's problems is not this app change, it's the fact that all of Facebook's products are trying to eat them. Instagram, WhatsApp, Messenger etc ... these are the real threats.
- deleted 8y ago
  [deleted]

askafriend 8y ago

I think you hit the nail on the head here. There were reports that came out from several sources that Evan ignored all feedback from product and design including hard numbers from early tests. There were apparently several groups internally that tried to stop the massive changes from dropping without adequate testing and iteration. Those groups were overridden by Evan in favor of "vision" and that apparent lack of discipline led to the situation they're in today. If Evan can't show that he has the discipline to lead a company at this stage of it's life in the public market then the board should absolutely give him the boot in the next 24 months. I think the narrative of "product visionary" has been spun a bit too hard by the tech media and it likely has gotten to his head. He also took a $500mm bonus for successfully executing the IPO at a time when the company lost over $700mm in 2017. That's not a great look - but I digress.

ReverseCold 8y ago
> He also took a $500mm bonus for successfully executing the IPO at a time when the company lost over $700mm in 2017. So part of that $700mm loss was his bonus? Meaning if he hadn't taken a bonus the loss would "only" be $200mm?
- DSingularity 8y ago
  No, wrong way to interpret that. He just received a massive bonus in $ value. It was massive because it was lots of Snapchat stock back when it was expensive. Although it’s quite normal to sell such stock award as soon as they vest, in this case it looks bad.
- askafriend 8y ago
  I was a little off on the numbers. Snap posted a $2.2B loss in Q1 of 2017, it's first quarter as a public company. A lot of that was one-time stock payout to various employees, including a massive bonus to Spiegel which was 3% of Snap worth roughly ~$550mm at the time. If you take out all the one time stock compensation from that quarter, the loss comes out to $188mm. Here's a Recode article with some sources: https://www.recode.net/2017/5/10/15617740/snapchat-two-billion-loss-evan-spiegel-stock-compensation-q1-wall-street https://www.recode.net/2017/5/10/15617740/snapchat-two-billi... The $700mm loss figure I initially stated for 2017 is likely the loss figure with the one-time stock expenses deducted: https://investor.snap.com/news-releases/2018/02-06-2018-211639653 https://investor.snap.com/news-releases/2018/02-06-2018-2116... This stock award made Evan Spiegel the highest paid CEO in America in 2017, that's the kicker.

usrusr 8y ago

> Didn’t they have a way to release changes and measure impact before just forcing it on their entire user base? Many possible changes have positive effects short term, but negative or even disastrous effects long term. A/B tests won't ever run long enough to discover these long term costs. When A/B facts displace common sense instead of it complementing it, you can a/b yourself into your own grave.

throwaway080383 8y ago
If you run an experiment and time spent per day is cut in half, that's a pretty clear signal the experiment shouldn't ship, vision or not.
- lukeschlather 8y ago
  What sort of web site do you run where customers pay by the minute? It's generally assumed that with advertising, more time == more money, but even that's somewhat questionable. In a B2B context, time spent per day cut in half sounds like a huge productivity boost. Of course, it could also be that you've made it so difficult to use now that people do a little bit, export to CSV and then use some other tool. But the data won't tell you that.
  - graeme 8y ago
    This isna thread about snapchat. People aren't on snapchat for efficiency. Those people moved to instagram instead.

_pmf_ 8y ago

> Once a company gets to this level, they need to recognize the role that luck played into their original product and design. They hit a resonance with people, and it was probably more luck and timing than anything else. Perfectly stated. "Pivoting from something shiny to an actual business plan" should be SV CEO's prime skill.

celticninja 8y ago

This is the lesson everyone should have taken from digg.com but it is still being played out again and again.

rl3 8y ago

To expand on what you're saying: when a user base forms habits around a product—especially something they use on a daily basis—changes no matter how good can be jarring as a result, and thus have a negative impact. Perhaps Snapchat as it exists now would've taken off just the same, but because the new version represents such a jarring departure from the old, it flops. A better example of this might be Microsoft keeping the Start Menu for over 20 years. While some cheered for its demise, most users decried the change at the time. That said, the problems with Snapchat as it exists today run far deeper than merely disrupting people's existing habits, so it's not the best example. Personally I think the app is now poorly-engineered trash. Nonetheless, the notion is an interesting one to at least consider, if only in an abstract sense.

akira2501 8y ago

> they need to recognize the role that luck played into their original product and design. That's a strong lesson I've learned in life. Successful people _rarely_ know the precise reasons that they are successful, almost exactly for this reason. You may be aware of many of the components of your success, but you won't be aware of all of them and it will be impossible for you to rank their importance correctly. Keep this in mind when successful people talk about their success and offer suggestions on how to replicate it.

profalseidol 8y ago
The hand that was dealt to you when you were born is probably the biggest luck factor of all.
- Tempest1981 8y ago
  There is a good Warren Buffett documentary on HBO where he says the same thing. Or he’s just being humble. Or both.
  - profalseidol 8y ago
    If he is saying it because he simply wants to point this fact out (or even brag about it), and do not care doing anything about it. Then not so humble IMO. It is however still a merit that somebody raises it. It's probably the biggest Elephant in the room that there ever was. Especially if you are born in the third-world slums paying rent to the slumlord.

FooHentai 8y ago

>Just because you have control over a platform with millions of users and can make changes doesn’t automatically mean you can do no wrong at this point. Reminiscent of Digg v4.

simooooo 8y ago

They lost it as soon as they thought glasses were a good idea

agorabinary 8y ago

Did they ever reverse the product redesign back to the original design? Haven't really used Snapchat at all but curious to know, thanks

dannyw 8y ago
They completely reverted some small changes, and partially reverted more changes. But it wasn't a clean reverse; I still prefer the old snapchat to the new. I genuinely believe the best path forward for Snapchat is to completely reset back to pre-redesign as a way of making amends to all the users that left, and then re-designing again from there with their new focus.

calimac 8y ago

BINGO!