5 ms·

This, I suppose, is unexpected. It reflects a similar fate of a lot of technology plays that grew fast. I would like to reflect however on this comment in the

This, I suppose, is unexpected. It reflects a similar fate of a lot of technology plays that grew fast. I would like to reflect however on this comment in the article -- "The truth is that the Periscope app is in an unsustainable maintenance-mode state, and has been for a while. Over the past couple of years, we’ve seen declining usage and know that the cost to support the app will only continue to go up over time." I understand how a startup madly codes and adapts to get to something that "sticks" with the user community. But where I get lost is how, in the process of acquisition and integration of a startup's technology into a company that is nominally not a startup, how is it that you go 5 years after acquisition and NOT retire the technical debt? How does a VP of Engineering oversee the integration of a startup into their domain and not spend the time it takes to reduce the operational costs of maintenance to as close to zero as possible? In today's world of BigCorp adding features by buying HotStartup, how can they not follow through an actually make that new tech a feature? If a feature is not getting anything new, it should take zero engineering resources. If the plaform's on which it runs change/update, there can be documentation on all of the platform features used, and how, a build system and a regression system, so that you take the code out of the box. Port it, test it, put it back in the box and go away again. Same thing with bugs or CVEs. When I was at Google it amazed me how many pieces of infrastructure were 'hot messes' in the sense that the people maintaining them didn't really know anything about how they worked, just how to run a test and revert if they broke. But there was no senior leadership pushing to understand the component interfaces and how the 'fleet' was at risk or interconnected. Just a bunch of fiefdoms. And inevitably people who kept things complicated so that they were "irreplaceable" for their own job security anxieties. If you are an engineer and you start moving up into senior leadership roles, understand that this is your job. Not to be be the whiz kid that just came up with a billion dollar idea, but to be the careful, thoughtful, strategist that makes sure the billion dollar idea doesn't eat all of its revenue in operational costs.

zestyping 6y ago
The most plausible explanation, to me, is that operating the product was not considered the primary purpose of the acquisition.
- bonoboTP 6y ago
  To be less subtle: it's to eliminate competition. If my neighbor has a nice big cow, it's not crucial for me to obtain that cow for myself; it suffices that it die, while my cows live.
  - TT3351 6y ago
    Nitpick but this implicitly requires I buy-in to a zero sum system
    - bonoboTP 6y ago
      It's enough that you understand that others have that view. You don't make the rules. The view of others shapes the incentive landscape. Things aren't zero sum, but our psychology has developed for a more or less zero sum environment. There was no exponential growth or constant innovation in every generation in those times. It is valuable to learn what drives people and what interpersonal forces elevate some strategies and suppress others.
    - minikites 6y ago
      Many things are in fact zero-sum systems.
    - cowpig 6y ago
      It doesn't; it requires you buy into the economics of monopoly. Your neighbor doesn't want your cow to die, it's the megacowcorp trying to charge four times the equilibrium price for milk.
    - jholman 6y ago
      No, it does not. Just mathematically the thing you said is clearly false. Suppose I say that, for some system, the return to each player is a function of the number of players: r(n) = 100 / (n+1). One player makes 50, two players make 33 each for a total of 66, three players make 25 each for a total of 75. Clearly not zero-sum, but I can still move up in the world by killing someone else's cows. One might observe that much of the world is like this; not truly zero-sum, but with payoff matrices that incentivize hurting or helping strangers. Note that almost all traditional microeconomics is this kind of curve-following system, one way or another. I'm not saying that kind of economics is right or wrong, just that it's a very traditional model, worth understanding as a starting point.
JPKab 6y ago
It should be noted that I strongly suspect the "declining usage" and "unsustainable maintenance mode" was a self-fulfilling prophecy. I would frequently want to tune in to a Periscope of a commentator, and then discover that if I locked my phone to listen to, rather than watch, the commentator, the stream would immediately halt. That shitty behavior that EVERY FREAKING USER complained about made me stop using the app. I've seen this many times in my career: The suits refuse to invest in a piece of tech, and then cite the loss of users as a reason to not invest, despite clear evidence that the loss of users is caused by the lack of investment.
draw_down 6y ago
> how is it that you go 5 years after acquisition and NOT retire the technical debt? That seems like the most simple thing in the world to me- you just ignore it. Have you ever talked to a product manager about "technical debt"? Newsflash- they don't give a shit. I don't know how things are run at Twitter but in general, people want to do "impactful" things and get promoted/get raises. Paying down technical debt is the opposite of that. You're moralizing about "this is your job" and all that junk. Feel free to do that, but the incentive structures are what they are. And it really ain't about morality, Jack. I'm not saying it's right or good, of course.
- bonoboTP 6y ago
  People should be thinking more in terms of game theory, incentives, iterated games or one-off games, the effects of imperfect information, lemon markets, delayed effects and the difficulties of credit assignment, job hopping and the lack of long-term reputation the way we evolved to have in close communities we stayed in for our lives in the ancestral environment, etc.
- xyzelement 6y ago
  >> Have you ever talked to a product manager about "technical debt"? Newsflash- they don't give a shit. Eng manager turned product manager here. It's not that we don't give a shit - but paying down the debt isn't always worth it. For example in this case, if they always knew they were going to wind down Periscope (or at least that it wasn't strategic compared to what they were doing with Twitter) then investing deeply into it would have just been a waste. Here's an example. You buy a house in the spring. You know the heating system is very old and will likely fail soon (technical debt.) Should you invest in it? The answer is "it depends." If you plan to live in this house for a long time, then yes because the last thing you want is for heat to fail in the middle of next winter. However, what if this is just your summer residence? Suddenly fixing the heat is much less urgent (you still have the debt but the risk of it coming due in a very bad way is much less.) Or, you bought the house for the land and plan to tear it down. You would be stupid to fix the heat. I think the Periscope situation is much closer to that last thing. They bought it for something other than the stand-alone Periscope product so throwing a bunch of resources at it didn't make sense. Which isn't the same as not giving a shit.
chambo622 6y ago
Per the announcement, the live video infrastructure that powers Periscope will continue to power live video on Twitter. Costs to support a standalone app are more than just infra and CDN costs - think support, moderation, legal, comms, QA, etc.
- ChuckMcM 6y ago
  I had read that, and so the 'cost' is the marginal cost of "support", "moderation", "legal" (assuming no features QA is probably not huge). So is it possible to operate a feature that has user generated content at at a positive margin? One without compromising privacy or incurring an untenable liability? I accept that this may not be possible, but it seems like that is where the engineering work is. And yes, I know product managers are "we need new stuff" to push all the time. And sales is constantly trying to sell something you don't make. But those are execution issues that good management will moderate.
kelnos 6y ago
> If you are an engineer and you start moving up into senior leadership roles, understand that this is your job Product wants new features and new products, and they want them now. Sales wants new features and new products, and they want them yesterday. Finance people want everything to be cheap, but since most engineers have no idea how to prove to them with actual hard data that it's cheaper in the long run ship features more carefully and slowly, and build things with reliability and robustness in mind, finance people go in the opposite direction and think that hiring inexperienced developers from the cheapest places on the planet is the way to go, because "cost of R&D per head" is a very easy number to quantify. Unfortunately very few companies have an engineering department that can overrule all of product, sales, and finance. In general even one of those three being stronger means engineering never has the buy-in to take the time to do things the right way. And if you're the engineer who consistently pushes back product and feature releases because you want things built correctly, you get fired, or at best marginalized. Interesting that you mention Google, since it has a reputation for being more engineering-focused that most companies. From what I read on HN (perhaps not a representative sample, but...), it seems like Google has a culture of "build something cool to get promoted and then stop caring about what you built because maintenance doesn't get you promoted any further". If true, I think that very easily explains Google's "hot messes".
- bonoboTP 6y ago
  > Interesting that you mention Google, since it has a reputation for being more engineering-focused that most companies. From what I read on HN (perhaps not a representative sample, but...), it seems like Google has a culture of "build something cool to get promoted and then stop caring about what you built because maintenance doesn't get you promoted any further". If true, I think that very easily explains Google's "hot messes". Countering my Gell-Mann amnesia here (many times I read about my area of expertise I see snappy criticisms trickle into the mainstream when the reality on the ground has long moved ahead and the supposedly insightful HN criticisms are cliches being routinely addressed within the expert circles, think "deep learning is just X, it doesn't address Y!" or "physicists are obviously boneheaded and got stuck at assumption Z!"). So, with the above in mind, I wonder if Google is actively countering this. If random people know this as a stereotype (and I read this often, and hear it as a stereotype of Google also in real life), then perhaps leadership is even more aware and either decides not to change or has already implemented things they think would help. Any current Googlers to chip in?
bonoboTP 6y ago
I don't want to be snarky, but there can totally be a non-technical outer context where all this was a very good strategic move. Certainly, one should not always assume dark Machiavellian moves, lest we become too bitter and cynical, but business isn't engineering. Read the book "The Elephant in the Brain: Hidden Motives in Everyday Life" by Kevin Simler and Robin Hanson. If not for other reasons, to acquire a different set of mental tools to understand motivation. Because I feel like you seem technically proficient but lacking in imagination of more important games of power in the background. And these aren't Disney villain machinations, they are normal everyday things that people implicitly understand and participate in, while being perfectly ordinary, nice, polite people, no mustache-twirling villains. Eat or be eaten. Periscope was eaten, now they are being shat out.
spullara 6y ago
My bet is that it doesn't make sense as a separate product when every other platform has live streaming. As simple as that.
karmicjustice99 6y ago
I disagree. It is NOT your job to fix any of this sh*t given the way you are rewarded and grow your career in BigCorp is by shipping new features and growing the customer base. Cost saving or optimization is barely ever an important priority in any tech organization BigCorp that wants to attract and retain talent because most developers wouldn’t want to work there - there would likely also not be delicious food in the cafes and kitchens either. Startups similarly spend money extensively to attract talent and keep them, not by being cost, efficiency and order focused.
plif 6y ago
> When I was at Google it amazed me how many pieces of infrastructure were 'hot messes' in the sense that the people maintaining them didn't really know anything about how they worked, just how to run a test and revert if they broke. But there was no senior leadership pushing to understand the component interfaces and how the 'fleet' was at risk or interconnected. Just a bunch of fiefdoms. And inevitably people who kept things complicated so that they were "irreplaceable" for their own job security anxieties. This is the curse of a large bureaucracy. It seemed like Google was immune to this for a while, perhaps due to the engineering culture, but it's only a matter of time when there's enough money and organizational inertia at play. Once things reach a boiling point, the transformational programs pop up, and the cycle continues.
byecomputer 6y ago
> 5 years after acquisition Similar timeline with the Vine acquisition and subsequent dismemberment, except Twitter announced Vine's discontinuation after 4 years rather than 5. > and NOT retire the technical debt? I would be surprised if this were the honest reason, at least more surprised than I'd be were it any other web giant, mostly because Twitter seems to have a mature grasp on why an average tenure of 2 years is insane for an engineering job and attempts to avoid the problems you mentioned. I reckon Twitter had motivations that simply don't scan well with people who enjoy using a product of yours and don't want to see it quashed.

zestyping 6y ago

The most plausible explanation, to me, is that operating the product was not considered the primary purpose of the acquisition.

bonoboTP 6y ago
To be less subtle: it's to eliminate competition. If my neighbor has a nice big cow, it's not crucial for me to obtain that cow for myself; it suffices that it die, while my cows live.
- TT3351 6y ago
  Nitpick but this implicitly requires I buy-in to a zero sum system
  - bonoboTP 6y ago
    It's enough that you understand that others have that view. You don't make the rules. The view of others shapes the incentive landscape. Things aren't zero sum, but our psychology has developed for a more or less zero sum environment. There was no exponential growth or constant innovation in every generation in those times. It is valuable to learn what drives people and what interpersonal forces elevate some strategies and suppress others.
  - minikites 6y ago
    Many things are in fact zero-sum systems.
  - cowpig 6y ago
    It doesn't; it requires you buy into the economics of monopoly. Your neighbor doesn't want your cow to die, it's the megacowcorp trying to charge four times the equilibrium price for milk.
  - jholman 6y ago
    No, it does not. Just mathematically the thing you said is clearly false. Suppose I say that, for some system, the return to each player is a function of the number of players: r(n) = 100 / (n+1). One player makes 50, two players make 33 each for a total of 66, three players make 25 each for a total of 75. Clearly not zero-sum, but I can still move up in the world by killing someone else's cows. One might observe that much of the world is like this; not truly zero-sum, but with payoff matrices that incentivize hurting or helping strangers. Note that almost all traditional microeconomics is this kind of curve-following system, one way or another. I'm not saying that kind of economics is right or wrong, just that it's a very traditional model, worth understanding as a starting point.

JPKab 6y ago

It should be noted that I strongly suspect the "declining usage" and "unsustainable maintenance mode" was a self-fulfilling prophecy. I would frequently want to tune in to a Periscope of a commentator, and then discover that if I locked my phone to listen to, rather than watch, the commentator, the stream would immediately halt. That shitty behavior that EVERY FREAKING USER complained about made me stop using the app. I've seen this many times in my career: The suits refuse to invest in a piece of tech, and then cite the loss of users as a reason to not invest, despite clear evidence that the loss of users is caused by the lack of investment.

draw_down 6y ago

> how is it that you go 5 years after acquisition and NOT retire the technical debt? That seems like the most simple thing in the world to me- you just ignore it. Have you ever talked to a product manager about "technical debt"? Newsflash- they don't give a shit. I don't know how things are run at Twitter but in general, people want to do "impactful" things and get promoted/get raises. Paying down technical debt is the opposite of that. You're moralizing about "this is your job" and all that junk. Feel free to do that, but the incentive structures are what they are. And it really ain't about morality, Jack. I'm not saying it's right or good, of course.

bonoboTP 6y ago
People should be thinking more in terms of game theory, incentives, iterated games or one-off games, the effects of imperfect information, lemon markets, delayed effects and the difficulties of credit assignment, job hopping and the lack of long-term reputation the way we evolved to have in close communities we stayed in for our lives in the ancestral environment, etc.
xyzelement 6y ago
>> Have you ever talked to a product manager about "technical debt"? Newsflash- they don't give a shit. Eng manager turned product manager here. It's not that we don't give a shit - but paying down the debt isn't always worth it. For example in this case, if they always knew they were going to wind down Periscope (or at least that it wasn't strategic compared to what they were doing with Twitter) then investing deeply into it would have just been a waste. Here's an example. You buy a house in the spring. You know the heating system is very old and will likely fail soon (technical debt.) Should you invest in it? The answer is "it depends." If you plan to live in this house for a long time, then yes because the last thing you want is for heat to fail in the middle of next winter. However, what if this is just your summer residence? Suddenly fixing the heat is much less urgent (you still have the debt but the risk of it coming due in a very bad way is much less.) Or, you bought the house for the land and plan to tear it down. You would be stupid to fix the heat. I think the Periscope situation is much closer to that last thing. They bought it for something other than the stand-alone Periscope product so throwing a bunch of resources at it didn't make sense. Which isn't the same as not giving a shit.

chambo622 6y ago

Per the announcement, the live video infrastructure that powers Periscope will continue to power live video on Twitter. Costs to support a standalone app are more than just infra and CDN costs - think support, moderation, legal, comms, QA, etc.

ChuckMcM 6y ago
I had read that, and so the 'cost' is the marginal cost of "support", "moderation", "legal" (assuming no features QA is probably not huge). So is it possible to operate a feature that has user generated content at at a positive margin? One without compromising privacy or incurring an untenable liability? I accept that this may not be possible, but it seems like that is where the engineering work is. And yes, I know product managers are "we need new stuff" to push all the time. And sales is constantly trying to sell something you don't make. But those are execution issues that good management will moderate.

kelnos 6y ago

> If you are an engineer and you start moving up into senior leadership roles, understand that this is your job Product wants new features and new products, and they want them now. Sales wants new features and new products, and they want them yesterday. Finance people want everything to be cheap, but since most engineers have no idea how to prove to them with actual hard data that it's cheaper in the long run ship features more carefully and slowly, and build things with reliability and robustness in mind, finance people go in the opposite direction and think that hiring inexperienced developers from the cheapest places on the planet is the way to go, because "cost of R&D per head" is a very easy number to quantify. Unfortunately very few companies have an engineering department that can overrule all of product, sales, and finance. In general even one of those three being stronger means engineering never has the buy-in to take the time to do things the right way. And if you're the engineer who consistently pushes back product and feature releases because you want things built correctly, you get fired, or at best marginalized. Interesting that you mention Google, since it has a reputation for being more engineering-focused that most companies. From what I read on HN (perhaps not a representative sample, but...), it seems like Google has a culture of "build something cool to get promoted and then stop caring about what you built because maintenance doesn't get you promoted any further". If true, I think that very easily explains Google's "hot messes".

bonoboTP 6y ago
> Interesting that you mention Google, since it has a reputation for being more engineering-focused that most companies. From what I read on HN (perhaps not a representative sample, but...), it seems like Google has a culture of "build something cool to get promoted and then stop caring about what you built because maintenance doesn't get you promoted any further". If true, I think that very easily explains Google's "hot messes". Countering my Gell-Mann amnesia here (many times I read about my area of expertise I see snappy criticisms trickle into the mainstream when the reality on the ground has long moved ahead and the supposedly insightful HN criticisms are cliches being routinely addressed within the expert circles, think "deep learning is just X, it doesn't address Y!" or "physicists are obviously boneheaded and got stuck at assumption Z!"). So, with the above in mind, I wonder if Google is actively countering this. If random people know this as a stereotype (and I read this often, and hear it as a stereotype of Google also in real life), then perhaps leadership is even more aware and either decides not to change or has already implemented things they think would help. Any current Googlers to chip in?

bonoboTP 6y ago

I don't want to be snarky, but there can totally be a non-technical outer context where all this was a very good strategic move. Certainly, one should not always assume dark Machiavellian moves, lest we become too bitter and cynical, but business isn't engineering. Read the book "The Elephant in the Brain: Hidden Motives in Everyday Life" by Kevin Simler and Robin Hanson. If not for other reasons, to acquire a different set of mental tools to understand motivation. Because I feel like you seem technically proficient but lacking in imagination of more important games of power in the background. And these aren't Disney villain machinations, they are normal everyday things that people implicitly understand and participate in, while being perfectly ordinary, nice, polite people, no mustache-twirling villains. Eat or be eaten. Periscope was eaten, now they are being shat out.

spullara 6y ago

My bet is that it doesn't make sense as a separate product when every other platform has live streaming. As simple as that.

karmicjustice99 6y ago

I disagree. It is NOT your job to fix any of this sh*t given the way you are rewarded and grow your career in BigCorp is by shipping new features and growing the customer base. Cost saving or optimization is barely ever an important priority in any tech organization BigCorp that wants to attract and retain talent because most developers wouldn’t want to work there - there would likely also not be delicious food in the cafes and kitchens either. Startups similarly spend money extensively to attract talent and keep them, not by being cost, efficiency and order focused.

plif 6y ago

> When I was at Google it amazed me how many pieces of infrastructure were 'hot messes' in the sense that the people maintaining them didn't really know anything about how they worked, just how to run a test and revert if they broke. But there was no senior leadership pushing to understand the component interfaces and how the 'fleet' was at risk or interconnected. Just a bunch of fiefdoms. And inevitably people who kept things complicated so that they were "irreplaceable" for their own job security anxieties. This is the curse of a large bureaucracy. It seemed like Google was immune to this for a while, perhaps due to the engineering culture, but it's only a matter of time when there's enough money and organizational inertia at play. Once things reach a boiling point, the transformational programs pop up, and the cycle continues.

byecomputer 6y ago

> 5 years after acquisition Similar timeline with the Vine acquisition and subsequent dismemberment, except Twitter announced Vine's discontinuation after 4 years rather than 5. > and NOT retire the technical debt? I would be surprised if this were the honest reason, at least more surprised than I'd be were it any other web giant, mostly because Twitter seems to have a mature grasp on why an average tenure of 2 years is insane for an engineering job and attempts to avoid the problems you mentioned. I reckon Twitter had motivations that simply don't scan well with people who enjoy using a product of yours and don't want to see it quashed.