4 ms·

It breaks when you hit the main CTA button to start a free trial now. I don’t understand how a company with billions of dollars in resources can fail to solve s

It breaks when you hit the main CTA button to start a free trial now. I don’t understand how a company with billions of dollars in resources can fail to solve such a well trodden problem. Even if there were a billion users on day 1. I wonder if the problem is due to an abdication of parts of the engineering problem, an over reliance on 3rd party services, that meant that no-one in the engineering team was able to study the whole engineering problem from first principles, which would result in inadequate fail overs.

berdon 7y ago
"well trodden problem" does not make it any easier. Scaling is hard. For everyone. Testing that scaling works is harder.
- sterileopinions 7y ago
  >"well trodden problem" does not make it any easier. Yes it does.
  - OnlineGladiator 7y ago
    I have to agree with sterileopinions here. The fact that it's been done multiple times before means you know a lot about how hard it will be, the necessary resources, and can learn from past mistakes. That doesn't make it easy. But it definitely makes it easier. To say otherwise is to argue that experience isn't useful.
    - sterileopinions 7y ago
      Thank you.
    - pilsetnieks 7y ago
      > The fact that it's been done multiple times before means you know a lot about how hard it will be, the necessary resources, and can learn from past mistakes. "All happy families are alike; each unhappy family is unhappy in its own way." -- Leo Tolstoy, Anna Karenina
- riazrizvi 7y ago
  It’s an ‘embarrassingly parallel’ problem. They could use a test load of a 1000 accounts, design a service for that, buy a sufficient proxy host and multiply. Scaling is hard when the problem is difficult to scale, browsing and streaming media is not a hard class of problem to scale, unlike an MMO say where real-time content updates occur in intricate patterns of locality and game logic.
  - JshWright 7y ago
    Do you have real world experience building out a system of this scale (even with a couple orders of magnitude)?
SigmundA 7y ago
No plan survives contact with the enemy. Simulating the real world is hard.
- riazrizvi 7y ago
  'No plan survives contact with the enemy', means you need an adaptable system to succeed. The way you use it though, it's like you are saying 'expect to fail'.
  - Macuyiko 7y ago
    That's exactly what you should be saying
zzzcpan 7y ago
> I don’t understand how a company with billions of dollars in resources can fail to solve such a well trodden problem. Billions of dollars don't give you good technical management and people with expertise to build such things. They only give an organization where there is no pressure to be good at anything and, you know, actually compete, there is only pressure to make sure they don't have to compete, that's how they got their billions after all. And it's not like there is a definitive guide or something that you can follow to build reliable systems without experts.
- matt-attack 7y ago
  Wasn’t this the whole point of buying BAMTech though? So they would have the technical leadership?
  - myrandomcomment 7y ago
    Issue is that BAMTech has not had to deal with this scale. Everything breaks at scale. That being said, the issues with the signup and login and servicing webpages are an understood problem at scale, so I do not give them a pass for those failures. On the stream side, Netflix wrote the book on it. There is a great deal of published work by Netflix team on this, down to kernel level coding and tweaking to get 100Gbs out of a single FreeBSD server. I know if I had to build this I would throw money at the issue by making an outrageous offer to someone that has built this before (from maybe Netflix?) to come and do it for me. As an aside I have worked on a project like this before on the networking side for the Olympics around 8 years ago. Heck, back in the .bomb I worked at a startup national ISP were doing QoS for video delivery was part of what we were doing. Never worked quite right.
gamblor956 7y ago
There's a gigantic arctic front moving across most of the US East coast right now. Thousands of schools and businesses are closed due to the weather. The point of launching on a Tuesday morning was to test out the system with a small audience ahead of Thanksgiving, or even the weekend, when they would properly expect to be slammed. But due to the weather, they're getting slammed by all the people stuck at home with nothing else to do.
- riazrizvi 7y ago
  Given that a good plan would involve a contingency of handling initial users between 10mm and 100mm, bad weather is a poor excuse.
  - reaperducer 7y ago
    You keep trying to flog that same horse, and you only end up sounding like someone who's never launched anything at scale.
rosege 7y ago
Netflix is renown for paying their engineers very well. I would be interested to know if Disney+ also pay such high salaries to attract the best?
- ProfessorLayton 7y ago
  I interviewed with them and no, they do not pay all that well, and their recruiter was pretty upfront about disclosing that early in the process. Additionally, I've hired a candidate that had an offer in hand from Disney+ and we were easily able to outbid them, (I was at a non-FAANG company at the time)
  - rosege 7y ago
    Thanks I guess that answers my question.
    - cookiecaper 7y ago
      Disney is notorious for paying below-market rates. There's a massive supply of naive vict--err, candidates-- sufficiently enamored with the idea of working at a well-loved cultural touchstone that they'll jump at the opportunity for almost any salary. It usually doesn't take too long for employees to be disabused of their childhood hopes and dreams, but it's often a very painful process, and there are always some diehards who never seem to be able to let go. I lived less than a mile from the Magic Kingdom for about four years and saw this process play out many times. Most people burn out and learn that maybe working on a boring business backend isn't that bad after all, but there's always a huge line of wide-eyed candidates eager to backfill. Companies that fit this profile never learn to treat employees well because, frankly, they don't have to. The same thing happens in games.
  - null_vector 7y ago
    What were the offer amounts?
- zamalek 7y ago
  I also strongly doubt you can simply bootstrap a Netflix caliber development team, no matter how much you spend. I have no doubt that the people who came up with Disney+ had no concept of this.

berdon 7y ago

"well trodden problem" does not make it any easier. Scaling is hard. For everyone. Testing that scaling works is harder.

sterileopinions 7y ago
>"well trodden problem" does not make it any easier. Yes it does.
- OnlineGladiator 7y ago
  I have to agree with sterileopinions here. The fact that it's been done multiple times before means you know a lot about how hard it will be, the necessary resources, and can learn from past mistakes. That doesn't make it easy. But it definitely makes it easier. To say otherwise is to argue that experience isn't useful.
  - sterileopinions 7y ago
    Thank you.
  - pilsetnieks 7y ago
    > The fact that it's been done multiple times before means you know a lot about how hard it will be, the necessary resources, and can learn from past mistakes. "All happy families are alike; each unhappy family is unhappy in its own way." -- Leo Tolstoy, Anna Karenina
riazrizvi 7y ago
It’s an ‘embarrassingly parallel’ problem. They could use a test load of a 1000 accounts, design a service for that, buy a sufficient proxy host and multiply. Scaling is hard when the problem is difficult to scale, browsing and streaming media is not a hard class of problem to scale, unlike an MMO say where real-time content updates occur in intricate patterns of locality and game logic.
- JshWright 7y ago
  Do you have real world experience building out a system of this scale (even with a couple orders of magnitude)?

SigmundA 7y ago

No plan survives contact with the enemy. Simulating the real world is hard.

riazrizvi 7y ago
'No plan survives contact with the enemy', means you need an adaptable system to succeed. The way you use it though, it's like you are saying 'expect to fail'.
- Macuyiko 7y ago
  That's exactly what you should be saying

zzzcpan 7y ago

> I don’t understand how a company with billions of dollars in resources can fail to solve such a well trodden problem. Billions of dollars don't give you good technical management and people with expertise to build such things. They only give an organization where there is no pressure to be good at anything and, you know, actually compete, there is only pressure to make sure they don't have to compete, that's how they got their billions after all. And it's not like there is a definitive guide or something that you can follow to build reliable systems without experts.

matt-attack 7y ago
Wasn’t this the whole point of buying BAMTech though? So they would have the technical leadership?
- myrandomcomment 7y ago
  Issue is that BAMTech has not had to deal with this scale. Everything breaks at scale. That being said, the issues with the signup and login and servicing webpages are an understood problem at scale, so I do not give them a pass for those failures. On the stream side, Netflix wrote the book on it. There is a great deal of published work by Netflix team on this, down to kernel level coding and tweaking to get 100Gbs out of a single FreeBSD server. I know if I had to build this I would throw money at the issue by making an outrageous offer to someone that has built this before (from maybe Netflix?) to come and do it for me. As an aside I have worked on a project like this before on the networking side for the Olympics around 8 years ago. Heck, back in the .bomb I worked at a startup national ISP were doing QoS for video delivery was part of what we were doing. Never worked quite right.

gamblor956 7y ago

There's a gigantic arctic front moving across most of the US East coast right now. Thousands of schools and businesses are closed due to the weather. The point of launching on a Tuesday morning was to test out the system with a small audience ahead of Thanksgiving, or even the weekend, when they would properly expect to be slammed. But due to the weather, they're getting slammed by all the people stuck at home with nothing else to do.

riazrizvi 7y ago
Given that a good plan would involve a contingency of handling initial users between 10mm and 100mm, bad weather is a poor excuse.
- reaperducer 7y ago
  You keep trying to flog that same horse, and you only end up sounding like someone who's never launched anything at scale.

rosege 7y ago

Netflix is renown for paying their engineers very well. I would be interested to know if Disney+ also pay such high salaries to attract the best?

ProfessorLayton 7y ago
I interviewed with them and no, they do not pay all that well, and their recruiter was pretty upfront about disclosing that early in the process. Additionally, I've hired a candidate that had an offer in hand from Disney+ and we were easily able to outbid them, (I was at a non-FAANG company at the time)
- rosege 7y ago
  Thanks I guess that answers my question.
  - cookiecaper 7y ago
    Disney is notorious for paying below-market rates. There's a massive supply of naive vict--err, candidates-- sufficiently enamored with the idea of working at a well-loved cultural touchstone that they'll jump at the opportunity for almost any salary. It usually doesn't take too long for employees to be disabused of their childhood hopes and dreams, but it's often a very painful process, and there are always some diehards who never seem to be able to let go. I lived less than a mile from the Magic Kingdom for about four years and saw this process play out many times. Most people burn out and learn that maybe working on a boring business backend isn't that bad after all, but there's always a huge line of wide-eyed candidates eager to backfill. Companies that fit this profile never learn to treat employees well because, frankly, they don't have to. The same thing happens in games.
- null_vector 7y ago
  What were the offer amounts?
zamalek 7y ago
I also strongly doubt you can simply bootstrap a Netflix caliber development team, no matter how much you spend. I have no doubt that the people who came up with Disney+ had no concept of this.