3 ms·

Well, correct me if I'm wring but Shopfiy is a completely different scaling problem from Twitter. As I understand it Shopify's individual hosted stores are pret

Well, correct me if I'm wring but Shopfiy is a completely different scaling problem from Twitter. As I understand it Shopify's individual hosted stores are pretty much self-contained. So you can pretty much stick each one on it's own server with it's own database and it'll be fine. Twitter accounts all have to be able to talk to eachother in real time so you can't do that. My current startup has a Shopify-like architecture which is what I'm counting on to help me if I ever need to scale fast. So I think the first question you have to ask yourself when considering scaling is: what is my architecture like?

jsavimbi 15y ago
> So you can pretty much stick each one on it's own server with it's own database and it'll be fine. That is not a scalable solution. Yes, it'll get you up and running out of the box, but as you keep spinning up servers to host each store and it's database you'll have to keep adding exponential resources (hardware, software, meatware) to the problem and keep you from achieving economies of scale.
- psykotic 15y ago
  >Yes, it'll get you up and running out of the box, but as you keep spinning up servers to host each store and it's database you'll have to keep adding exponential resources (hardware, software, meatware) to the problem and keep you from achieving economies of scale. No, you'd be adding resources at a _linear_ rate relative to the growth of the customer base. The point about economies of scale is true enough but has nothing to do with a lack of exponential growth in costs.
  - jsavimbi 15y ago
    You're right; I'm becoming stupid in my old age.
- lionheart 15y ago
  Yes, its not an ideal solution for the long term, but its something easy you can do to scale quickly if you have this architecture. It'll make sure your site stays up. Afterwards, you can spend the time doing it properly when things have calmed down a little. That's a lot easier than scaling something like Twitter.
  - jsavimbi 15y ago
    No, sites go down all the time and need to rebooted, reconfigured, redeployed, strangled, etc. by a human who has to watch the servers 24 hours a day. I'm ignorant to the number of customers that Shopify is hosting these stores for, but let's say for example that one human can monitor 100 virtual machines during an eight-hour shift and you have 500 vm's running at the same time over a series of physical servers. That means, in HR terms, you'll need five people per shift or fifteen per day to monitor and act upon the vm's, along with at least two operations people, one of whom will be carrying the pager for the entire 24-hour period. That's seventeen people needed to run your operation not including sick/vacation/leave time that needs to be covered. In salary costs alone, that's over $1million/yr., not including recruiting and management salary, benefits and compensation. And does not take into effect any development, real estate, office essentials, hardware and software costs. > Afterwards, you can spend the time doing it properly when things have calmed down a little. That'll cost you an extra million dollars to develop and deploy while simultaneously running your existing operations and migrating your clients over to the new solution. It doesn't scale.
    - true_religion 15y ago
      I doubt that every single customer has to be on their own VM. It's more likely that many people share the same VM, and only larger customers need their own VM. Let's run some numbers though, using your assumptions: 500 VMs @ $179 per customer (business class licence) means a little over $1,000,000. At that point, you'd be right---it wouldn't scale. However, this is a business and not an HR exercise. So you could easily hire less people, and have less reliability. For a business like Shopify, I imagine their best customers are those who pay the $99 plan, accept a transaction cost, and other limitations. These customers can be piled onto the same VM together, and probably don't have such a huge throughput that a few minutes of downtime means money lost.
      - jsavimbi 15y ago
        > you could easily hire less people, and have less reliability. If your business has to be up and running 100% of the time, those are the minimal numbers needed to run the operation I've described. I cite a real word example, with real people and real money, the afterwards part if you will, which I'm currently working on to replace, not an hr exercise. And yes, it doesn't scale and it's very costly to replace, and one of the main reasons it was built in such a manner is because the original architect never thought it would have to scale and there are licensing considerations, something rarely mentioned around these parts. Licensing can really fuck things up.

jsavimbi 15y ago

> So you can pretty much stick each one on it's own server with it's own database and it'll be fine. That is not a scalable solution. Yes, it'll get you up and running out of the box, but as you keep spinning up servers to host each store and it's database you'll have to keep adding exponential resources (hardware, software, meatware) to the problem and keep you from achieving economies of scale.

psykotic 15y ago
>Yes, it'll get you up and running out of the box, but as you keep spinning up servers to host each store and it's database you'll have to keep adding exponential resources (hardware, software, meatware) to the problem and keep you from achieving economies of scale. No, you'd be adding resources at a _linear_ rate relative to the growth of the customer base. The point about economies of scale is true enough but has nothing to do with a lack of exponential growth in costs.
- jsavimbi 15y ago
  You're right; I'm becoming stupid in my old age.
lionheart 15y ago
Yes, its not an ideal solution for the long term, but its something easy you can do to scale quickly if you have this architecture. It'll make sure your site stays up. Afterwards, you can spend the time doing it properly when things have calmed down a little. That's a lot easier than scaling something like Twitter.
- jsavimbi 15y ago
  No, sites go down all the time and need to rebooted, reconfigured, redeployed, strangled, etc. by a human who has to watch the servers 24 hours a day. I'm ignorant to the number of customers that Shopify is hosting these stores for, but let's say for example that one human can monitor 100 virtual machines during an eight-hour shift and you have 500 vm's running at the same time over a series of physical servers. That means, in HR terms, you'll need five people per shift or fifteen per day to monitor and act upon the vm's, along with at least two operations people, one of whom will be carrying the pager for the entire 24-hour period. That's seventeen people needed to run your operation not including sick/vacation/leave time that needs to be covered. In salary costs alone, that's over $1million/yr., not including recruiting and management salary, benefits and compensation. And does not take into effect any development, real estate, office essentials, hardware and software costs. > Afterwards, you can spend the time doing it properly when things have calmed down a little. That'll cost you an extra million dollars to develop and deploy while simultaneously running your existing operations and migrating your clients over to the new solution. It doesn't scale.
  - true_religion 15y ago
    I doubt that every single customer has to be on their own VM. It's more likely that many people share the same VM, and only larger customers need their own VM. Let's run some numbers though, using your assumptions: 500 VMs @ $179 per customer (business class licence) means a little over $1,000,000. At that point, you'd be right---it wouldn't scale. However, this is a business and not an HR exercise. So you could easily hire less people, and have less reliability. For a business like Shopify, I imagine their best customers are those who pay the $99 plan, accept a transaction cost, and other limitations. These customers can be piled onto the same VM together, and probably don't have such a huge throughput that a few minutes of downtime means money lost.
    - jsavimbi 15y ago
      > you could easily hire less people, and have less reliability. If your business has to be up and running 100% of the time, those are the minimal numbers needed to run the operation I've described. I cite a real word example, with real people and real money, the afterwards part if you will, which I'm currently working on to replace, not an hr exercise. And yes, it doesn't scale and it's very costly to replace, and one of the main reasons it was built in such a manner is because the original architect never thought it would have to scale and there are licensing considerations, something rarely mentioned around these parts. Licensing can really fuck things up.