3 ms·

A related and also very useful usage pattern: "backend instance per customer". Because… There are many ways to implement multi-tenant SaaS; but a highly under

A related and also very useful usage pattern: "backend instance per customer". Because… There are many ways to implement multi-tenant SaaS; but a highly underrated approach is to write a single tenant app, then use infrastructure to run an instance of it per (currently logged in) customer. Plus a persistent database per customer of course. This has the tremendous advantage that you can add another column to your pricing page easily: the "bring a truckload of money and we will set you up to run this behind your firewall" tier. There are still a lot of orgs out there, large ones with considerable financial capacity, who really want this.

deleted 4y ago
[deleted]
bluejekyll 4y ago
This model works but requires that the minimum costs of the stack for supporting a single tenant are low enough for the smallest tenant and revenue stream from them. Often times an overlooked aspect of this is that, even without a freemium option, that revenue can be 0. Consider all the demos for potential customers, and all the examples setup for testing, etc. these will all cost money, and if they can’t be shared, than those costs may make it unworkable on a per-instance basis.
- kylecordes 4y ago
  Yes certainly, this approach to multi-tenancy is not well-suited for apps that will support a large number of active free users. The idea is to only allocate a running instance while there is at least one active user of that instance. So for occasional-use apps the ongoing cost per idle customer would be just the storage cost of the database, very close to 0. Obviously for something like a chat app, email app, anything else that people tend to leave open all day, a different approach is better.
  - travisjungroth 4y ago
    > The idea is to only allocate a running instance while there is at least one active user of that instance. That first user is going to have a bit of a wait while you turn on their server. Maybe you keep some empties warmed up that just need a reconfig and restart so it’s fast. Personally, I’d only do any type of multi-tenant if the cost of some micro instance behind an auto scaler was negligible to the value of the contract anyway.
    - mattbee 4y ago
      I wrote a system at work that does this for VS Code instances - on a commodity server and not much optimisation effort it goes from a click to the UI starting to appear in about 10s (mostly thank you to Firecracker and Alpine) . There's a loading screen that's instant though, and probes for when the VM is ready. I think that would work fine for a lot of other apps, at least where you're looking to start a lengthier session.
      - intelVISA 4y ago
        Have done similar although 10s seems a bit far, were you using K8s? With our custom VMM could get similar instances up and routed within 2s w/o K8s.
        mattbee 4y ago
        I'm copying a 4GB root filesystem (on ext4!) per connection, so that's a couple of seconds, more like 10s on my laptop. The boot time for the kernel is about 2s to userspace initialisation. Then the userspace is running node.js to bring OpenVSCode up (2-3s?), starting a couple of other processes, and then the front-end is polling only every 1s. It's all on 1 server, just a Go https proxy launching firecracker on demand, no k8s. If I were optimising, I'd first make sure the root filesystem copy ops happed on a CoW filesystem, then change the readiness polling for a "long" poll that's under the server's control. And next buy a faster server :) But I'm using a relatively heavyweight userspace app, so I'm curious if you can see there might be other gains to be had. Our application is using VS Code as an environment for programming exercises. So even 30s would have been fine when people are using it after that for 1-2 hours. If I could get it down to 2s I'd certainly enjoy testing it more!
- travisjungroth 4y ago
  They could still be shared. Each customer is an org, not a single user. This model would only work at business pricing levels anyway. However many environments you have of Dev, Staging, QA, Integration, whatever is the same as normal. Sales gets an instance, but I’ve seen that anyway and it’s just one more. Can have a public Demo. As long as you don’t scale O(n) with prospects, it’s not a meaningful cost. Even if you did that, I’d bet it’s a tiny fraction of your cost of sales.
  - bluejekyll 4y ago
    Right, but that’s the point. Even orgs can be costly when you consider how you plan on sharing infrastructure. My point is these can add up as you have databases, k8s clusters, load balancers, CDN endpoints, etc. so if your strategy doesn’t include driving these costs down on a per instance basis, with idle usage and whatnot, it will become a cost problem quickly.
iLoveOncall 4y ago
I think you are putting together two concepts that are actually different: "backend instance per customer" and "deploy the service in the customer's infrastructure". I don't have much to say about the 2nd one, but my team does the first one for our (internal) customers, so we deploy and manage in our own accounts a whole service stack for each of our customers. It's a nightmare. We are going to do some rearchitecture work next year to move away from it, because it drains so much of our time in operational load. One example of a major pain point that we have is encountering random failures in 3rd party services, such as AWS. For a more classic service that you deploy 2 or 3 times a week, if CloudFormation deployments fail once every thousand deployments for random errors, you'll have one failure per year. Well, we deploy thousands of instances of our service 2-3 times a week. So we have failures every single time we deploy. Oncall is a nightmare (I don't love oncall anymore, I created my login in my previous team) because we just fix a tsunami of similar tickets that are each from a different instance of our service, for a single customer, but often with a different root cause. We probably have half of our headcount dedicated to initiatives that wouldn't need to exist if we had a more classic 1 service = 1 instance approach. Just don't do it.
- kylecordes 4y ago
  The connection between the concepts is: if your multi-tenant strategy is to deploy (hopefully only while in active use) an instance per customer, then the additional effort to provide on-site installations is relatively low, compared to other multi-tenant strategies. Whether on site installs are a good thing to offer, of course depends on how valuable that is to your customers versus the cost/effort to support it. As you point out there are major trade-offs! If your product architecture requires substantial multi-service complexity per tenant, that points away from the instance per customer strategy.
- travisjungroth 4y ago
  I can see how this is horrible for internal tools. The incentives are all messed up.
- gregplaysguitar 4y ago
  Out of interest are you deploying exactly the same software to each instance? Or are there customisations? My work deploys an instance per customer, and I haven’t encountered the same problem, but they number in the tens at this stage so not at the same scale yet. Would love to hear any dos or don’ts you’ve picked up, being further ahead in terms of scale!

deleted 4y ago

[deleted]

bluejekyll 4y ago

This model works but requires that the minimum costs of the stack for supporting a single tenant are low enough for the smallest tenant and revenue stream from them. Often times an overlooked aspect of this is that, even without a freemium option, that revenue can be 0. Consider all the demos for potential customers, and all the examples setup for testing, etc. these will all cost money, and if they can’t be shared, than those costs may make it unworkable on a per-instance basis.

kylecordes 4y ago
Yes certainly, this approach to multi-tenancy is not well-suited for apps that will support a large number of active free users. The idea is to only allocate a running instance while there is at least one active user of that instance. So for occasional-use apps the ongoing cost per idle customer would be just the storage cost of the database, very close to 0. Obviously for something like a chat app, email app, anything else that people tend to leave open all day, a different approach is better.
- travisjungroth 4y ago
  > The idea is to only allocate a running instance while there is at least one active user of that instance. That first user is going to have a bit of a wait while you turn on their server. Maybe you keep some empties warmed up that just need a reconfig and restart so it’s fast. Personally, I’d only do any type of multi-tenant if the cost of some micro instance behind an auto scaler was negligible to the value of the contract anyway.
  - mattbee 4y ago
    I wrote a system at work that does this for VS Code instances - on a commodity server and not much optimisation effort it goes from a click to the UI starting to appear in about 10s (mostly thank you to Firecracker and Alpine) . There's a loading screen that's instant though, and probes for when the VM is ready. I think that would work fine for a lot of other apps, at least where you're looking to start a lengthier session.
    - intelVISA 4y ago
      Have done similar although 10s seems a bit far, were you using K8s? With our custom VMM could get similar instances up and routed within 2s w/o K8s.
      - mattbee 4y ago
        I'm copying a 4GB root filesystem (on ext4!) per connection, so that's a couple of seconds, more like 10s on my laptop. The boot time for the kernel is about 2s to userspace initialisation. Then the userspace is running node.js to bring OpenVSCode up (2-3s?), starting a couple of other processes, and then the front-end is polling only every 1s. It's all on 1 server, just a Go https proxy launching firecracker on demand, no k8s. If I were optimising, I'd first make sure the root filesystem copy ops happed on a CoW filesystem, then change the readiness polling for a "long" poll that's under the server's control. And next buy a faster server :) But I'm using a relatively heavyweight userspace app, so I'm curious if you can see there might be other gains to be had. Our application is using VS Code as an environment for programming exercises. So even 30s would have been fine when people are using it after that for 1-2 hours. If I could get it down to 2s I'd certainly enjoy testing it more!
travisjungroth 4y ago
They could still be shared. Each customer is an org, not a single user. This model would only work at business pricing levels anyway. However many environments you have of Dev, Staging, QA, Integration, whatever is the same as normal. Sales gets an instance, but I’ve seen that anyway and it’s just one more. Can have a public Demo. As long as you don’t scale O(n) with prospects, it’s not a meaningful cost. Even if you did that, I’d bet it’s a tiny fraction of your cost of sales.
- bluejekyll 4y ago
  Right, but that’s the point. Even orgs can be costly when you consider how you plan on sharing infrastructure. My point is these can add up as you have databases, k8s clusters, load balancers, CDN endpoints, etc. so if your strategy doesn’t include driving these costs down on a per instance basis, with idle usage and whatnot, it will become a cost problem quickly.

iLoveOncall 4y ago

I think you are putting together two concepts that are actually different: "backend instance per customer" and "deploy the service in the customer's infrastructure". I don't have much to say about the 2nd one, but my team does the first one for our (internal) customers, so we deploy and manage in our own accounts a whole service stack for each of our customers. It's a nightmare. We are going to do some rearchitecture work next year to move away from it, because it drains so much of our time in operational load. One example of a major pain point that we have is encountering random failures in 3rd party services, such as AWS. For a more classic service that you deploy 2 or 3 times a week, if CloudFormation deployments fail once every thousand deployments for random errors, you'll have one failure per year. Well, we deploy thousands of instances of our service 2-3 times a week. So we have failures every single time we deploy. Oncall is a nightmare (I don't love oncall anymore, I created my login in my previous team) because we just fix a tsunami of similar tickets that are each from a different instance of our service, for a single customer, but often with a different root cause. We probably have half of our headcount dedicated to initiatives that wouldn't need to exist if we had a more classic 1 service = 1 instance approach. Just don't do it.

kylecordes 4y ago
The connection between the concepts is: if your multi-tenant strategy is to deploy (hopefully only while in active use) an instance per customer, then the additional effort to provide on-site installations is relatively low, compared to other multi-tenant strategies. Whether on site installs are a good thing to offer, of course depends on how valuable that is to your customers versus the cost/effort to support it. As you point out there are major trade-offs! If your product architecture requires substantial multi-service complexity per tenant, that points away from the instance per customer strategy.
travisjungroth 4y ago
I can see how this is horrible for internal tools. The incentives are all messed up.
gregplaysguitar 4y ago
Out of interest are you deploying exactly the same software to each instance? Or are there customisations? My work deploys an instance per customer, and I haven’t encountered the same problem, but they number in the tens at this stage so not at the same scale yet. Would love to hear any dos or don’ts you’ve picked up, being further ahead in terms of scale!