4 ms·

Q: Do ALL your customers need to access EVERYONE ELSES data!? For most businesses, one customer is only interested in their own data. So that's where you can s

by z3t4 3y ago

Q: Do ALL your customers need to access EVERYONE ELSES data!? For most businesses, one customer is only interested in their own data. So that's where you can scale out - put each customer on their own virtual instance.

rwoerz 3y ago
CQ: What exactly do you mean with "virtual instance"?
- imtringued 3y ago
  CREATE DATABASE ${app}_${username}.
  - rwoerz 3y ago
    Okay, you use the username as the the key. What is the benefit over putting everthing in a partitioned database with partition keys?
    - z3t4 3y ago
      You don't need to to sharding if there is no relations between users. Just spread them out on different servers. If there is a relation, like who is friend with who, and not much else, you can lift out that into it's on service and scale it individually.
- nimrody 3y ago
  My take: have a few database instances (machines) each holding the data for a group of customers. With postgres you can even put different customers on the same database instances but in different schemas. This way you get all the benefits of the relational model (you can use foreign keys, transactions consisting of multiple tables, etc.) and the performance benefits of additional machines that are not just read-replicas. Centralized shared tables can be in a separate database which can also hold the mapping between customer-ids and database instances. Only drawback is that management is more difficult -- backup, migrations, etc. Specifically, you need to handle the case where some customers have migrated their database and others had not yet.
- z3t4 3y ago
  Run a separate db instance for each customer. With virtualization you can run many instances on the same physical machine. Depending on the tech stack all instances can be light weight. We did this in the early 2000 and our code was slow and machines even slower, but we could still scale to over 1k instances cheaply by spreading them out between different machines and used DNS for "routing" and its faster then todays centralized services.
  - rwoerz 3y ago
    AFAIR, you can register in Figma as a single user, i.e., user = customer. So, one DB instance for each of the 4 million users [1]? Sounds like a lot of overhead. [1] https://techcrunch.com/2022/06/08/figmas-dylan-field-will-discuss-evolving-as-a-leader-and-why-fun-is-an-essential-company-value-at-techcrunch-disrupt https://techcrunch.com/2022/06/08/figmas-dylan-field-will-di...
mcdonje 3y ago
How is that different from horizontally scaling (which they said they considered but couldn't find a managed solution for)?

by z3t4 3y ago

rwoerz 3y ago

CQ: What exactly do you mean with "virtual instance"?

imtringued 3y ago
CREATE DATABASE ${app}_${username}.
- rwoerz 3y ago
  Okay, you use the username as the the key. What is the benefit over putting everthing in a partitioned database with partition keys?
  - z3t4 3y ago
    You don't need to to sharding if there is no relations between users. Just spread them out on different servers. If there is a relation, like who is friend with who, and not much else, you can lift out that into it's on service and scale it individually.
nimrody 3y ago
My take: have a few database instances (machines) each holding the data for a group of customers. With postgres you can even put different customers on the same database instances but in different schemas. This way you get all the benefits of the relational model (you can use foreign keys, transactions consisting of multiple tables, etc.) and the performance benefits of additional machines that are not just read-replicas. Centralized shared tables can be in a separate database which can also hold the mapping between customer-ids and database instances. Only drawback is that management is more difficult -- backup, migrations, etc. Specifically, you need to handle the case where some customers have migrated their database and others had not yet.
z3t4 3y ago
Run a separate db instance for each customer. With virtualization you can run many instances on the same physical machine. Depending on the tech stack all instances can be light weight. We did this in the early 2000 and our code was slow and machines even slower, but we could still scale to over 1k instances cheaply by spreading them out between different machines and used DNS for "routing" and its faster then todays centralized services.
- rwoerz 3y ago
  AFAIR, you can register in Figma as a single user, i.e., user = customer. So, one DB instance for each of the 4 million users [1]? Sounds like a lot of overhead. [1] https://techcrunch.com/2022/06/08/figmas-dylan-field-will-discuss-evolving-as-a-leader-and-why-fun-is-an-essential-company-value-at-techcrunch-disrupt https://techcrunch.com/2022/06/08/figmas-dylan-field-will-di...

mcdonje 3y ago

How is that different from horizontally scaling (which they said they considered but couldn't find a managed solution for)?