4 ms·

Facebook can do that because they have a homogeneous environment using only MySQL. And they have a strict engineering discipline and strong DBA team to limit th

Facebook can do that because they have a homogeneous environment using only MySQL. And they have a strict engineering discipline and strong DBA team to limit the allowed schema structure. Not so much for other companies. Disclaimer: I don't work for Facebook, but I used to work at Google and build its Cloud SQL service.

evanelias 5y ago
I probably should have disclosed in my previous comment that I'm a former member of Facebook's MySQL infra/automation team, although I didn't work on schema management there specifically. However, subsequent to FB, I independently built the declarative schema management tool skeema.io which is used by several large well-known companies. So Atlas is a competitor, and I may inherently be biased in my skepticism of tools that use DSLs to generically handle multiple DBMS. (It sounds like we are in agreement there though, regarding your comment on FB's all-MySQL environment.) Anyway, in my previous comment, my point was that it is certainly possible to build trustworthy declarative schema management tools. It's difficult and requires a lot of effort, but it is not impossible. Facebook didn't limit schema structure too much, btw. One of my main projects there was building the in-house DBaaS interface, which was used for all sorts of things across the company, quite a wide variety of workloads and table designs. iirc the only major table structure limitations were: you must have a primary key; you must not use foreign key constraints; you must use InnoDB (or later MyRocks, but that was after my time). These are fairly common requirements among other large MySQL-backed companies though, definitely not specific to Facebook.
- 1500100900 5y ago
  > Anyway, in my previous comment, my point was that it is certainly possible to build trustworthy declarative schema management tools. It's difficult and requires a lot of effort, but it is not impossible. Unless the schema is self-modifying. Then obviously the database schema definition is the only source of truth.
  - evanelias 5y ago
    Can you provide an example of what you're referring to? The overall topic in this subthread was whether or not declarative schema management is appropriate and safe for "real" applications, as compared to traditional migration (imperative) schema management, for popular relational database systems. If you're referring to exclusively using stored procs to generate dynamic tables or something goofy like that, then sure -- in that case you can't use any external schema management at all, whether declarative or imperative. But that doesn't mean that declarative schema management isn't a safe or possible approach for everyone else.
    - 1500100900 5y ago
      Yes, something goofy like that. I don't know about declarative vs imperative. I can see "versioned migrations" are "coming soon", so maybe that could work. Depending on what it will be.
- tianzhou 5y ago
  You are right we are actually from the same clan which prefers pure SQL over another DSL. FWIW, google does take the same approach as skeema to describe the desired schema state with pure SQL. But it's the Google Spanner does the heavy lifting. And that adds more respect for your work on skeema, since skeema is doing the heavy lifting from the tooling side, which in my opinion is more challenge. BTW, I also need to disclose that I am currently building bytebase.com which is also a schema migration tool.
rossmohax 5y ago
Maybe you can explain why there is still no in place upgrade for CloudSQL PostgreSQL?
- tianzhou 5y ago
  Actually I was the one leading to build their initial PostgreSQL offering. But I left the team later. Technically, there's nothing prevent them doing that. It's probably just priority conflicts. A bit sigh though since it's a worthwhile feature IMO.

evanelias 5y ago

I probably should have disclosed in my previous comment that I'm a former member of Facebook's MySQL infra/automation team, although I didn't work on schema management there specifically. However, subsequent to FB, I independently built the declarative schema management tool skeema.io which is used by several large well-known companies. So Atlas is a competitor, and I may inherently be biased in my skepticism of tools that use DSLs to generically handle multiple DBMS. (It sounds like we are in agreement there though, regarding your comment on FB's all-MySQL environment.) Anyway, in my previous comment, my point was that it is certainly possible to build trustworthy declarative schema management tools. It's difficult and requires a lot of effort, but it is not impossible. Facebook didn't limit schema structure too much, btw. One of my main projects there was building the in-house DBaaS interface, which was used for all sorts of things across the company, quite a wide variety of workloads and table designs. iirc the only major table structure limitations were: you must have a primary key; you must not use foreign key constraints; you must use InnoDB (or later MyRocks, but that was after my time). These are fairly common requirements among other large MySQL-backed companies though, definitely not specific to Facebook.

1500100900 5y ago
> Anyway, in my previous comment, my point was that it is certainly possible to build trustworthy declarative schema management tools. It's difficult and requires a lot of effort, but it is not impossible. Unless the schema is self-modifying. Then obviously the database schema definition is the only source of truth.
- evanelias 5y ago
  Can you provide an example of what you're referring to? The overall topic in this subthread was whether or not declarative schema management is appropriate and safe for "real" applications, as compared to traditional migration (imperative) schema management, for popular relational database systems. If you're referring to exclusively using stored procs to generate dynamic tables or something goofy like that, then sure -- in that case you can't use any external schema management at all, whether declarative or imperative. But that doesn't mean that declarative schema management isn't a safe or possible approach for everyone else.
  - 1500100900 5y ago
    Yes, something goofy like that. I don't know about declarative vs imperative. I can see "versioned migrations" are "coming soon", so maybe that could work. Depending on what it will be.
tianzhou 5y ago
You are right we are actually from the same clan which prefers pure SQL over another DSL. FWIW, google does take the same approach as skeema to describe the desired schema state with pure SQL. But it's the Google Spanner does the heavy lifting. And that adds more respect for your work on skeema, since skeema is doing the heavy lifting from the tooling side, which in my opinion is more challenge. BTW, I also need to disclose that I am currently building bytebase.com which is also a schema migration tool.

rossmohax 5y ago

Maybe you can explain why there is still no in place upgrade for CloudSQL PostgreSQL?

tianzhou 5y ago
Actually I was the one leading to build their initial PostgreSQL offering. But I left the team later. Technically, there's nothing prevent them doing that. It's probably just priority conflicts. A bit sigh though since it's a worthwhile feature IMO.