3 ms·

Reposting here. At Global Day of Code Retreat this past weekend I was surprised that our Game of Life implementation in SQL was the easiest. Why are we doing d

by crb002 8y ago

Reposting here. At Global Day of Code Retreat this past weekend I was surprised that our Game of Life implementation in SQL was the easiest. Why are we doing data schema type erasure at the Application layer? The "flexibility" and "encapsulation" aren't worth the manual hell of hand coding it back in all the way to the front end. Why not expose SQL schemas all the way to the front end? Even if the Application layer makes a view that the data store doesn't know about, everything downstream could benefit through automation. From there you get into issues mostly having to do with APIs that can be paginated, and how to optimize those connections to balance caching with network latency.

jmull 8y ago
> Why not expose SQL schemas all the way to the front end? Your API is a promise and if you break your promise bad things happen. If your API is SQL against some tables, you're promising you won't make any breaking changes to that -- or else you're promising you can roll out an update to all the servers and clients at the same time. SQL against tables has a very large surface area, and there are many details that are different depending on the version of your database software. So, you'll find you can't update your database software without also updating (and testing) all of your clients. The more complex and more widely distributed your software is, the more time-consuming and expensive it will be to update anything. If you have clients outside your direct control and running on the same release cycle, you'll find you can't update your database software at all without breaking your clients.
- TheDong 8y ago
  > SQL against tables has a very large surface area, and there are many details that are different depending on the version of your database software So you only offer an ANSI SQL compatible parser, not specific to any database (and it would then be compatible / compile-able to postgres, sqlite, whatever), and you only expose specific views into the database. Once you create such a public view, you treat it as immutable, and all API changes require new views. Each old view must function forever. This is not actually all that different from REST apis today; we version them, old versions must exist forever (or all clients must be updated), you can't update your server software that serves the rest API without being darn sure it's backwards compatible... The only real difference is we encode our CRUD operations in a non-sql language right now (ad-hoc json schemas usually) vs encoding them as the ANSI subset of SQL against some fictional database views. All of your complaints apply equally well to REST apis I think.
  - jmull 8y ago
    You’re ignoring the massive surface area of SQL, ANSI or no. You’re going to have to maintain that — for each of your API versions, including any/all semantics, quirks, etc clients may rely on. It’s a much larger commitment than you need to be making for many, many APIs.
jimbokun 8y ago
"Why not expose SQL schemas all the way to the front end?" Because providing a SQL interface to all of your external consumers is a good way to quickly bring your service to its knees.
Jach 8y ago
"We" do sometimes put declarative languages at the client level, whether SQL or something else like Prolog, Datalog, MDX... It's a long tradition. As for why "we" don't do it more often, many times you don't need to, or I think a lot of the barrier is a combination of lack of tooling plus lack of expressiveness in the client programming language. The alternative of "hand coding" actually has a lot of automated tooling already, too, so there's not much handwork involved at all if you don't want there to be. Feed the tool nothing but your schema, out pops a de-sql'd MVC-esque set of program files in whatever languages you care for. For some things it's nicest to work with these autogen'd artifacts over anything else as easily available. A somewhat related question that's fun to think about is why we don't put databases directly in our clients very often (especially SPAs). The ClojureScript community has been doing some good work exploring what it's like to put your client state in a client DB type of structure. There was also a compilation of SQLite to JavaScript done, I wonder how much usage that has?

by crb002 8y ago

jmull 8y ago

> Why not expose SQL schemas all the way to the front end? Your API is a promise and if you break your promise bad things happen. If your API is SQL against some tables, you're promising you won't make any breaking changes to that -- or else you're promising you can roll out an update to all the servers and clients at the same time. SQL against tables has a very large surface area, and there are many details that are different depending on the version of your database software. So, you'll find you can't update your database software without also updating (and testing) all of your clients. The more complex and more widely distributed your software is, the more time-consuming and expensive it will be to update anything. If you have clients outside your direct control and running on the same release cycle, you'll find you can't update your database software at all without breaking your clients.

TheDong 8y ago
> SQL against tables has a very large surface area, and there are many details that are different depending on the version of your database software So you only offer an ANSI SQL compatible parser, not specific to any database (and it would then be compatible / compile-able to postgres, sqlite, whatever), and you only expose specific views into the database. Once you create such a public view, you treat it as immutable, and all API changes require new views. Each old view must function forever. This is not actually all that different from REST apis today; we version them, old versions must exist forever (or all clients must be updated), you can't update your server software that serves the rest API without being darn sure it's backwards compatible... The only real difference is we encode our CRUD operations in a non-sql language right now (ad-hoc json schemas usually) vs encoding them as the ANSI subset of SQL against some fictional database views. All of your complaints apply equally well to REST apis I think.
- jmull 8y ago
  You’re ignoring the massive surface area of SQL, ANSI or no. You’re going to have to maintain that — for each of your API versions, including any/all semantics, quirks, etc clients may rely on. It’s a much larger commitment than you need to be making for many, many APIs.

jimbokun 8y ago

"Why not expose SQL schemas all the way to the front end?" Because providing a SQL interface to all of your external consumers is a good way to quickly bring your service to its knees.

Jach 8y ago

"We" do sometimes put declarative languages at the client level, whether SQL or something else like Prolog, Datalog, MDX... It's a long tradition. As for why "we" don't do it more often, many times you don't need to, or I think a lot of the barrier is a combination of lack of tooling plus lack of expressiveness in the client programming language. The alternative of "hand coding" actually has a lot of automated tooling already, too, so there's not much handwork involved at all if you don't want there to be. Feed the tool nothing but your schema, out pops a de-sql'd MVC-esque set of program files in whatever languages you care for. For some things it's nicest to work with these autogen'd artifacts over anything else as easily available. A somewhat related question that's fun to think about is why we don't put databases directly in our clients very often (especially SPAs). The ClojureScript community has been doing some good work exploring what it's like to put your client state in a client DB type of structure. There was also a compilation of SQLite to JavaScript done, I wonder how much usage that has?