8 ms·

This just runs a request proxy that turns off after 10 seconds of no activity and starts it up (with a half second delay) when there is a new request. It runs S

This just runs a request proxy that turns off after 10 seconds of no activity and starts it up (with a half second delay) when there is a new request. It runs SQLite with Prisma. Prisma is an API server that puts a GraphQL API in front of a DB. It's a nice blog post about gluing technology and I can see how this could be a really nice way to run some lower-cost databases in a non-demanding development environment. However, it is not a reliable way of operating a database. For me it isn't really serverless since it only scales between 0 and 1 instance whereas a serverless DB ideally would scale-out but should at least have some ability to scale to greater load in response to demand, along with higher reliability and availability, and backing up data to object storage.

jensneuse 4y ago
As discussed in the post, the next steps are to add read replicas. Regarding backups, that's possible with Litestream: https://github.com/benbjohnson/litestream https://github.com/benbjohnson/litestream
- gregwebs 4y ago
  Are you going to use LiteFS then for replication [1]? LiteFS replication is asynchronous, meaning failover can lose the latest data. Will LiteFS scale down to 0? Does scaling down to zero mean electing a leader when scaling back up to 1 and will that have a delay? Will the read replicas scale down to 0 along with LiteFS when the primary scales down to 0? [1] https://github.com/superfly/litefs https://github.com/superfly/litefs
  - benbjohnson 4y ago
    LiteFS/Litestream author here. LiteFS will scale to zero with a persistent volume attached. For short-lived instances (aka serverless), we still have a few more features to complete on the road map (e.g. S3 replication, synchronous replication) to make that work well. Pure serverless (e.g. Lambda, Vercel) is also something we plan to support but we want to get LiteFS working well on more traditional deployments (e.g. longer running instances, Kubernetes, etc) first.
lbhdc 4y ago
I was pretty interested to see how this worked as well. I think you are right, this is a toy. It will be interesting to see if they can solve scaling. It would be cool if you could ditch the graphql layer. It seems like there are other alternatives that go the vfs route so you still get to use a standard sqlite client.
tptacek 4y ago
You can, of course, scale to >1 with the Fly Machines API. I don't know enough about how they're managing the SQLite part of this to say more about how this scales, except that I think scaling out SQLite is about to get a lot more interesting. But I mostly agree that we need a better term than "serverless" for this kind of stuff. The big things people seem to want from "serverless" solutions are "not managing long-running server instances" and "true usage-metered billing". Whether or not there are servers, like, at all has not all that much to do with things.
- gregwebs 4y ago
  This is SQLite, so how would you scale itr to > 1? Certainly you can put an app tier in front of this DB tier and scale the app tier to infinity.
  - tptacek 4y ago
    By replicating the SQLite transactions to other SQLite databases.
    - soamv 4y ago
      Hey, I like SQLite and I also like fly.io, but "distributed DB built with sqlite as storage" is really a very different beast from just "sqlite".
      - tptacek 4y ago
        I'm mostly interested in the SQLite part of this, and we don't have an SQLite offering, only Postgres, just to be clear. So you can't hurt my feelings here. When does SQLite become "a distributed DB built with sqlite as storage"? Did Postgres stop being Postgres when someone plugged log shipping into it? That's basically what we're talking about here --- not stuff like rqlite, which I'm also pretty interested in, but which really is a new database built on SQLite.
        4y ago
ithrow 4y ago
Couldn't find anything in the Prisma docs about it exposing a GraphQL API.
- jensneuse 4y ago
  https://github.com/prisma/prisma-engines#query-engine https://github.com/prisma/prisma-engines#query-engine
nikolasburk 4y ago
Hey there, I'm Nikolas from the Prisma team. Just came here to quickly clarify this notion: > Prisma is an API server that puts a GraphQL API in front of a DB. Prisma is an ORM which generates a JavaScript/TypeScript client library for your database. Your description is very true for Prisma 1 (which has been in maintenance mode for several years and is officially deprecated by now [1]), but the latest version(s) of Prisma (v2+) don't expose a GraphQL API any more. Prisma 1 also used GraphQL SDL for data modeling, the Prisma ORM on the other hand has its own, custom modeling language for describing database schemas in a declarative way and also comes with a flexible migration system. That being said (and as Jens also mentioned elsewhere), the Prisma ORM does use GraphQL _internally_ as a wire protocol. However, as a developer, you _never_ touch this internal GraphQL layer and are not even supposed to be aware of it (you actually have to jump through a lot of hoops to even "find" it). It's also very likely that we'll replace GraphQL as a wire protocol in the future, so "GraphQL" really isn't something you should be thinking about as a developer who is using Prisma. Hope that clarifies the situation a bit, let me know if you have any further questions around this topic. [1] https://github.com/prisma/prisma1/issues/5208 https://github.com/prisma/prisma1/issues/5208
- searchableguy 4y ago
  Is there any post which goes into evolution of prisma from initial days of exposing graphql API directly to using it as wire protocol in the query engine? And now the likely removal of graphql as wire protocol. What are the key reasons to remove the use of graphql query language?
  - nikolasburk 4y ago
    > Is there any post which goes into evolution of prisma from initial days of exposing graphql API directly to using it as wire protocol in the query engine? To be honest, I think the most comprehensive resource for this is a Twitter thread [1] that I posted from my personal account that explains the different turns we took in the product direction :D Otherwise, there's yhr blog post "How Prisma & GraphQL fit together" [2] that also touches on the same topics but might be a bit dated since it was published before we released the Prisma ORM for production use. > And now the likely removal of graphql as wire protocol. What are the key reasons to remove the use of graphql query language? The short answer is that it would allow us to make us more optimizations for the bridge between JS- and Rust-land in Prisma Client. However, this is not an urgent issue for us at the moment so very likely won't become a priority in the near/mid-term future. [1] https://twitter.com/nikolasburk/status/1384908813069869058 https://twitter.com/nikolasburk/status/1384908813069869058 [2] https://www.prisma.io/blog/prisma-and-graphql-mfl5y2r7t49c https://www.prisma.io/blog/prisma-and-graphql-mfl5y2r7t49c

jensneuse 4y ago

As discussed in the post, the next steps are to add read replicas. Regarding backups, that's possible with Litestream: https://github.com/benbjohnson/litestream https://github.com/benbjohnson/litestream

gregwebs 4y ago
Are you going to use LiteFS then for replication [1]? LiteFS replication is asynchronous, meaning failover can lose the latest data. Will LiteFS scale down to 0? Does scaling down to zero mean electing a leader when scaling back up to 1 and will that have a delay? Will the read replicas scale down to 0 along with LiteFS when the primary scales down to 0? [1] https://github.com/superfly/litefs https://github.com/superfly/litefs
- benbjohnson 4y ago
  LiteFS/Litestream author here. LiteFS will scale to zero with a persistent volume attached. For short-lived instances (aka serverless), we still have a few more features to complete on the road map (e.g. S3 replication, synchronous replication) to make that work well. Pure serverless (e.g. Lambda, Vercel) is also something we plan to support but we want to get LiteFS working well on more traditional deployments (e.g. longer running instances, Kubernetes, etc) first.

lbhdc 4y ago

I was pretty interested to see how this worked as well. I think you are right, this is a toy. It will be interesting to see if they can solve scaling. It would be cool if you could ditch the graphql layer. It seems like there are other alternatives that go the vfs route so you still get to use a standard sqlite client.

tptacek 4y ago

You can, of course, scale to >1 with the Fly Machines API. I don't know enough about how they're managing the SQLite part of this to say more about how this scales, except that I think scaling out SQLite is about to get a lot more interesting. But I mostly agree that we need a better term than "serverless" for this kind of stuff. The big things people seem to want from "serverless" solutions are "not managing long-running server instances" and "true usage-metered billing". Whether or not there are servers, like, at all has not all that much to do with things.

gregwebs 4y ago
This is SQLite, so how would you scale itr to > 1? Certainly you can put an app tier in front of this DB tier and scale the app tier to infinity.
- tptacek 4y ago
  By replicating the SQLite transactions to other SQLite databases.
  - soamv 4y ago
    Hey, I like SQLite and I also like fly.io, but "distributed DB built with sqlite as storage" is really a very different beast from just "sqlite".
    - tptacek 4y ago
      I'm mostly interested in the SQLite part of this, and we don't have an SQLite offering, only Postgres, just to be clear. So you can't hurt my feelings here. When does SQLite become "a distributed DB built with sqlite as storage"? Did Postgres stop being Postgres when someone plugged log shipping into it? That's basically what we're talking about here --- not stuff like rqlite, which I'm also pretty interested in, but which really is a new database built on SQLite.
      - 4y ago

ithrow 4y ago

Couldn't find anything in the Prisma docs about it exposing a GraphQL API.

jensneuse 4y ago
https://github.com/prisma/prisma-engines#query-engine https://github.com/prisma/prisma-engines#query-engine

nikolasburk 4y ago

Hey there, I'm Nikolas from the Prisma team. Just came here to quickly clarify this notion: > Prisma is an API server that puts a GraphQL API in front of a DB. Prisma is an ORM which generates a JavaScript/TypeScript client library for your database. Your description is very true for Prisma 1 (which has been in maintenance mode for several years and is officially deprecated by now [1]), but the latest version(s) of Prisma (v2+) don't expose a GraphQL API any more. Prisma 1 also used GraphQL SDL for data modeling, the Prisma ORM on the other hand has its own, custom modeling language for describing database schemas in a declarative way and also comes with a flexible migration system. That being said (and as Jens also mentioned elsewhere), the Prisma ORM does use GraphQL _internally_ as a wire protocol. However, as a developer, you _never_ touch this internal GraphQL layer and are not even supposed to be aware of it (you actually have to jump through a lot of hoops to even "find" it). It's also very likely that we'll replace GraphQL as a wire protocol in the future, so "GraphQL" really isn't something you should be thinking about as a developer who is using Prisma. Hope that clarifies the situation a bit, let me know if you have any further questions around this topic. [1] https://github.com/prisma/prisma1/issues/5208 https://github.com/prisma/prisma1/issues/5208

searchableguy 4y ago
Is there any post which goes into evolution of prisma from initial days of exposing graphql API directly to using it as wire protocol in the query engine? And now the likely removal of graphql as wire protocol. What are the key reasons to remove the use of graphql query language?
- nikolasburk 4y ago
  > Is there any post which goes into evolution of prisma from initial days of exposing graphql API directly to using it as wire protocol in the query engine? To be honest, I think the most comprehensive resource for this is a Twitter thread [1] that I posted from my personal account that explains the different turns we took in the product direction :D Otherwise, there's yhr blog post "How Prisma & GraphQL fit together" [2] that also touches on the same topics but might be a bit dated since it was published before we released the Prisma ORM for production use. > And now the likely removal of graphql as wire protocol. What are the key reasons to remove the use of graphql query language? The short answer is that it would allow us to make us more optimizations for the bridge between JS- and Rust-land in Prisma Client. However, this is not an urgent issue for us at the moment so very likely won't become a priority in the near/mid-term future. [1] https://twitter.com/nikolasburk/status/1384908813069869058 https://twitter.com/nikolasburk/status/1384908813069869058 [2] https://www.prisma.io/blog/prisma-and-graphql-mfl5y2r7t49c https://www.prisma.io/blog/prisma-and-graphql-mfl5y2r7t49c