4 ms·

If anyone from cloudflare comes here - it's not possible to create D1 databases on the fly and interact them because databases must be mentioned in the worker b

by wg0 11mo ago

If anyone from cloudflare comes here - it's not possible to create D1 databases on the fly and interact them because databases must be mentioned in the worker bindings. This hampers the per user databases workflow. Would be awesome if a fix lands.

ashwindharne 11mo ago
I'm always a little hesitant to use D1 due to some of these constraints. I know I may not ever hit 10GB for some of my side projects so I just neglect sharding, but also it unsettles me that it's a hard cap.
dom96 11mo ago
(I work at Cloudflare, but not on D1) I believe this is possible, you can create D1 databases[1] using Cloudflare's APIs and then deploy a worker using the API as well[2]. 1 - https://developers.cloudflare.com/api/resources/d1/subresources/database/methods/create/ https://developers.cloudflare.com/api/resources/d1/subresour... 2 - https://developers.cloudflare.com/api/resources/workers/subresources/scripts/methods/update/ https://developers.cloudflare.com/api/resources/workers/subr...
- wg0 11mo ago
  Thank you! That's great and it is possible but... With some limitations. The idea is from sign up form to a D1 Database that can be accessed from the worker itself. That's not possible without updating worker bindings like you showed and further - there is an upper limit of 5000 bindings per worker and just 5000 users then becomes the upper limit although D1 allows 50,000 databases easily with further possible by requesting a limit increase. edit: Missed opening.
- ewuhic 11mo ago
  Hey, would you happen to know if/when D1 can get support for ICU (https://sqlite.org/src/dir/ext/icu https://sqlite.org/src/dir/ext/icu) and transactions?
  - kentonv 11mo ago
    Transactions are supported in Durable Objects. In fact, with DO you are interacting with the SQLite database locally and synchronously, so transactions are essentially free with no possibility of conflicts and no worry about blocking other queries. Extensions are easy to enable, file a bug on https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd . (Though this one might be trickier than most as we might have to do some build engineering.)
    - ewuhic 11mo ago
      Re DO - I am definitely not rewriting my web wasm rust-sqlx app to use DO. Re filing an issue - sounds straightforward, will do!
      - kentonv 11mo ago
        You can run your wasm app in a DO, same as you run it in a Worker.
educhana 11mo ago
Why not durable objects? I think it's the recommended pattern for having a db per user
kentonv 11mo ago
Try Durable Objects. D1 is actually just a thin layer over Durable Objects. In the past D1 provided a lot of DX benefits like better observability, but those are increasingly being merged back into DO directly. What is a Durable Object? It's just a Worker that has a name, so you can route messages specifically to it from other Workers. Each one also has its own SQLite database attached. In fact, the SQLite database is local, so you can query it synchronously (no awaits), which makes a lot of stuff faster and easier. You can easily create millions of Durable Objects. (I am the lead engineer for Workers.)

by wg0 11mo ago

ashwindharne 11mo ago

I'm always a little hesitant to use D1 due to some of these constraints. I know I may not ever hit 10GB for some of my side projects so I just neglect sharding, but also it unsettles me that it's a hard cap.

dom96 11mo ago

(I work at Cloudflare, but not on D1) I believe this is possible, you can create D1 databases[1] using Cloudflare's APIs and then deploy a worker using the API as well[2]. 1 - https://developers.cloudflare.com/api/resources/d1/subresources/database/methods/create/ https://developers.cloudflare.com/api/resources/d1/subresour... 2 - https://developers.cloudflare.com/api/resources/workers/subresources/scripts/methods/update/ https://developers.cloudflare.com/api/resources/workers/subr...

wg0 11mo ago
Thank you! That's great and it is possible but... With some limitations. The idea is from sign up form to a D1 Database that can be accessed from the worker itself. That's not possible without updating worker bindings like you showed and further - there is an upper limit of 5000 bindings per worker and just 5000 users then becomes the upper limit although D1 allows 50,000 databases easily with further possible by requesting a limit increase. edit: Missed opening.
ewuhic 11mo ago
Hey, would you happen to know if/when D1 can get support for ICU (https://sqlite.org/src/dir/ext/icu https://sqlite.org/src/dir/ext/icu) and transactions?
- kentonv 11mo ago
  Transactions are supported in Durable Objects. In fact, with DO you are interacting with the SQLite database locally and synchronously, so transactions are essentially free with no possibility of conflicts and no worry about blocking other queries. Extensions are easy to enable, file a bug on https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd . (Though this one might be trickier than most as we might have to do some build engineering.)
  - ewuhic 11mo ago
    Re DO - I am definitely not rewriting my web wasm rust-sqlx app to use DO. Re filing an issue - sounds straightforward, will do!
    - kentonv 11mo ago
      You can run your wasm app in a DO, same as you run it in a Worker.

educhana 11mo ago

Why not durable objects? I think it's the recommended pattern for having a db per user

kentonv 11mo ago

Try Durable Objects. D1 is actually just a thin layer over Durable Objects. In the past D1 provided a lot of DX benefits like better observability, but those are increasingly being merged back into DO directly. What is a Durable Object? It's just a Worker that has a name, so you can route messages specifically to it from other Workers. Each one also has its own SQLite database attached. In fact, the SQLite database is local, so you can query it synchronously (no awaits), which makes a lot of stuff faster and easier. You can easily create millions of Durable Objects. (I am the lead engineer for Workers.)