5 ms·

Chiselstore: an embeddable, distributed SQLite for Rust

58 points by patrickdevivo 5y ago

ryanworl 5y ago
Am I correct that this orders and makes SQL statements durable via Raft and then executes them serially against a local SQLite database? How would you use this to perform a transaction, or at minimum a compare-and-swap operation against a row? There doesn't seem to be any binding between a client and a connection for you to implement transactions here.
- cube2222 5y ago
  I haven't read much into it, but since it's Raft, it's most certainly executing all writes/updates as well as consistent reads against a single leader node. All changes to the database would then be propagated through Raft using a commit log, which would be read and accepted by follower nodes. This is also reinforced by the fact, that nondeterministic operations are supported. Thus, everything around transactions and CAS's will work the same as it normally does, as it executes on the leader.
- otoolep 5y ago
  Yes, that's exactly how rqlite works. If I ever get around to adding proper transaction support it will have to overcome the challenge you identify. I believe it can be done, but it will require significant changes to the current design. But an explicit goal for rqlite has always been to do a certain set of things very well, before doing something new. So I'm cautious. Systems like Chiselstore and dqlite operate at a different architectural layer, so can offer transaction support. But require more programming to use. It's a trade off.
quaffapint 5y ago
Same idea as https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite I presume but in rust vs go?
- edoceo 5y ago
  I came here to mention rqlite too. It's awesome, and non-voting nodes let you replicate out to many, many nodes w/o much performance issues.
  - otoolep 5y ago
    Cheers, glad you like it -- great to hear.
- PeterCorless 5y ago
  And likewise, Dqlite for the C folks in the crowd: https://dqlite.io/ https://dqlite.io/ [Edit] The only thing that'd be different for Rust and/or C is the avoidance of "stop-the-world" GC pauses, which affect Java more, but are still present in Golang.
  - pstuart 5y ago
    dqlite has 2 other advantages over rqlite: it supports transactions, and more importantly (IMHO), it works with Go's native sql.DB handler.
    - otoolep 5y ago
      Yeah, I need to get around to creating a Go sql.DB layer. Been too busy adding to the database itself. :-) Soon!
      - pstuart 5y ago
        Please do! That was the deal breaker when I was trying to migrate an internal project to SQLite/rqlite (couldn't slap an ORM on it).
- stavros 5y ago
  Have you used this? Is it good? Having a distributed database seems like a great thing, but it kind of rubs me the wrong way that that database is SQLite; I associate it more with lightweight/in-process stuff, which is the opposite of distributed.
malkia 5y ago
I've looked at the https://github.com/chiselstrike/chiselstore/blob/main/proto/proto.proto https://github.com/chiselstrike/chiselstore/blob/main/proto/... - and calling the service "RPC" is not great if this is going to be bundled with other gRPC services - why not call it "chiselStore" or something more concrete? on a 2nd note, each method should have it's own Request Response, and ideally they should be suffixed same way. Returning same proto for multiple methods might break future compatibility - e.g. what if AppendEntries needs to return somehing else than Void? - ideally you make it from the start to return an empty AppendEntriesResponse, and then you can extend that proto. https://developers.google.com/protocol-buffers/docs/proto3#updating https://developers.google.com/protocol-buffers/docs/proto3#u... (edit again, also the package should not be called just "proto")
- returnfalse 5y ago
  They can replace Void with AppendEntriesResponse anytime they want and that won't break anything since the change is wire compatible.
  - malkia 5y ago
    Didn't know that! (Good to know, and thanks!)
DeepYogurt 5y ago
Isn't the whole point of SQLite to not be distributed?
- dymk 5y ago
  Not really, just that you can use it in-process with little to no configuration. But a distributed system using sqlite as the storage seems useful in its own right.
dochtman 5y ago
The announcement here talks more about the context: https://glaubercosta-11125.medium.com/728187331f53 https://glaubercosta-11125.medium.com/728187331f53 It also mentions rqlite and dqlite.
fulafel 5y ago
How's the security and security defaults and are these documented somewhere? Are the "cargo run" examples safe to run as is in a networked environment without random people on the internet pwning your cluster? Is there confidentiality & authentication in the inter node communications?

58 points by patrickdevivo 5y ago

Am I correct that this orders and makes SQL statements durable via Raft and then executes them serially against a local SQLite database? How would you use this to perform a transaction, or at minimum a compare-and-swap operation against a row? There doesn't seem to be any binding between a client and a connection for you to implement transactions here.

cube2222 5y ago
I haven't read much into it, but since it's Raft, it's most certainly executing all writes/updates as well as consistent reads against a single leader node. All changes to the database would then be propagated through Raft using a commit log, which would be read and accepted by follower nodes. This is also reinforced by the fact, that nondeterministic operations are supported. Thus, everything around transactions and CAS's will work the same as it normally does, as it executes on the leader.
otoolep 5y ago
Yes, that's exactly how rqlite works. If I ever get around to adding proper transaction support it will have to overcome the challenge you identify. I believe it can be done, but it will require significant changes to the current design. But an explicit goal for rqlite has always been to do a certain set of things very well, before doing something new. So I'm cautious. Systems like Chiselstore and dqlite operate at a different architectural layer, so can offer transaction support. But require more programming to use. It's a trade off.

quaffapint 5y ago

Same idea as https://github.com/rqlite/rqlite https://github.com/rqlite/rqlite I presume but in rust vs go?

edoceo 5y ago
I came here to mention rqlite too. It's awesome, and non-voting nodes let you replicate out to many, many nodes w/o much performance issues.
- otoolep 5y ago
  Cheers, glad you like it -- great to hear.
PeterCorless 5y ago
And likewise, Dqlite for the C folks in the crowd: https://dqlite.io/ https://dqlite.io/ [Edit] The only thing that'd be different for Rust and/or C is the avoidance of "stop-the-world" GC pauses, which affect Java more, but are still present in Golang.
- pstuart 5y ago
  dqlite has 2 other advantages over rqlite: it supports transactions, and more importantly (IMHO), it works with Go's native sql.DB handler.
  - otoolep 5y ago
    Yeah, I need to get around to creating a Go sql.DB layer. Been too busy adding to the database itself. :-) Soon!
    - pstuart 5y ago
      Please do! That was the deal breaker when I was trying to migrate an internal project to SQLite/rqlite (couldn't slap an ORM on it).
stavros 5y ago
Have you used this? Is it good? Having a distributed database seems like a great thing, but it kind of rubs me the wrong way that that database is SQLite; I associate it more with lightweight/in-process stuff, which is the opposite of distributed.

malkia 5y ago

I've looked at the https://github.com/chiselstrike/chiselstore/blob/main/proto/proto.proto https://github.com/chiselstrike/chiselstore/blob/main/proto/... - and calling the service "RPC" is not great if this is going to be bundled with other gRPC services - why not call it "chiselStore" or something more concrete? on a 2nd note, each method should have it's own Request Response, and ideally they should be suffixed same way. Returning same proto for multiple methods might break future compatibility - e.g. what if AppendEntries needs to return somehing else than Void? - ideally you make it from the start to return an empty AppendEntriesResponse, and then you can extend that proto. https://developers.google.com/protocol-buffers/docs/proto3#updating https://developers.google.com/protocol-buffers/docs/proto3#u... (edit again, also the package should not be called just "proto")

returnfalse 5y ago
They can replace Void with AppendEntriesResponse anytime they want and that won't break anything since the change is wire compatible.
- malkia 5y ago
  Didn't know that! (Good to know, and thanks!)

DeepYogurt 5y ago

Isn't the whole point of SQLite to not be distributed?

dymk 5y ago
Not really, just that you can use it in-process with little to no configuration. But a distributed system using sqlite as the storage seems useful in its own right.

dochtman 5y ago

The announcement here talks more about the context: https://glaubercosta-11125.medium.com/728187331f53 https://glaubercosta-11125.medium.com/728187331f53 It also mentions rqlite and dqlite.

fulafel 5y ago

How's the security and security defaults and are these documented somewhere? Are the "cargo run" examples safe to run as is in a networked environment without random people on the internet pwning your cluster? Is there confidentiality & authentication in the inter node communications?