5 ms·

Hi RayVR, being built in Rust definitely isn't a main selling point, but it has made building SurrealDB easier. We initially built our prototype in Golang, but

Hi RayVR, being built in Rust definitely isn't a main selling point, but it has made building SurrealDB easier. We initially built our prototype in Golang, but decided to re-write it in Rust so that we could improve on certain aspects that were hard to do with Golang. That aside, we wanted to create a database that people didn't have to manage, so they could focus on building applications, not the infrastructure. We wanted users to be able to use schema-less and schema-full data patterns effortlessly, a database to operate like a relational database (without the JOINs), but with the same functionality as the best document and graph databases. And with security and access permissions to be handled right within the database itself. We wanted users to be able to build modern real-time applications effortlessly - right from Chrome, Edge, or Safari. No more complicated backends. It's only the beginning of our journey (we're just a bootstrapped team of 2 at the moment), and we welcome all feedback!

arunc 4y ago
> certain aspects that were hard to do with Golang. Do you mind sharing those aspects?
- tobiemh 4y ago
  Absolutely arunc! You can actually see the Golang version - it's on a separate branch in the main repository. We had written our query parser from scratch (which would read character by character). This worked well, up to a point, but was hard to extend. Building in Rust we've used the nom crate for parsing, and it has enabled us to do some pretty clever and advanced stuff with the SurrealQL query language (futures, embedded JavaScript functions, while at the same time remaining performant. Other things we wrote from scratch (at the time), were our own binary serialisation framework (similar to MsgPack, but with slight differences and performance improvements). We then had to write our own serialisation logic on top of this, so that each data type would serialise and deserialise efficiently. With Rust, using the Serde crate, this is done completely differently, and allows us to have the same performance (or better), but with no custom writing of data tags! In addition, many other improvements, especially around generics mainly. It made building SurrealDB much quicker, but it also means that all data passed around throughout the system is able to be reasoned about easier. Instead of brute-forcing with the race checker, you really have to have a concrete idea of how and where your data will be used. This starts out harder (fighting against the borrow checker), but in the long run it makes writing safe and performant software so much more pleasant. I'll try and go into this in detail in one of our future blog posts!
  - arunc 4y ago
    Fantastic, thank you!!
debuggerpk 4y ago
Would love to hear about the pain points you had with go.
- tobiemh 4y ago
  Absolutely debuggerpk. My answer to arunc above goes into it in slight detail! We will definitely plan to do an in-depth look at the decision to transfer from Golang to Rust in a future blog post!

arunc 4y ago

> certain aspects that were hard to do with Golang. Do you mind sharing those aspects?

tobiemh 4y ago
Absolutely arunc! You can actually see the Golang version - it's on a separate branch in the main repository. We had written our query parser from scratch (which would read character by character). This worked well, up to a point, but was hard to extend. Building in Rust we've used the nom crate for parsing, and it has enabled us to do some pretty clever and advanced stuff with the SurrealQL query language (futures, embedded JavaScript functions, while at the same time remaining performant. Other things we wrote from scratch (at the time), were our own binary serialisation framework (similar to MsgPack, but with slight differences and performance improvements). We then had to write our own serialisation logic on top of this, so that each data type would serialise and deserialise efficiently. With Rust, using the Serde crate, this is done completely differently, and allows us to have the same performance (or better), but with no custom writing of data tags! In addition, many other improvements, especially around generics mainly. It made building SurrealDB much quicker, but it also means that all data passed around throughout the system is able to be reasoned about easier. Instead of brute-forcing with the race checker, you really have to have a concrete idea of how and where your data will be used. This starts out harder (fighting against the borrow checker), but in the long run it makes writing safe and performant software so much more pleasant. I'll try and go into this in detail in one of our future blog posts!
- arunc 4y ago
  Fantastic, thank you!!

debuggerpk 4y ago

Would love to hear about the pain points you had with go.

tobiemh 4y ago
Absolutely debuggerpk. My answer to arunc above goes into it in slight detail! We will definitely plan to do an in-depth look at the decision to transfer from Golang to Rust in a future blog post!