4 ms·

perhaps I'm daft, but why should I care that you've written it in Rust? The number of times Rust is mentioned on the landing page feels...odd. I love shiny new

by RayVR 4y ago

perhaps I'm daft, but why should I care that you've written it in Rust? The number of times Rust is mentioned on the landing page feels...odd. I love shiny new things. However, without some sense of how this compares to alternatives, it's difficult to see why I should dive down the surrealdb rabbit hole. Feature lists are ok, but what I really want is context. Why should I care about what surrealdb can do?

tobiemh 4y ago
Hi RayVR, being built in Rust definitely isn't a main selling point, but it has made building SurrealDB easier. We initially built our prototype in Golang, but decided to re-write it in Rust so that we could improve on certain aspects that were hard to do with Golang. That aside, we wanted to create a database that people didn't have to manage, so they could focus on building applications, not the infrastructure. We wanted users to be able to use schema-less and schema-full data patterns effortlessly, a database to operate like a relational database (without the JOINs), but with the same functionality as the best document and graph databases. And with security and access permissions to be handled right within the database itself. We wanted users to be able to build modern real-time applications effortlessly - right from Chrome, Edge, or Safari. No more complicated backends. It's only the beginning of our journey (we're just a bootstrapped team of 2 at the moment), and we welcome all feedback!
- arunc 4y ago
  > certain aspects that were hard to do with Golang. Do you mind sharing those aspects?
  - tobiemh 4y ago
    Absolutely arunc! You can actually see the Golang version - it's on a separate branch in the main repository. We had written our query parser from scratch (which would read character by character). This worked well, up to a point, but was hard to extend. Building in Rust we've used the nom crate for parsing, and it has enabled us to do some pretty clever and advanced stuff with the SurrealQL query language (futures, embedded JavaScript functions, while at the same time remaining performant. Other things we wrote from scratch (at the time), were our own binary serialisation framework (similar to MsgPack, but with slight differences and performance improvements). We then had to write our own serialisation logic on top of this, so that each data type would serialise and deserialise efficiently. With Rust, using the Serde crate, this is done completely differently, and allows us to have the same performance (or better), but with no custom writing of data tags! In addition, many other improvements, especially around generics mainly. It made building SurrealDB much quicker, but it also means that all data passed around throughout the system is able to be reasoned about easier. Instead of brute-forcing with the race checker, you really have to have a concrete idea of how and where your data will be used. This starts out harder (fighting against the borrow checker), but in the long run it makes writing safe and performant software so much more pleasant. I'll try and go into this in detail in one of our future blog posts!
    - arunc 4y ago
      Fantastic, thank you!!
- debuggerpk 4y ago
  Would love to hear about the pain points you had with go.
  - tobiemh 4y ago
    Absolutely debuggerpk. My answer to arunc above goes into it in slight detail! We will definitely plan to do an in-depth look at the decision to transfer from Golang to Rust in a future blog post!
zinxq 4y ago
In the context of "History doesn't repeat, but it rhymes", this is just what happened when Java appeared. Everyone noting it was written in Java (as, believe it or not, it was the cool new language). It might have actually been worse, because there was a strong penchant for naming products starting with a "J" to indicate the fact (i.e. JNotepad, JDatabase, etc). It might actually be a good marketing technique to get other Rust aficionados to try your product. But otherwise, there isn't any real value now that "write once, run anywhere" doesn't just belong to Java anymore.
- arunc 4y ago
  Even Javascript rode on the Java hype/marketing albeit having no correlation with it.
  - gpm 4y ago
    No real connection, but amusingly I think javascript ended up delivering on java's "run anywhere" pitch at least as well as java did in the end. I can run javascript in a query to this DB apparently for instance, I can't do that with java.
dmytrish 4y ago
From the operations point of view: Rust code is usually fast, safe, free from data races, easy on RAM, possible to optimize very well, compiles to native binaries (easy to deploy). Unlike C, it's easy to test and provides good abstractions, so the code generally will be less buggy. From the point of view of a developer: when it comes to extending the database, writing an extension in Rust might be a pleasant and welcome experience. Rust interfaces with C/Python/Javascript easily and compiles to WASM, so the code written in it can be easily reused on the front end. What's not to like?
- tobiemh 4y ago
  Hi dmytrish that's exactly it! We are really close to releasing our WebAssembly driver (built in Rust), a native Python driver which can connect to and embed SurrealDB (built in Rust), and a native Node.js driver which has the same functionality as the Python driver (built in Rust). The plan is to do the same for the C driver in due course too. The simplicity of what you described means that we can release a common set of libraries with similar functionality, but using a single common library.
gpm 4y ago
It mentions rust 3 times on a very very long landing page, and once is in a list of languages it has bindings for. That's really not "the number of times" your comment implies. It is worth mentioning in my opinion mostly because that you should care that it isn't written in - C/C++, because of security concerns - Java/Scala/<JVM family> because embedding these languages (and this is available as an embedded database) is a pain, also because of common (though not inevitable) memory bloat. - Python/Ruby/..., because of common performance issues (DBs being performance sensitive software) Obviously there is perfectly good software written in all those languages, and terrible software written in rust. The the odds of it being good however are substantially increased when you find out it's written in rust because for this domain, out of the common programming languages, rust really does have the best set of tradeoffs. A landing pages goal is to give you as many signals that "this software might be worthwhile" as possible, and being written in Rust (or really not being written in pretty much every other common programming language) signals that right now.
- dataflow 4y ago
  > It mentions rust 3 times on a very very long landing page, and once is in a list of languages it has bindings for. That's really not "the number of times" your comment implies. I count 7 times. Once with the cargo install, once "built entirely in Rust", once "built with Rust", three times quoted under "Loved by Developers", and once under "Database Libraries and Clients".
bluejekyll 4y ago
Did the link change? I just read through that landing page and didn’t really notice Rust mentioned. It’s mention a bit on the GitHub readme, but that would be expected as getting started with the code? Also, people should care about the language something is written in, especially in OSS, as that implies a number of things. Specifically, what the ecosystem of that language is like in terms of maturity, how easy it is for you as a developer to get started with the project, what the build tool chain is and will work in your environment. I know I disqualify projects that I will look at purely based on the language and toolchain it uses since I know it will be harder to integrate into my companies workflows.

by RayVR 4y ago

tobiemh 4y ago

Hi RayVR, being built in Rust definitely isn't a main selling point, but it has made building SurrealDB easier. We initially built our prototype in Golang, but decided to re-write it in Rust so that we could improve on certain aspects that were hard to do with Golang. That aside, we wanted to create a database that people didn't have to manage, so they could focus on building applications, not the infrastructure. We wanted users to be able to use schema-less and schema-full data patterns effortlessly, a database to operate like a relational database (without the JOINs), but with the same functionality as the best document and graph databases. And with security and access permissions to be handled right within the database itself. We wanted users to be able to build modern real-time applications effortlessly - right from Chrome, Edge, or Safari. No more complicated backends. It's only the beginning of our journey (we're just a bootstrapped team of 2 at the moment), and we welcome all feedback!

arunc 4y ago
> certain aspects that were hard to do with Golang. Do you mind sharing those aspects?
- tobiemh 4y ago
  Absolutely arunc! You can actually see the Golang version - it's on a separate branch in the main repository. We had written our query parser from scratch (which would read character by character). This worked well, up to a point, but was hard to extend. Building in Rust we've used the nom crate for parsing, and it has enabled us to do some pretty clever and advanced stuff with the SurrealQL query language (futures, embedded JavaScript functions, while at the same time remaining performant. Other things we wrote from scratch (at the time), were our own binary serialisation framework (similar to MsgPack, but with slight differences and performance improvements). We then had to write our own serialisation logic on top of this, so that each data type would serialise and deserialise efficiently. With Rust, using the Serde crate, this is done completely differently, and allows us to have the same performance (or better), but with no custom writing of data tags! In addition, many other improvements, especially around generics mainly. It made building SurrealDB much quicker, but it also means that all data passed around throughout the system is able to be reasoned about easier. Instead of brute-forcing with the race checker, you really have to have a concrete idea of how and where your data will be used. This starts out harder (fighting against the borrow checker), but in the long run it makes writing safe and performant software so much more pleasant. I'll try and go into this in detail in one of our future blog posts!
  - arunc 4y ago
    Fantastic, thank you!!
debuggerpk 4y ago
Would love to hear about the pain points you had with go.
- tobiemh 4y ago
  Absolutely debuggerpk. My answer to arunc above goes into it in slight detail! We will definitely plan to do an in-depth look at the decision to transfer from Golang to Rust in a future blog post!

zinxq 4y ago

In the context of "History doesn't repeat, but it rhymes", this is just what happened when Java appeared. Everyone noting it was written in Java (as, believe it or not, it was the cool new language). It might have actually been worse, because there was a strong penchant for naming products starting with a "J" to indicate the fact (i.e. JNotepad, JDatabase, etc). It might actually be a good marketing technique to get other Rust aficionados to try your product. But otherwise, there isn't any real value now that "write once, run anywhere" doesn't just belong to Java anymore.

arunc 4y ago
Even Javascript rode on the Java hype/marketing albeit having no correlation with it.
- gpm 4y ago
  No real connection, but amusingly I think javascript ended up delivering on java's "run anywhere" pitch at least as well as java did in the end. I can run javascript in a query to this DB apparently for instance, I can't do that with java.

dmytrish 4y ago

From the operations point of view: Rust code is usually fast, safe, free from data races, easy on RAM, possible to optimize very well, compiles to native binaries (easy to deploy). Unlike C, it's easy to test and provides good abstractions, so the code generally will be less buggy. From the point of view of a developer: when it comes to extending the database, writing an extension in Rust might be a pleasant and welcome experience. Rust interfaces with C/Python/Javascript easily and compiles to WASM, so the code written in it can be easily reused on the front end. What's not to like?

tobiemh 4y ago
Hi dmytrish that's exactly it! We are really close to releasing our WebAssembly driver (built in Rust), a native Python driver which can connect to and embed SurrealDB (built in Rust), and a native Node.js driver which has the same functionality as the Python driver (built in Rust). The plan is to do the same for the C driver in due course too. The simplicity of what you described means that we can release a common set of libraries with similar functionality, but using a single common library.

gpm 4y ago

It mentions rust 3 times on a very very long landing page, and once is in a list of languages it has bindings for. That's really not "the number of times" your comment implies. It is worth mentioning in my opinion mostly because that you should care that it isn't written in - C/C++, because of security concerns - Java/Scala/<JVM family> because embedding these languages (and this is available as an embedded database) is a pain, also because of common (though not inevitable) memory bloat. - Python/Ruby/..., because of common performance issues (DBs being performance sensitive software) Obviously there is perfectly good software written in all those languages, and terrible software written in rust. The the odds of it being good however are substantially increased when you find out it's written in rust because for this domain, out of the common programming languages, rust really does have the best set of tradeoffs. A landing pages goal is to give you as many signals that "this software might be worthwhile" as possible, and being written in Rust (or really not being written in pretty much every other common programming language) signals that right now.

dataflow 4y ago
> It mentions rust 3 times on a very very long landing page, and once is in a list of languages it has bindings for. That's really not "the number of times" your comment implies. I count 7 times. Once with the cargo install, once "built entirely in Rust", once "built with Rust", three times quoted under "Loved by Developers", and once under "Database Libraries and Clients".

bluejekyll 4y ago

Did the link change? I just read through that landing page and didn’t really notice Rust mentioned. It’s mention a bit on the GitHub readme, but that would be expected as getting started with the code? Also, people should care about the language something is written in, especially in OSS, as that implies a number of things. Specifically, what the ecosystem of that language is like in terms of maturity, how easy it is for you as a developer to get started with the project, what the build tool chain is and will work in your environment. I know I disqualify projects that I will look at purely based on the language and toolchain it uses since I know it will be harder to integrate into my companies workflows.