10 ms·
Diesel: A Safe, Extensible ORM and Query Builder for Rust
- steveklabnik 11y agoSean Griffin (and others), the maintainer of Rails' ActiveRecord, has been working on this ORM for a while now. It's not a port of ActiveRecord, but a look at what an ORM that takes advantages of Rust's strengths would look like. It makes heavy usage of Rust's type system features, particularly zero-sized types, to provide a large degree of safety with no overhead. Sean has mentioned several times that he even believes that he is faster than literal SQL strings, due to pre-compiling them in some way, but that's a bit out of my league. He also wants to have features like "type-check all of your queries at compile time and then query the database to make sure that the schemas all line up" and other such compile-time safety shenanigans.
- schwap 11y agoI know the ".rs" TLD is common to rust projects, but maybe add "for Rust" to the title?
- baddox 11y agoI had no idea that .rs was conventionally for Rust projects, but I'm also not too bothered by having to figure it out.
- schwap 11y agoI'm not sure it's convention but it's common just due to matching the file extension.
- rabidferret 11y agoIt's technically for the Republic of Serbia. I only chose it because .io was taken by a python library (which I wish I'd checked before deciding on the name) It's also a pain in the ass to register because you have to have a Serbian tax ID
- 15155 11y agoHow does this reconcile the async IO problem? I couldn't figure out from cursory reading how "PgConnection" works or how it would be compatible with other IO-related code.
- rabidferret 11y agoI'm not sure I entirely understand the question, but we ultimately end up hooking into libpq, and right now the queries are synchronous. As we look towards features like associations, this will actually have to be true to some extent, since if we want the return type to be `(User, Vec<Post>)`, we actually need to allocate the entire result set we're reading into memory to implement `next()`.
- Manishearth 11y agoWould it be desirable or possible to instead return a lazy collection that only fetches perhaps 10 results and fetches more as you iterate?
- rabidferret 11y agoIt's impossible. We can't guarantee the order of the result set, so we'd have to iterate the entire collection just to implement `next`
- emergentcypher 11y agoThis is still one of Rust's major shortcomings. Async IO, particularly async database and network IO, are still pretty much nonexistent in any practical sense. Some libraries are in the works, but for the time being, I'm still passing over Rust for any real work.
- steveklabnik 11y agoIt's very close to landing in hyper, which should then percolate up through the rest of the ecosystem, at least, HTTP-wise.
- usaphp 11y agoReally nice home page, clearly explains what the product is for and how it works, I really wish more tools would have such landing pages, most of the times I am spending a lot of time just figuring out what the hell is that new tool about. Great job!
- ixtli 11y agoI am a big fan of typography but I have to draw the line at ligatures in code samples. It's just not appropriate for type that actually has a compelling reason to be monospaced =)
- rabidferret 11y agoMind posting a screenshot of what you're seeing? All our code samples are `font-family: monospace`
- bschwindHN 11y agoI think they're referring to the "fi" in filter: http://i.imgur.com/tJBeeNt.png http://i.imgur.com/tJBeeNt.png
- ixtli 11y ago^ This is what I'm seeing. Sorry it took so long :(
- anonova 11y agoIt's being inherited. // application.css:529 body { -webkit-font-feature-settings: "kern", "liga", "pnum"; -ms-font-feature-settings: "kern", "liga", "pnum"; font-feature-settings: "kern", "liga", "pnum"; // ... }
- erickt 11y agoIf anyone is interested in hearing from the creator, Sean Griffin will be talking about Diesel at the next Bay Area Rust Meetup on February 18th [1], which will be live streamed and eventually archived on Air Mozilla [2]: [1]: http://www.meetup.com/Rust-Bay-Area/events/219697075/ http://www.meetup.com/Rust-Bay-Area/events/219697075/ [2]: https://air.mozilla.org/bay-area-rust-meetup-february-2016/ https://air.mozilla.org/bay-area-rust-meetup-february-2016/
- dikaiosune 11y agoHe also did a good podcast interview about Diesel and about Rust (and what makes a Rust ORM special with compile time magic) a week or so ago: http://bikeshed.fm/49 http://bikeshed.fm/49
- jaytaylor 11y agoFYI It looks like the in-person slots for that meetup are already fully booked.
- deleted 11y ago[deleted]
- Animats 11y agoWhy is Post::belonging_to(user).load(&connection) an improvement over SELECT * FROM posts WHERE user_id = 1; ? If you use this, it nails the database schema into your Rust code, because this needs to know the table formats. If someone adds a new field to a row, you have to rebuild all your Rust programs that use the database. Now your deployment process just got much more complicated. It's a neat hack, but seems an overly complex way to do something that's not that complicated. Unless this has some huge performance gain over rust-postgres[1], it's probably too much. If rust-postgres is too slow, it would be worth having an optional generic for rust-postgres to help with unmarshalling returned rows. Rust-postgres returns row objects from which fields can be extracted. Converting that to a Rust struct is the only operation which needs to go fast. [1] https://github.com/sfackler/rust-postgres https://github.com/sfackler/rust-postgres
- rabidferret 11y agoIn that very specific example, you are right -- the benefits are pretty minimal (mostly just the deserialization). However, you're actually able to represent the relationship between those two models however you want, but still deal with relationships between types, and not what the exact join clause is. Additionally, SQL is extremely hard to compose. In my applications, I very rarely have some specific one off query that I want to run, I have multiple predicates that I want to be able to mix and match easily. String munging is fragile, while this actually lets me build reasonable domain abstractions. To your final point "Unless this has some huge performance gain over rust-postgres" Diesel performs 33% faster than rust-postgres, and will be competitive with if not faster than ideomatic C.
- BinaryIdiot 11y ago> Additionally, SQL is extremely hard to compose. I've seen almost everyone who doesn't use SQL directly say this but I always feel it's the exact opposite. SQL is easy to write once you do it enough. I've gotten to the point where I compose zero SQL in any application's code and, instead, do it all inside of stored procedures (of which they do not generate dynamic SQL). Then my code just calls stored procedures, my database users for the web application never have more access than executing a specific subset of stored procedures, and if there is a bug found in a stored procedure I can deploy them separately from the web application. I'm guessing it's just different philosophies but almost every time I've used something that abstracts an existing interface to another, completely different interface, I always end up running into major issues.
- dikaiosune 11y agoIn the getting started guide, it looks like they use an as-yet-non-existent Post struct to create the migration SQL. Where is the migration CLI getting the struct's fields from if it hasn't yet been written in that part of the tutorial? Am I missing something obvious?
- steveklabnik 11y agoYou're supposed to write that SQL yourself, I believe. I'll file an issue to clear this up either way.
- rabidferret 11y agoThe CLI just generates the file structure. We fill in the SQL manually as part of the guide, and then create the structs immediately after that.
- dikaiosune 11y agoWhat I get for reading too quickly! Thanks for clarifying.
- rabidferret 11y agoIt wasn't super clear, anyway. Fixed in https://github.com/sgrif/diesel.rs-website/pull/5 https://github.com/sgrif/diesel.rs-website/pull/5 :)
- kbenson 11y agoAn FYI to anyone responsible for this site, or in contact with them, the visited and unvisited link colors in the getting started guide[1] are very close to the background color, which makes them hard to read. Specifically, the link to dotenv and the CLI. 1: http://diesel.rs/guides/getting-started/ http://diesel.rs/guides/getting-started/
- rabidferret 11y agoThanks. The designer is taking another pass at it soon, I didn't expect to have so much traffic. I'll bump up the color on the links until then.
- lobster_johnson 11y agoVery cool. I like that it (as far as I can see) is actually built on top of an SQL AST. This is one of the shortcomings of ActiveRecord. The SQL building used to be extremely ad-hoc, and though ARel was eventually written to fix this problem, the integration between ARel and ActiveRecord is not very fluid (and ARel itself has some design issues; if I remember correctly from the last time I worked with it, it's not immutable, which can be very confusing when chaining expressions).
- rabidferret 11y agoYeah, I'm planning on killing Arel at some point. I actually want to extract `Relation` to a gem entirely, as it's really not an adequate query builder for what we need in AR. Also random fun story about Arel not being immutable. We had a bug in 4.1 where basically PG bind params are ordered, but Relation can't actually know the order ahead of time (e.g. because a Relation can be used in a from clause). Turned out actually fixing it was somewhat hard (the "real" fix was eventually https://github.com/rails/arel/commit/590c784a30b13153667f8db7915998d7731e24e5 https://github.com/rails/arel/commit/590c784a30b13153667f8db...). But as a hack, basically someone took advantage of the fact that `Arel::Nodes::BindParam < SqlLiteral` and `SqlLiteral < String`, and Strings are mutable. So they just ignored the lack of a proper map function, and fiddled with the strings. >_> Anyway, yeah our AST is fun because Rust's compiler lets us set it up so that we can walk it as many times as we need basically for free (because it gets monomorphised and the walking ultimately gets inlined and optimized away)
- kevindeasis 11y agoThis would be really awesome if someone used this with a rust framework being used in production It would be really cool if this had an easy integration with postgis that uses simple syntax to query location When I'm using a node framework and using postgres I have to use a nosql db for playing with coordinates as postgis syntax is not something i am very fond of
- deleted 11y ago[deleted]
- rabidferret 11y agoFWIW making sure it's easy to add extensions like this is part of the core design of Diesel. For example, this adds full text search support (it's slightly out of date) https://github.com/sgrif/diesel_full_text_search/blob/master/src/lib.rs https://github.com/sgrif/diesel_full_text_search/blob/master... If you're interested in adding postgis support, ping me in our Gitter room. I'd be happy to help.
- DannoHung 11y agoWhy did a NewPost struct have to be created to actually add a new Post? Is it generally not possible to have the result type of a query be used to update the same records?
- rabidferret 11y agoUpdate yes. It's possible to use the same struct for create, too if you really want, but you'd need to wrap `id` in an `Option`. Generally we're designed for the idea that you'll have a separate struct for read, create, and update. We also try and design for the latter two mapping roughly to web forms, as this is how most large applications end up wanting to be structured from my experience. The main thing I'm trying to avoid is lumping all your logic in a single place, and ending up in many of the same rabbit holes that Active Record has led us to (which is at least partially my fault)
- DannoHung 11y agoI guess that makes sense. The types should be able to ensure you don't have any errors even with so many "redundant" types, but my first thought is of how much boiler plate will result. Like, I would guess most record types will be very similar between create, update, and retrieve. Also, what about upsert?
- rabidferret 11y ago> The types should be able to ensure you don't have any errors even with so many "redundant" types Correct. Sorry, I should have been clearer in my last comment. The reason you'd want a separate type for insert is because wrapping your id in an `Option` would be painful for anything except insert. > but my first thought is of how much boiler plate will result. Yeah, if you're coming from Active Record (which is the other project I maintain), we'll probably seem a little bit verbose. My goal here is to be 80% as nice to get started with, but nudge users towards significantly more maintainable code as their product grows. > I would guess most record types will be very similar between create, update, and retrieve. Early on, yes. 2 years in, not so much. > Also, what about upsert? This is something I'd like to add support for in the future. This comment made me realize I didn't have a tracking issue, so here it is: https://github.com/sgrif/diesel/issues/196 https://github.com/sgrif/diesel/issues/196
- brandonbloom 11y agoHonest question: When would it ever really make sense to use Rust on the server for a service that interacts with a SQL database heavily enough to justify using such a library? Wouldn't waiting on the network eat up any possible performance gain you could expect from Rust vs something like the JVM or Go runtime? Before modern day JavaScript JITs, relational database systems were the epitome of dynamic runtime systems. There's so much performance variability in the query planner, network, disk caching, etc, to make garbage collection a rounding error. Zero-cost abstractions are just not a particularly useful thing in this context from what I can tell. Even if you do have genuinely CPU-bound tasks, why wouldn't you use Rust for those modules + call it via FFI from whatever is doing most of the boring business logic? If it's just about static-type-check-all-the-things! I just can't get excited about it vs something like Go.
- joseph 11y agoOn the other hand, if you're already using Rust, why switch to another language just to talk to the database?
- Animats 11y agoThere are much simpler standard ways to talk to a database from Rust: https://crates.io/crates/postgres/ https://crates.io/crates/postgres/ https://crates.io/crates/mysql/ https://crates.io/crates/mysql/ This is an experimental approach. It might or might not be useful, but you don't have to use it.
- brandonbloom 11y ago+1 It seems like the intersection of good use case for Rust and need to use SQL, will also intersect with "minimal SQL needs", hence preference for simpler libraries.
- deleted 11y ago[deleted]
- dutchrapley 11y agoI like how the logo uses a red gas can. Diesel fuel is actually goes in a yellow gas can.
- rabidferret 11y agoI think it's green, at least in the US. XD
- jdpedrie 11y agonope, diesel is yellow. never seen a green fuel can.
- rabidferret 11y agoI'll take your word for it. I'm no gas can expert. :)
- andrewaylett 11y agoFor plastic 'cans' in the UK, we use green for unleaded petrol. Red was for leaded petrol and diesel is black.
- gleb 11y agoHow does this compare to Elixir's Ecto?
- rabidferret 11y agoI'm having Jose on The Bikeshed (bikeshed.fm) next month to compare notes!
- dschiptsov 11y agoSafe from what?
- steveklabnik 11y agoType safe, memory safe.
- dschiptsov 11y agoHow is a strong-typed language, such as CL, is less safe? Edit: let's paraphrase - how excessive safety nets for construction workers makes buildings more safe? A strong-typed language without implicit coersions is not more "unsafe". Baning heterogeneous containers and conditionals, maybe-wrapping of everything to be able to check some constraints at compile time and catch simple errors (which also would be caught in a strong-typed language at run time) does not quality for this "safe" meme.
- AYBABTME 11y agoThis is pretty cool, makes me want to take a look at Rust.
- cetra3 11y agoCan you just use stock-standard parametrised SQL queries or do you have to use the ORM to build the SQL statement?
- Svenstaro 11y agoI would really like to know this, too!
- rabidferret 11y agoWe have a SQL literal ast node but you lose most of the benefits of the library that way
- frik 11y agoNaming something "Diesel" is problematic. While the name is well known because of Rudolf Diesel, a German inventor of the Diesel engine and the Diesel fuel. There is an Italian clothing company "Diesel S.p.A." that tries to protect their brand by suing almost anyone using "Diesel". For example a German video game called "DieselStormers": due to a successful trademark lawsuit by Diesel S.p.A., who owns the trademark "Diesel" in relation to video games, Black Forest Games had been required to drop the title DieselStormers. http://www.eurogamer.net/articles/2015-09-30-dieselstormers-forced-to-change-name-after-diesel-wins-trademark-dispute http://www.eurogamer.net/articles/2015-09-30-dieselstormers-... Such greedy companies act very similar to patent trolls. Well, there are sadly several other common words that are tainted by such "brands". Edit: In 2015/2016 the Volkswagen (VW) Diesel scandal tainted the "Diesel" name too: http://autoweek.com/car-news/vw-diesel-scandal http://autoweek.com/car-news/vw-diesel-scandal
- walterstucco 11y agoChosing Diesel as a name in 2016 is not exactly smart. greedy companies or not
- tracker1 11y agoI don't quite get why nobody simply creates function bindings that have a query template string on one side, that is represented as a function that creates a parameterized query under the covers, and a solid type that represents a record that is returned? I mean instead of jumping through all the binding chains.. just have a string and type... call the generated function based on the string, return an array/stream/whatever of <type> I'd rather use template string generated promises in node, tbh, than any ORM.
- rabidferret 11y agoThat is our `sql` function
- rabidferret 11y agoThat is our `sql` function
- rabidferret 11y agoThat is our `sql` function