9 ms·
Toasty, an async ORM for Rust
- alilleybrinker 2y agoVery interested in exploring how this will compare to Diesel [1] and SeaORM [2], the other two options in this space today. Joshua Mo at Shuttle did a comparison between Diesel and SeaORM in January of this year that was really interesting [3]. [1]: https://diesel.rs/ https://diesel.rs/ [2]: https://www.sea-ql.org/SeaORM/ https://www.sea-ql.org/SeaORM/ [3]: https://www.shuttle.dev/blog/2024/01/16/best-orm-rust https://www.shuttle.dev/blog/2024/01/16/best-orm-rust
- karunamurti 2y agoSea ORM is too opinionated in my experience. Even making migration is not trivial with their own DSL. Diesel was ok, but I never use it anymore since rocket moved to async. I'm mainly use sqlx, it's simple to use, there's query! and query_as! macro which is good enough for most of the case.
- sverro2 2y agoI like sqlx, but have been eyeing diesel for some time. Any reasons you don't use diesel_async?
- malodyets 2y agoWith Diesel async integrating everything with the pooling is a bit hairy. With sqlx everything just works.
- sampullman 2y agoI use SQLx, but I'm not totally convinced it's better than writing raw SQL with the underlying postgres/sqlite3/mysql driver. The macros and typing fall apart as soon as you need anything more complicated than basic a SELECT with one to one relationships, much less one/many to many. I remember fighting with handling enums in relations for a while, and now just default to manually mapping everything.
- echelon 2y agoSQLx can handle complicated queries as long as they're completely static strings. We've got SELECT FOR UPDATE, upserts, and some crazy hundred-line queries that are fine with their macros. SQLx sucks at dynamic queries. Dynamic predicates, WHERE IN clauses, etc. For SQLx to be much more useful, their static type checker needs to figure out how to work against these. And it needs a better query builder DSL.
- sampullman 2y agoRight, it's not bad if you stick with what the type checker can handle, but I usually end up falling back on manual building with the majority of queries in any semi-complex app. It doesn't end up being too bad though, except for the loss of compile time syntax checking. Manually handling joins can be kind of nice, it's easier to see optimizations when everything is explicit.
- Onavo 2y agoIt's nice seeing more Django/Prisma style ORMs where the non-SQL source code is the source of truth for the schema and migrations are automatically generated.
- tuetuopay 2y agoMy first reaction is this feels like a nice middleground between Diesel and SeaORM. The codegen part makes all columns and tables and stuff checked at compile-time (name and type) like Diesel, with a query builder that's more natural like SeaORM. I hope the query builder does not end up too magical like SQLAlchemy with its load of footguns, and stay close in spirit to Diesel that's "write sql in rust syntax". I think time will tell, and for now I'm keeping my Diesel in production :D
- arandomusername 2y agoLooks awesome. Would love to see the table definitions that are generated from the schema as well.
- mbrumlow 2y ago[flagged]
- thimabi 2y agoI concur. I’ve used ORMs in certain projects, but iterating and optimizing them later on became untenable. I’ve learned my lesson — all my new projects forgo ORMs by default.
- Daishiman 2y agoHow? ORM logic is as simple or as complicated as the underlying queries and results. In 15 years of building web apps with Python ORMs this has never been a problem across many, many different teams.
- _hcuq 2y agoYep, plus learn SQL and you can use it everywhere. Every language has it's own (multiple) language specific ORMs.
- bb88 2y agoAnd yet, the Vietnam of Computer Science paper didn't stop the creation of ORM's. There's something fundamentally broken with SQL syntax, and yelling at people to "Just Use SQL" doesn't really help.
- mbrumlow 2y agoIts better than sitting silent and doing nothing. What it does is validate other peoples feelings who feel the same way. And just maybe, that validation will lead to them saying "fuck ORMs" when some junior dev comes in and tries to use one.
- bb88 2y agoFair, but it also reinforces the stereotype that the pro-SQL types are obstinate relics yelling at the youngsters to "Get off my lawn!" There's a pattern of "We can't change X, so we'll write Y that transpiles down to X". It happens often with closed source tools and others that can't or won't implement new languages. Verilog, SQL, Javascript, all fit that bill. E.g. Why is Javascript the only first class language for the browser? For the longest time JS was the only game in town.
- revskill 2y agoO in orm is misleading term. To me orm is about table.
- OJFord 2y agoO refers to the OOP object, it's the R that's the (relational) database.
- deleted 2y ago[deleted]
- randomdata 2y agoI expect the parent meant to write R instead of O. It is misleading as nobody ever maps relational databases. As the parent points out, usually they reach for tablational databases. This project also supports a key/value database. But it does not even support relational databases. Further, the project is focused on implementing the active record pattern, so it would be more appropriately called an async active record than an "async ORM".
- 0x457 2y agobut active record is just one of patterns that can be used to implement an ORM?
- randomdata 2y agoActive record traditionally depends on ORM, but cannot implement it. ORM exists at a lower layer of abstraction.
- colesantiago 2y agoI don't get the pent up anger with ORMs, I used it for my SaaS on Flask that I run and own for 4 years bringing in over $2M+ ARR with no issues. Great to see some development in this for Rust, perhaps after it becomes stable I may even switch my SaaS to it.
- kyleee 2y agoHas it benefited you? Have you moved to a different underlying SQL software without having to make any changes to your codebase? Or some other benefit?
- carlgreene 2y agoFor me it’s speed of development. I’m frankly not very good at SQL, but an ORM in a familiar syntax to the language I use most (Typescript) increases my dev speed tremendously. I also have a relatively successful saas that uses Prisma and it’s been phenomenal. Queries are more than fast enough for my use case and it allows me to just focus on writing more difficult business logic than dealing with complex joins
- jeremyloy_wt 2y agoThe second that you would benefit from using a DBMS specific feature, the ORM begins getting in the way. It is highly unlikely that an ORM provides support, much less a good abstraction, over features that only 1/N supported DBMS have. Your code ends up using the driver raw in these cases, so why not just use the driver for everything? Your codebase would be consistent at that point
- viraptor 2y agoBecause you only need the specific features in a tiny amount of cases, while 99% is some flavour of SELECT * ... LEFT JOIN ... (If it's not, then sure, ORM would be annoying) Making that 99% smaller, simpler and automatically mapping to common types makes development a lot easier/faster. This applies to pretty much any higher level language. It's why you can write in C, but embed an ASM fragment for that one very specific thing instead of going 100% with either one.
- Sytten 2y agoFor me diesel hits right balance since it is more a query builder and it is close to the SQL syntax. But sometimes it doesn't work because it is very strongly typed, right now I use sea-query for those scenarios and I built the bridge between the two. Ideally I would use something akin to Go Jet.
- tricked 2y agoLooks well thought out i like that for the most part this seems faster/easier than rolling your own sql query mapping etc compared to the other solutions I've come across in rust
- cyndunlop 2y agoToasty was the focus of Carl Lerche's P99 CONF keynote on Wednesday. It provoked some interesting discussion in the chat.
- andrewstuart 2y agoThe days of the ORM have passed. AI writes amazing SQL, modern SQL databases are incredible and the best way to get the most out of your DB is write SQL. Invest your learning budget in SQL, not in some random developers abstraction.
- t-writescode 2y agoA good ORM is just a pre-built and externally tested validator of common DB work. There's plenty of value in knowing both. "AI writes amazing SQL" and "AI writes amazing DB to Application Translation Layer Code" just means "AI can write your company's bespoke ORM".
- benatkin 2y agoAI can give better responses to a lot of requests when it has a well designed high level API available. And many ORMs don’t produce well designed APIs but it seems this one will.
- andrewstuart 2y agoAi is as good as its training data and there’s vast sql documentation and source code that has been ingested by the AI engines.
- benatkin 2y agoI think that for some things user intent would be better expressed as customized ORM than SQL queries. If the ORM isn’t customized and is just generated from the tables then, yeah, not much of a help.
- randomdata 2y agoYou seem to be thinking of query builders. I would posit that they are still useful in many real-world scenarios because SQL still stupidly does not support composition. Maybe some day it will catch up with the 1950s, but until that day... ORM has nothing to do with queries. It is about the data. You will still want to map your database structures to your application structures (and vice-versa), else you'll end up with a weird and ugly chain of dependencies that will forever ruin your codebase.
- aabhay 2y agoInteresting take! In my experience, Dynamo and other NoSQL systems are really expressive and powerful when you take the plunge and make your own ORM. That’s because the model of nosql can often play much nicer with somewhat unique structures like - single table patterns - fully denormalized or graph style structures - compound sort keys (e.g. category prefixed) Because of that, I would personally recommend developing your own ORM layer, despite the initial cost
- smt88 2y agoWhy does a NoSQL or denormalized database need an ORM? Developing your own ORM is almost always a waste of time and a bad idea.
- aabhay 2y agoTrue, but there are benefits in some instances as well. For example, we store all rows as entity properties, not entities themselves. So a row would be the user’s email, one row for user name, etc. which makes it possible to do razor sharp queries over exactly what is needed. So while that doesn’t imply a standard ORM, if you want a `User` object you must write an ORM layer
- fulafel 2y agoDo you find that you value the relational model that a ORM constructs on top a non-relational DB? Or do you use it more like a "OM" without the R?
- aabhay 2y agoThat’s a great point. We don’t really use the R part so much. However, you can’t always avoid it. That said, if your concepts in the table themselves can be atomic or isolated then yes your object model can just be a wrapper of sorts that bundles convenience functionality around the row data.
- fulafel 2y agoWhy is asynchronity (sp?) a concern of the ORM in this case?
- BiteCode_dev 2y agoBecause ORM attribute access usually trigger requests, and you must design the API so that those requests, which trigger called to the network, don't block.
- baq 2y agothis has been a solved problem for... a long time, can't remember how long even. https://docs.sqlalchemy.org/en/20/orm/queryguide/relationships.html#preventing-unwanted-lazy-loads-using-raiseload https://docs.sqlalchemy.org/en/20/orm/queryguide/relationshi... the older I get the more I'm convinced this should be the default behavior.
- GolDDranks 2y agoI think you are talking past each others. Preventing N+1 by doing lazy fetching and having synchronous / asynchronous API are orthogonal issues. Async API must not block the thread/event loop when the data loading is being done. Diesel hasn't been providing an async API for reason told in this thread: https://github.com/diesel-rs/diesel/issues/399 https://github.com/diesel-rs/diesel/issues/399 The situation might change some day though, once async support in the core language and surrounding ecosystem gets stronger.
- satvikpendem 2y agodiesel_async exists and is also maintained by the same creator.
- GolDDranks 2y agoYes, but it started as a experiment, and isn't official part of Diesel yet.
- Ciantic 2y agoIt is nice to see more ORMs, but inventing a new file format and language `toasty` isn't my cup of tea. I'd rather define the models in Rust and let the generator emit more Rust files. Creating your own file format is always difficult. Now, you have to come up with syntax highlighting, refactoring support, go to definition, etc. When I prototype, I tend to rename a lot of my columns and move them around. That is when robust refactoring support, which the language's own LSP already provides, is beneficial, and this approach throws them all away.
- simonask 2y agoI mostly agree with this, but the trouble is (probably) that proc-macros are heavy-handed, inflexible, and not great for compile times. In this case, for example, it looks like the generated code needs global knowledge of related ORM types in the data model, and that just isn't supported by proc-macros. You could push some of that into the trait system, but it would be complex to the point where a custom DSL starts to look appealing. Proc-macros also cannot be run "offline", i.e. you can't commit their output to version control. They run every time the compiler runs, slowing down `cargo check` and rust-analyzer.
- trevyn 2y agoYou can absolutely do global knowledge in proc macros via the filesystem and commit their output to version control: https://github.com/trevyn/turbosql https://github.com/trevyn/turbosql
- simonask 2y agoYou can introduce side effects to a proc macro (but please avoid if at all possible), but you cannot control the order in which proc macros are run. If you need to reason about the global schema while generating code, that won’t work.
- deleted 2y ago[deleted]
- BluSyn 2y ago
- isodev 2y agoNice, I love it! It reminds me of Prisma and yet, it's all Rust. Also good to see that async is the focus point of the API so the usage feels ergonomic.
- OtomotO 2y agoORM has never worked for me in any language. Sooner or later we always hit the n+1 query problem which could only be resolved by a query builder or just plain old sql. It always was a mess and these days I can't be bothered to try it even anymore because it has cost me a lot of hours and money.
- srik 2y agoYes, plain sql is indeed the bees knees but there are good ORMs like django/ecto etc. that let you consider N+1 query issues ahead of time. Most ORMs these days have escape hatches anyway. Patience might be needed to keep it all tidy but they don't necessarily have to be a mess.
- s6af7ygt 2y agoI don't get why to use an ORM in the first place. Just define a bunch of structs, run a query, map results to structs. It's a few lines of simple code. You're in control of everything (the SQL, the running, the mapping). It's transparent. With any ORM, you give away control and make everything more complex, only to make it slightly easier to run a query and map some results.
- JodieBenitez 2y ago> Just define a bunch of structs, run a query, map results to structs Congrats, you now have your own little ORM.
- RandomThoughts3 2y agoNo, absolutely not. Op is never implying they intend to maintain one to one correspondence between the DB and objects and do that through manipulating objects only. Mapping hand written queries results to structs and updating the DB yourself on the basis of what is in structs is not at all an ORM.
- deleted 2y ago
- the__alchemist 2y agoEt tu, toasty? As time passes, the more I feel a minority in adoring rust, while detesting Async. I have attempted it a number of times, but it seems incompatible with my brain's idea of structure. Not asynchronous or concurrent programming, but Async/Await in rust. It appears that most of the networking libraries have committed to this path, and embedded it moving in its direction. I bring this up because a main reason for my distaste is Async's incompatibility with non-Async. I also bring this up because lack of a Django or SQLAlchemy-style ORM is one reason I continue to write web applications in Python.
- littlestymaar 2y agoAsync code is not incompatible with blocking one, in Rust it's quite straightforward to make the two interoperate: calling a blocking code from async is donne with spawn_blocking and the reverse (async from blocking code) is done with block_on.
- the__alchemist 2y agoI think this is core to the disconnect: Non Async/await does not imply blocking.
- littlestymaar 2y agoIf it's neither blocking nor async then it's a completely regular function and you don't even have to call it with spawn blocking, there's nothing that prevent calling a normal function from an async one. And in the opposite situation, if you call an async function then you are doing IO so your function must be either async or blocking, there's no third way in this direction, so when you're doing IO you have to make a choice: you either make it explicit (and thus declare the function async) or you hide it (by making a blocking call). A blocking function is just a function doing IO that hides it from the type system and pretend to be a regular function.
- tempest_ 2y ago
- didip 2y agoI think the custom schema definition file is not needed. Just define it in plain Rust. Not sure what the win is for this tool.
- jruz 2y agoMeh, the article makes it sound as if there were nothing and we have already solid options like SeaORM and Diesel. Still to me they all suck and nothing beats SQLx
- ericyd 2y agoI find the syntax confusing. Setting properties and even creating associated model instances is done with opaque method names like `.name()` and `.todo()`. I'm not always a fan of using set/get prefixes, but I think there should be some differentiation for an ORM which is inherently involved in property access. I'm particular it is strange and surprising to me that `.todo()` would associate another model. Why not "add_todo" or "create_todo"? What if the association is not one to many but one to one? The method `.todos()` retrieves a list of Todos, but what if we're talking about a 1:1 Profile model? How would a user differentiate between setting and getting a `.profile()`? I'm not a rust person so I might just be exposing my ignorance here, just wanted to provide feedback since it's on early development.
- satvikpendem 2y agoLooks similar to Prisma Client Rust but because Prisma and its file format are already established unlike toasty files, might be easier to use that. However, this is by Tokio and PCR is relatively unknown with development being not too fast, so your mileage may vary. I've been using diesel (with diesel_async) so far.
- xpe 2y agoI wish the following three paragraphs were widely read and understood by all software developers, especially web developers: > The common wisdom is to maximize productivity when performance is less critical. I agree with this position. When building a web application, performance is a secondary concern to productivity. So why are teams adopting Rust more often where performance is less critical? It is because once you learn Rust, you can be very productive. > Productivity is complex and multifaceted. We can all agree that Rust's edit-compile-test cycle could be quicker. This friction is countered by fewer bugs, production issues, and a robust long-term maintenance story (Rust's borrow checker tends to incentivize more maintainable code). Additionally, because Rust can work well for many use cases, whether infrastructure-level server cases, higher-level web applications, or even in the client (browser via WASM and iOS, MacOS, Windows, etc. natively), Rust has an excellent code-reuse story. Internal libraries can be written once and reused in all of these contexts. > So, while Rust might not be the most productive programming language for prototyping, it is very competitive for projects that will be around for years.
- rfoo 2y agoI'd add that a lot of the described advantages come from culture. For web applications manual memory management is 100% a friction instead of a relief. But the culture in Rust community in general, at least for the past ten years or so, is to encourage a coding style with inherently fewer bugs and more reusable, maintainable code, to the point of consistently want something to not happen if they weren't sure they got it right (one may argue that this is counter-production short-term). It is this culture thing makes adopting Rust for web apps worthwhile - it counters the drawback of manual memory management. If you hire an engineer already familiar with Rust you are sure you get someone who is sane. If you onboard someone with no Rust background you can be pretty sure that they are going to learn the right way (tm) to do everything, or fail to make any meaningful contribution, instead of becoming a -10x engineer. If you work in a place with a healthy engineering culture, trains people well, with good infra, it doesn't really matter, you may as well use C++. But for us not so lucky, Rust helps a lot, and it is not about memory safety, at all.
- xpe 2y ago
- dvdbloc 2y agoIs it just me or why does Rust have all these confusing names that seemingly have nothing to do with the functionality of the module/crate? Or maybe I’m just used to the names in Python and C++ packages for performing common tasks. It just seems to make it harder for a newcomer to locate what packages they should be using when they want to perform some common function.
- LtdJorge 2y agoYep, for example hearing Jinja tells you right away what the package does.
- dzarasovlloyd 2y ago[dead]
- cutler 2y agoWhy, oh why? Just SAY NO TO ORMs, especially in non-OO languages.