5 ms·

In that very specific example, you are right -- the benefits are pretty minimal (mostly just the deserialization). However, you're actually able to represent th

In that very specific example, you are right -- the benefits are pretty minimal (mostly just the deserialization). However, you're actually able to represent the relationship between those two models however you want, but still deal with relationships between types, and not what the exact join clause is. Additionally, SQL is extremely hard to compose. In my applications, I very rarely have some specific one off query that I want to run, I have multiple predicates that I want to be able to mix and match easily. String munging is fragile, while this actually lets me build reasonable domain abstractions. To your final point "Unless this has some huge performance gain over rust-postgres" Diesel performs 33% faster than rust-postgres, and will be competitive with if not faster than ideomatic C.

BinaryIdiot 11y ago
> Additionally, SQL is extremely hard to compose. I've seen almost everyone who doesn't use SQL directly say this but I always feel it's the exact opposite. SQL is easy to write once you do it enough. I've gotten to the point where I compose zero SQL in any application's code and, instead, do it all inside of stored procedures (of which they do not generate dynamic SQL). Then my code just calls stored procedures, my database users for the web application never have more access than executing a specific subset of stored procedures, and if there is a bug found in a stored procedure I can deploy them separately from the web application. I'm guessing it's just different philosophies but almost every time I've used something that abstracts an existing interface to another, completely different interface, I always end up running into major issues.
- DannoHung 11y ago
  SQL is hard to programatically compose. At the very least, you have to use a Lang/SQL.
  - Pxtl 11y ago
    That's a new term to me - lang/SQL?
    - d215 11y ago
      I guess he means a procedural language for sql, like PL/Pyton, PL/V8, etc., so PL/
      - Pxtl 11y ago
        Oh god, ick. I've used T-SQL and Pl/SQL procedurally and hate them with the burning passion of many suns.
- rabidferret 11y ago
  Stored procedures are no more composeable than SQL strings are. Now it's certainly easy to say that I haven't used SQL directly. But I did spend years working that way (though of course this statement shouldn't give you much confidence by itself). Ultimately an ORM can help you to build more maintainable and resilient (to changing market conditions) software than direct SQL. Diesel in particular can catch mistakes that SQL strings cannot. I do understand why a lot of people have reservations about ORMs. I share a lot of them. I've maintained Active Record for about 2 years now, and learned a lot from doing so (note: This does not mean that I like AR or want this to be anything like it). TL;DR: Your points aren't wrong. Diesel is different. Maybe give it a shot. ;)
  - BinaryIdiot 11y ago
    > Stored procedures are no more composeable than SQL strings are[...]Ultimately an ORM can help you to build more maintainable and resilient (to changing market conditions) software than direct SQL. I couldn't disagree more. Creating the separation between SQL DB and web application allows you to maintain, deploy, fix and upgrade things independently and it's easy (maybe not quite as easy as an ORM would make it but not really harder either; you're going to have to know how SQL works anyway if you want to effectively optimize and secure your data). Not only that but you can provide far better security controls because you can limit certain users access to certain procedures, never direct access to tables and various other things and if you don't do any dynamic SQL in the procedures itself you not only never have to worry about SQL injection but unless there is a critical vulnerability in the SQL system itself it's going to be impossible to conduct an injection attack. Yeah you can separate your code to make it easier to deploy separately to get a similar benefit and yeah your code can prevent SQL injections too but it's so much more work, more possible points of failure and the differences between databases can really bite you in the ass when you do everything generically. SQL is fun :) > TL;DR: Your points aren't wrong. Diesel is different. Maybe give it a shot. ;) Yeah I'm not saying NEVER use ORMs; they're great for creating things quickly that don't need to scale. But after my experience with them I'm highly skeptical using them outside of that type of use case. Of course you can scale ORMs but I've never seen it done well.
    - 11y ago
- skywhopper 11y ago
  "Compose" here doesn't mean "to write". It means to build up a database query in multiple steps, gradually adding (or removing) restrictions or relations dynamically in response to input. So one method might add a certain restriction that will get translated to a part of the where clause. Another might change which fields are being selected for or joined on or add transformations to the field. Meanwhile, the developer doesn't have to worry about special cases, or whether you need an AND or an OR, or how to go about wrapping a field in the SELECT clause after the fact. These methods might be in totally different contexts in the underlying code, and it would not be possible to make that style of query composition possible without abstracting away the SQL syntax. And that's exactly what an ORM does for you.
  - BinaryIdiot 11y ago
    Oh I know what compose means. Having written a ton of stored procedures for large scale web applications I've never found the need to dynamically build such queries. Instead I focus on what I need a specific query to do, provide the options to a stored procedure or function (depending on the RDMS) and that's it. Much like unix with simple programs that do very niche things I look at procedures much the same way. Don't get me wrong I see the simplicity in it from the ORM side I've just never had a good experience with ORM tools. Having worked directly against many RDMS and key value stores at least for me I don't see much value in them. But anything that helps get a product out the door more power to ya, in my opinion.
    - barrkel 11y ago
      What do you do when you want e.g. user controlled sorting and filtering? I just implemented this recently in my job, but it was even tougher than that; I was joining dynamically created tables, and the number of joins depended on criteria. I ended up building my own AST to represent filters, and compiling to SQL. Every extra table join hurts performance - typically filters are implented as joins against a filtered sorted limited subquery returning ids, so fewer different table joins means less data to pull in when evaluating filters even when all the data is needed for the results page. Point being, I had to compose complex dynamic queries and SQL was not a particularly pleasant target language to target directly. Abstractions (like asts) help with composition.
Ygg2 11y ago
Diesel performs 33% faster than rust-postgres, and will be competitive with if not faster than ideomatic C. Interesting. What makes it faster than rust-postgress?
- rabidferret 11y ago
  Short version: Our compile time guarantees allow us to omit a lot of runtime checks that you would otherwise have. Long version: http://bikeshed.fm/49 http://bikeshed.fm/49 Longer version: Planning on doing a really in depth write up for the website soon. I'll also be going into more detail during my talk at the SF rust meetup, which there's a link to the live stream towards the bottom of the thread
  - Ygg2 11y ago
    I'm interested if some of those guarantees can be used upstream in rust-postgres? As far as I saw, Diesel is essentially shipping with its own PostgreSQL driver.
    - rabidferret 11y ago
      It's possible. I tried to build on rust-postgres originally, but its design basically makes it impossible to abstract over. We also have fundamentally different views on how `ToSql` should work.
      - Ygg2 11y ago
        I'd love to find out what made it impossible to abstract over? Please include in your larger talk!
        rabidferret 11y ago
        The short version is its refusal to box things means that you have to have the connection, the statement, and the cursor all as local variables in the same lexical scope (you almost always want to abstract away the statement and cursor), and it's use of trait objects for serialization and deserialization. I might go into it briefly in my talk, but I don't like to rag on other people's work so I probably won't say much on it. (Sorry for the double post, I think I replied to my own comment and not yours!)
gleb 11y ago
I think somebody needs to actually justify the need for composable SQL. With examples. It's far from obvious. In my experience outside of extension code the need for composable SQL is very limited. Most uses cases can be satisfied by Rails 2 scopes. Now, if composability was free - sure it's great. But it's not free. So, on one hand: * it would be nice to have infinitely composable SQL On the other hand: * it would be nice to just have the actual SQL with all the features, well understood syntax and semantics that cost decades and billions of dollars to develop * and no wrapper-on-top-of-SQL limitations and translation bugs, and need for training With these language-on-top-of-SQL you don't get both. In fact you get severe compromises in the second group.
- rabidferret 11y ago
  Rails 2 scopes are literally the case for composeability...
  - gleb 11y ago
    Yes, but a very limited one. I'd love to see a solution with tradeoffs made the other way: * syntax and semantics 99% like underlying SQL * support for 99% of underlying DB functionality * traditional areas where ORMs add value: * deserialization into convenient data structures * casting, quoting and escaping on the way in and out * using stored association metadata for concise joins in the query
    - rabidferret 11y ago
      Every single one of those bullet points is a primary design goal of Diesel (caveat: Our semantics generally map to PG, because they apply well to other backends and are the least nonsensical)
- barrkel 11y ago
  User-generated data in a user-defined schema, where the user wants an Excel-autofilter-like sorting and filtering experience over rowsets that range from 10 thousand to 3 million rows, and 2 columns to 150 columns. Columns may be sorted and filtered individually or in combination. Filters may be based on value set membership or range criteria, comparisons with fields in the data or via a join to other tables, or based on the presence of an entry in a join table (think: tags or labels, and you want to filter by presence or absence of tag). (This is what I spent the past couple of months implementing. The simplest way of composing SQL is via union / except / intersect, but it's also the worst performing; doesn't work great past a few hundred thousand rows. And you still need to compose together the filter conditions into a where clause.)

BinaryIdiot 11y ago

> Additionally, SQL is extremely hard to compose. I've seen almost everyone who doesn't use SQL directly say this but I always feel it's the exact opposite. SQL is easy to write once you do it enough. I've gotten to the point where I compose zero SQL in any application's code and, instead, do it all inside of stored procedures (of which they do not generate dynamic SQL). Then my code just calls stored procedures, my database users for the web application never have more access than executing a specific subset of stored procedures, and if there is a bug found in a stored procedure I can deploy them separately from the web application. I'm guessing it's just different philosophies but almost every time I've used something that abstracts an existing interface to another, completely different interface, I always end up running into major issues.

DannoHung 11y ago
SQL is hard to programatically compose. At the very least, you have to use a Lang/SQL.
- Pxtl 11y ago
  That's a new term to me - lang/SQL?
  - d215 11y ago
    I guess he means a procedural language for sql, like PL/Pyton, PL/V8, etc., so PL/
    - Pxtl 11y ago
      Oh god, ick. I've used T-SQL and Pl/SQL procedurally and hate them with the burning passion of many suns.
rabidferret 11y ago
Stored procedures are no more composeable than SQL strings are. Now it's certainly easy to say that I haven't used SQL directly. But I did spend years working that way (though of course this statement shouldn't give you much confidence by itself). Ultimately an ORM can help you to build more maintainable and resilient (to changing market conditions) software than direct SQL. Diesel in particular can catch mistakes that SQL strings cannot. I do understand why a lot of people have reservations about ORMs. I share a lot of them. I've maintained Active Record for about 2 years now, and learned a lot from doing so (note: This does not mean that I like AR or want this to be anything like it). TL;DR: Your points aren't wrong. Diesel is different. Maybe give it a shot. ;)
- BinaryIdiot 11y ago
  > Stored procedures are no more composeable than SQL strings are[...]Ultimately an ORM can help you to build more maintainable and resilient (to changing market conditions) software than direct SQL. I couldn't disagree more. Creating the separation between SQL DB and web application allows you to maintain, deploy, fix and upgrade things independently and it's easy (maybe not quite as easy as an ORM would make it but not really harder either; you're going to have to know how SQL works anyway if you want to effectively optimize and secure your data). Not only that but you can provide far better security controls because you can limit certain users access to certain procedures, never direct access to tables and various other things and if you don't do any dynamic SQL in the procedures itself you not only never have to worry about SQL injection but unless there is a critical vulnerability in the SQL system itself it's going to be impossible to conduct an injection attack. Yeah you can separate your code to make it easier to deploy separately to get a similar benefit and yeah your code can prevent SQL injections too but it's so much more work, more possible points of failure and the differences between databases can really bite you in the ass when you do everything generically. SQL is fun :) > TL;DR: Your points aren't wrong. Diesel is different. Maybe give it a shot. ;) Yeah I'm not saying NEVER use ORMs; they're great for creating things quickly that don't need to scale. But after my experience with them I'm highly skeptical using them outside of that type of use case. Of course you can scale ORMs but I've never seen it done well.
  - 11y ago
skywhopper 11y ago
"Compose" here doesn't mean "to write". It means to build up a database query in multiple steps, gradually adding (or removing) restrictions or relations dynamically in response to input. So one method might add a certain restriction that will get translated to a part of the where clause. Another might change which fields are being selected for or joined on or add transformations to the field. Meanwhile, the developer doesn't have to worry about special cases, or whether you need an AND or an OR, or how to go about wrapping a field in the SELECT clause after the fact. These methods might be in totally different contexts in the underlying code, and it would not be possible to make that style of query composition possible without abstracting away the SQL syntax. And that's exactly what an ORM does for you.
- BinaryIdiot 11y ago
  Oh I know what compose means. Having written a ton of stored procedures for large scale web applications I've never found the need to dynamically build such queries. Instead I focus on what I need a specific query to do, provide the options to a stored procedure or function (depending on the RDMS) and that's it. Much like unix with simple programs that do very niche things I look at procedures much the same way. Don't get me wrong I see the simplicity in it from the ORM side I've just never had a good experience with ORM tools. Having worked directly against many RDMS and key value stores at least for me I don't see much value in them. But anything that helps get a product out the door more power to ya, in my opinion.
  - barrkel 11y ago
    What do you do when you want e.g. user controlled sorting and filtering? I just implemented this recently in my job, but it was even tougher than that; I was joining dynamically created tables, and the number of joins depended on criteria. I ended up building my own AST to represent filters, and compiling to SQL. Every extra table join hurts performance - typically filters are implented as joins against a filtered sorted limited subquery returning ids, so fewer different table joins means less data to pull in when evaluating filters even when all the data is needed for the results page. Point being, I had to compose complex dynamic queries and SQL was not a particularly pleasant target language to target directly. Abstractions (like asts) help with composition.

Ygg2 11y ago

Diesel performs 33% faster than rust-postgres, and will be competitive with if not faster than ideomatic C. Interesting. What makes it faster than rust-postgress?

rabidferret 11y ago
Short version: Our compile time guarantees allow us to omit a lot of runtime checks that you would otherwise have. Long version: http://bikeshed.fm/49 http://bikeshed.fm/49 Longer version: Planning on doing a really in depth write up for the website soon. I'll also be going into more detail during my talk at the SF rust meetup, which there's a link to the live stream towards the bottom of the thread
- Ygg2 11y ago
  I'm interested if some of those guarantees can be used upstream in rust-postgres? As far as I saw, Diesel is essentially shipping with its own PostgreSQL driver.
  - rabidferret 11y ago
    It's possible. I tried to build on rust-postgres originally, but its design basically makes it impossible to abstract over. We also have fundamentally different views on how `ToSql` should work.
    - Ygg2 11y ago
      I'd love to find out what made it impossible to abstract over? Please include in your larger talk!
      - rabidferret 11y ago
        The short version is its refusal to box things means that you have to have the connection, the statement, and the cursor all as local variables in the same lexical scope (you almost always want to abstract away the statement and cursor), and it's use of trait objects for serialization and deserialization. I might go into it briefly in my talk, but I don't like to rag on other people's work so I probably won't say much on it. (Sorry for the double post, I think I replied to my own comment and not yours!)

gleb 11y ago

I think somebody needs to actually justify the need for composable SQL. With examples. It's far from obvious. In my experience outside of extension code the need for composable SQL is very limited. Most uses cases can be satisfied by Rails 2 scopes. Now, if composability was free - sure it's great. But it's not free. So, on one hand: * it would be nice to have infinitely composable SQL On the other hand: * it would be nice to just have the actual SQL with all the features, well understood syntax and semantics that cost decades and billions of dollars to develop * and no wrapper-on-top-of-SQL limitations and translation bugs, and need for training With these language-on-top-of-SQL you don't get both. In fact you get severe compromises in the second group.

rabidferret 11y ago
Rails 2 scopes are literally the case for composeability...
- gleb 11y ago
  Yes, but a very limited one. I'd love to see a solution with tradeoffs made the other way: * syntax and semantics 99% like underlying SQL * support for 99% of underlying DB functionality * traditional areas where ORMs add value: * deserialization into convenient data structures * casting, quoting and escaping on the way in and out * using stored association metadata for concise joins in the query
  - rabidferret 11y ago
    Every single one of those bullet points is a primary design goal of Diesel (caveat: Our semantics generally map to PG, because they apply well to other backends and are the least nonsensical)
barrkel 11y ago
User-generated data in a user-defined schema, where the user wants an Excel-autofilter-like sorting and filtering experience over rowsets that range from 10 thousand to 3 million rows, and 2 columns to 150 columns. Columns may be sorted and filtered individually or in combination. Filters may be based on value set membership or range criteria, comparisons with fields in the data or via a join to other tables, or based on the presence of an entry in a join table (think: tags or labels, and you want to filter by presence or absence of tag). (This is what I spent the past couple of months implementing. The simplest way of composing SQL is via union / except / intersect, but it's also the worst performing; doesn't work great past a few hundred thousand rows. And you still need to compose together the filter conditions into a where clause.)