5 ms·

How to Compose Queries in Ecto (2019)

54 points by lobo_tuerto 6y ago

mmartinson 6y ago
Ecto is a real unsung here of the Elixir ecosystem. It's a joy to build queries and changesets through composable data structures, and to know exactly when they're going to be executed against the DB. I miss it deeply whenever I'm working with a different ORM. If anyone is curios and wants to get a high level overview, this conference talk by Darin Wilson provides a helpful intro to the patterns Ecto uses that differ from other popular ORMs. https://www.youtube.com/watch?v=YQxopjai0CU https://www.youtube.com/watch?v=YQxopjai0CU
methyl 6y ago
My biggest gripe with Ecto is that it's incredibly verbose and awkward to use for one-off tasks directly from the console. I know it's by design and its design works wonders from maintainability and composability point of view, but I really miss Rails' ActiveRecord when fiddling with data. It's to the point I sometimes prefer to use raw SQL, which is astounding. Part of the offender is terrible UX of iex (like, why the heck can't I cancel multi-line command and not loose all my context?). I believe improving on those two could lead to greater adoption of Elixir and Phoenix.
- dqv 6y ago
  >Part of the offender is terrible UX of iex (like, why the heck can't I cancel multi-line command and not loose all my context?). #iex:break? But I do agree that it would help if there were just a default helper module that collected Ecto.Query and all the app schemas. It can be pretty annoying to have to re-alias schemas.
  - methyl 6y ago
    > #iex:break? TIL. Does that differ from CTRL+G, i, c as in won't loose the context?
  - d3ckard 6y ago
    You actually can do that. Your project can define IIRC ".iex.exs" file that gets executed before you enter iex, so you can have all utility aliases and imports there.
    - davidw 6y ago
      One of the things I like about Elixir compared to Erlang is that it has a lot more beginner-friendly stuff as opposed to "oh, that's trivial to code up" that I have occasionally seen in the Erlang world. If it is, do it and make it available so that someone getting their feet wet has it available!
- realusername 6y ago
  On my case I've added first(), last() and a few other ones I was using with rails to MyApp.Schema, this way it's a bit more usable in the console.
dudul 6y ago
I actually find composed queries and the associated "bureaucracy" a little annoying in Ecto. The convention seems to be to put pure "query components" in your schema, to call them from your context to build the actual query you want to send to your Repo. The problem with that is that it becomes a bit harder to parse what the query actually is, which columns are effectively involved, and which indices to you want to create to make sure they support it. This is somewhat similar to what's briefly mentioned in a recent post by Sasa Juric: https://medium.com/very-big-things/towards-maintainable-elixir-the-core-and-the-interface-c267f0da43 https://medium.com/very-big-things/towards-maintainable-elix... Another thing is that I really wish they would find a way to make it more obvious/easier to understand that `Changeset` is a formidable tool for pure data validation, without necessarily involving a DB, or even a Schema. Other than that, it's a great tool :)
- ch4s3 6y ago
  I think the balance between reusable and composable query components and readable queries will depend on your use case. If you find a composed query too hard to read, you should probably do less of the query building in functions. I find decomposing into small functions is most useful when the query needs to come together dynamically.
- senjin 6y ago
  I completely agree about changesets. I wish they would extract that into it's own project that ecto uses. Seems like a lot of work to make it general enough but using it for form validation has worked out really well and I think using them for that case should be a more first class citizen in phoenix.
- sodapopcan 6y ago
  What you're describing is really no different than ActiveRecord scopes. The difference is is that Rails will silently do an N+1. I do agree about Changesets—they took me quite a while to grok, but I really like them now.
weatherlight 6y ago
I know a lot of developers compare Ecto to ActiveRecord, I feel like a more fair comparison is Arel. Is that the case?
- ravi-delia 6y ago
  I think Ecto is most commonly compared to ActiveRecord because it serves essentially the same role in the ecosystem. The scope is a little different, but for onboarding Ruby developers the ActiveRecord comparison is pretty good.
- sodapopcan 6y ago
  It's more true to say it's like arel than ActiveRecord, yes. It's more powerful than arel, though. It's otherwise not an ORM (more like an RM) and uses the Repo pattern.
conradfr 6y ago
Would have never thought about the default first argument. In fact it never occurred to me that you could use a default argument that is not towards the end of the arguments. Also I'm not surprised he uses the pipe format when compositing a query because I tried with the macro format and never succeeded to make it work.

54 points by lobo_tuerto 6y ago

Ecto is a real unsung here of the Elixir ecosystem. It's a joy to build queries and changesets through composable data structures, and to know exactly when they're going to be executed against the DB. I miss it deeply whenever I'm working with a different ORM. If anyone is curios and wants to get a high level overview, this conference talk by Darin Wilson provides a helpful intro to the patterns Ecto uses that differ from other popular ORMs. https://www.youtube.com/watch?v=YQxopjai0CU https://www.youtube.com/watch?v=YQxopjai0CU

methyl 6y ago

My biggest gripe with Ecto is that it's incredibly verbose and awkward to use for one-off tasks directly from the console. I know it's by design and its design works wonders from maintainability and composability point of view, but I really miss Rails' ActiveRecord when fiddling with data. It's to the point I sometimes prefer to use raw SQL, which is astounding. Part of the offender is terrible UX of iex (like, why the heck can't I cancel multi-line command and not loose all my context?). I believe improving on those two could lead to greater adoption of Elixir and Phoenix.

dqv 6y ago
>Part of the offender is terrible UX of iex (like, why the heck can't I cancel multi-line command and not loose all my context?). #iex:break? But I do agree that it would help if there were just a default helper module that collected Ecto.Query and all the app schemas. It can be pretty annoying to have to re-alias schemas.
- methyl 6y ago
  > #iex:break? TIL. Does that differ from CTRL+G, i, c as in won't loose the context?
- d3ckard 6y ago
  You actually can do that. Your project can define IIRC ".iex.exs" file that gets executed before you enter iex, so you can have all utility aliases and imports there.
  - davidw 6y ago
    One of the things I like about Elixir compared to Erlang is that it has a lot more beginner-friendly stuff as opposed to "oh, that's trivial to code up" that I have occasionally seen in the Erlang world. If it is, do it and make it available so that someone getting their feet wet has it available!
realusername 6y ago
On my case I've added first(), last() and a few other ones I was using with rails to MyApp.Schema, this way it's a bit more usable in the console.

dudul 6y ago

I actually find composed queries and the associated "bureaucracy" a little annoying in Ecto. The convention seems to be to put pure "query components" in your schema, to call them from your context to build the actual query you want to send to your Repo. The problem with that is that it becomes a bit harder to parse what the query actually is, which columns are effectively involved, and which indices to you want to create to make sure they support it. This is somewhat similar to what's briefly mentioned in a recent post by Sasa Juric: https://medium.com/very-big-things/towards-maintainable-elixir-the-core-and-the-interface-c267f0da43 https://medium.com/very-big-things/towards-maintainable-elix... Another thing is that I really wish they would find a way to make it more obvious/easier to understand that `Changeset` is a formidable tool for pure data validation, without necessarily involving a DB, or even a Schema. Other than that, it's a great tool :)

ch4s3 6y ago
I think the balance between reusable and composable query components and readable queries will depend on your use case. If you find a composed query too hard to read, you should probably do less of the query building in functions. I find decomposing into small functions is most useful when the query needs to come together dynamically.
senjin 6y ago
I completely agree about changesets. I wish they would extract that into it's own project that ecto uses. Seems like a lot of work to make it general enough but using it for form validation has worked out really well and I think using them for that case should be a more first class citizen in phoenix.
sodapopcan 6y ago
What you're describing is really no different than ActiveRecord scopes. The difference is is that Rails will silently do an N+1. I do agree about Changesets—they took me quite a while to grok, but I really like them now.

weatherlight 6y ago

I know a lot of developers compare Ecto to ActiveRecord, I feel like a more fair comparison is Arel. Is that the case?

ravi-delia 6y ago
I think Ecto is most commonly compared to ActiveRecord because it serves essentially the same role in the ecosystem. The scope is a little different, but for onboarding Ruby developers the ActiveRecord comparison is pretty good.
sodapopcan 6y ago
It's more true to say it's like arel than ActiveRecord, yes. It's more powerful than arel, though. It's otherwise not an ORM (more like an RM) and uses the Repo pattern.

conradfr 6y ago

Would have never thought about the default first argument. In fact it never occurred to me that you could use a default argument that is not towards the end of the arguments. Also I'm not surprised he uses the pipe format when compositing a query because I tried with the macro format and never succeeded to make it work.