3 ms·

> It used to get accused of scaling poorly, but tell that to Github, Gitlab, and other large-scale sites with millions of users. I've worked on one such large-

> It used to get accused of scaling poorly, but tell that to Github, Gitlab, and other large-scale sites with millions of users. I've worked on one such large-scale enterprise-level web app, and it really does scale poorly in some areas, specifically database management. ActiveRecord tends to get extremely expensive when db's take on large amounts of records and complex relationships start forming between those large tables. You need VERY good software engineering practices and understanding of which choices will utterly kill you down the line in terms of performance when using this ORM, and it's hard to see those consequences at the outset due to the fact that Rails code is so easy to pick up and understand.

taf2 8y ago
Isn't this true of any language? I mean the language isn't what decides how you connect or build your sql queries? In rails I could choose to write a pretty scary join ``` User.joins("left join ponies on ponies.user_id IN (select id from foobars where user_id=users.id").group("so_funny").count ``` or I could have a policy to avoid joins at all cost and reject any commit that uses a join and get fun things like: ``` users = User.all # all fucking hell JSON.parse(HTTP.get("http://your-things.json"))['results'].select {|r| users.detect {|u| u.id == r['user_id'] } # for realz?? } ``` [edit]: cause formatting
- Rooster61 8y ago
  You are right, but aren't really attacking the point I was making. Even with totally optimized sql, one language's ORM will be inherently slower than another due to the intrisics of how the records being operated on are represented in that particular language. In Rails's case, EVERY single record pulled out of a database gets its own object. There is overhead to being able to represent table members from a relational database in a language with all of the lovely convenience methods tacked on to the resulting object (things such as treating ranges of records pulled like a hash, for instance. This is compounded by the fact that ActiveRecord obfuscates much of the sql code you posted above (not always, but in many cases). When a user new to Rails goes in and sees "Oh, all I have to do to set up foreign key relations is to just say this model belongs to this other one? NICE!", it isn't immediately clear what the implications of whats happening under the hood to accommodate that are, and all of the ways which this can quickly be misused to cause a huge performance bottleneck. That syntactic sugar smooths over some of the pitfalls of efficiently interacting with a relational DB. It's important to note that these things do not make ActiveRecord a bad ORM in and of itself, as many users will not build projects large enough to have to worry about such bottlenecks, but it CAN and DOES cause problems when projects get larger and have to start taking on overhead from the resources allocated for all those records. That is what people are complaining about when they mention Rails doesn't scale well (or at least one of the reasons).
  - taf2 8y ago
    That is why we have pluck
    - throwaway427 8y ago
      Exactly. Use pluck, use exists, use count, use find_in_batches. Be thoughtful and have a culture of code reviews. Simple enough.
    - Rooster61 8y ago
      > it's hard to see those consequences at the outset due to the fact that Rails code is so easy to pick up and understand Yes, there are tools in ActiveRecord that can help mitigate some of the performance issues, but as I noted above, it's extremely easy to overlook such tools until you are too far down stream to really do much about it. The symptoms of not using such features may not show up until much later when the number of records swells, and some code someone wrote 2 years ago starts slowing down the whole shebang simply because they used a select rather than a pluck. This is ESPECIALLY true in large corporate web app environments where the number of records is large and the number of people with their fingers in the code base is equally sizable. And good luck convincing management to go back and fix old ORM tech debt until the whole thing starts falling apart at the seams. Hell, pluck didn't even exist until 2012 with the release of 3.2.1.
      - ehsanu1 8y ago
        Might depend on the amount of spaghetti, but I've typically been able to retroactively fix bad AR patterns backwards compatibly, and get huge speedups. The only time it's a real issue is when an API endpoint is just returning way too much and you have a lot of clients depending on that API returning all that data. What's a lot harder to deal with is AR callback hell, which can be very slow if you have enough nested callbacks, and can be really hard to fix without breaking everything. Using many AR callbacks is just a bad idea for a big app!

taf2 8y ago

Isn't this true of any language? I mean the language isn't what decides how you connect or build your sql queries? In rails I could choose to write a pretty scary join ``` User.joins("left join ponies on ponies.user_id IN (select id from foobars where user_id=users.id").group("so_funny").count ``` or I could have a policy to avoid joins at all cost and reject any commit that uses a join and get fun things like: ``` users = User.all # all fucking hell JSON.parse(HTTP.get("http://your-things.json"))['results'].select {|r| users.detect {|u| u.id == r['user_id'] } # for realz?? } ``` [edit]: cause formatting

Rooster61 8y ago
You are right, but aren't really attacking the point I was making. Even with totally optimized sql, one language's ORM will be inherently slower than another due to the intrisics of how the records being operated on are represented in that particular language. In Rails's case, EVERY single record pulled out of a database gets its own object. There is overhead to being able to represent table members from a relational database in a language with all of the lovely convenience methods tacked on to the resulting object (things such as treating ranges of records pulled like a hash, for instance. This is compounded by the fact that ActiveRecord obfuscates much of the sql code you posted above (not always, but in many cases). When a user new to Rails goes in and sees "Oh, all I have to do to set up foreign key relations is to just say this model belongs to this other one? NICE!", it isn't immediately clear what the implications of whats happening under the hood to accommodate that are, and all of the ways which this can quickly be misused to cause a huge performance bottleneck. That syntactic sugar smooths over some of the pitfalls of efficiently interacting with a relational DB. It's important to note that these things do not make ActiveRecord a bad ORM in and of itself, as many users will not build projects large enough to have to worry about such bottlenecks, but it CAN and DOES cause problems when projects get larger and have to start taking on overhead from the resources allocated for all those records. That is what people are complaining about when they mention Rails doesn't scale well (or at least one of the reasons).
- taf2 8y ago
  That is why we have pluck
  - throwaway427 8y ago
    Exactly. Use pluck, use exists, use count, use find_in_batches. Be thoughtful and have a culture of code reviews. Simple enough.
  - Rooster61 8y ago
    > it's hard to see those consequences at the outset due to the fact that Rails code is so easy to pick up and understand Yes, there are tools in ActiveRecord that can help mitigate some of the performance issues, but as I noted above, it's extremely easy to overlook such tools until you are too far down stream to really do much about it. The symptoms of not using such features may not show up until much later when the number of records swells, and some code someone wrote 2 years ago starts slowing down the whole shebang simply because they used a select rather than a pluck. This is ESPECIALLY true in large corporate web app environments where the number of records is large and the number of people with their fingers in the code base is equally sizable. And good luck convincing management to go back and fix old ORM tech debt until the whole thing starts falling apart at the seams. Hell, pluck didn't even exist until 2012 with the release of 3.2.1.
    - ehsanu1 8y ago
      Might depend on the amount of spaghetti, but I've typically been able to retroactively fix bad AR patterns backwards compatibly, and get huge speedups. The only time it's a real issue is when an API endpoint is just returning way too much and you have a lot of clients depending on that API returning all that data. What's a lot harder to deal with is AR callback hell, which can be very slow if you have enough nested callbacks, and can be really hard to fix without breaking everything. Using many AR callbacks is just a bad idea for a big app!