5 ms·

Database Performance Optimization and Scaling in Rails

83 points by Liriel 4y ago

WolfOliver 4y ago
I'm impressed how Rails seems not to become obsolete.
- brightball 4y ago
  There’s always going to be a demand for pure productivity.
- revskill 4y ago
  Boring technologies is just "boring, not obsolete.
- marginalia_nu 4y ago
  Java, PHP, RoR, C, they're all here to stay. Even Perl is still kicking, and it'll be decades still before the last line of COBOL has been written. I think there is a general misconception that if a language is not new and growing, then it is somehow going away. If anything, it's probably the opposite. Something like inertia is a much better predictor for language lifetime than than growth or features. In practice, the longer a language has been around and the more likely it is to stick around.
  - KronisLV 4y ago
    > In practice, the longer a language has been around and the more likely it is to stick around. I think this is called the Lindy effect: https://en.wikipedia.org/wiki/Lindy_effect https://en.wikipedia.org/wiki/Lindy_effect > The Lindy effect is a theorized phenomenon by which the future life expectancy of some non-perishable things, like a technology or an idea, is proportional to their current age. Thus, the Lindy effect proposes the longer a period something has survived to exist or be used in the present, the longer its remaining life expectancy.
- pyinstallwoes 4y ago
  Nothing matches it to this day really.
- JamesSwift 4y ago
  What would have replaced it?
- Wonnk13 4y ago
  There's a lot of selection bias wrt who submits posts to HN and the types of content they submit. It creates a sort of perception that tech/libraries/languages are moving near the speed of light and anyone who isn't treading along the bleeding edge is obsolete. As someone who doesn't write Ruby, even I can grok RoR in a few hours. And even though it's boring and not trendy, it gets the job done for a lot of use cases.
- pdntspa 4y ago
  Rails is fantastic! Especially with the new Hotwire/Turbo way of doing things. On a new personal project so far I haven't had to so much as TOUCH front-end JS!
- sytse 4y ago
  It is alive and kicking. Open Core Ventures had to decide on a tech stack and I think Rails is still the best option https://boards.greenhouse.io/opencoreventures/jobs/4751957004 https://boards.greenhouse.io/opencoreventures/jobs/475195700...
awesomegoat_com 4y ago
There is so much stuff happening in the ruby on rails. I follow the developments on YJIT and hotwire and absolutely love the communities there. The design is neat and that gives me confidence the stuff will be maintainable for decades to come.
rubyfan 4y ago
It goes without saying that there are a lot of things one would do prior to some of these steps. Because of the complexity these sort of configuration bring into your app one should weigh carefully whether they need these approaches. Database tuning and caching are preferable first steps in an app faced with performance challenges before fancy database routing. As with caching the app awareness of these approaches leaks into your code and can introduce testing challenges and misses in your process. Maybe I missed it but I’m surprised they didn’t mention Multi-Tenancy with Multi-Database since they have otherwise written about it*. For many apps this is not only a way to keep data isolated but is also a way to mitigate performance challenges that might come with a single large DB. https://blog.appsignal.com/2020/12/02/building-a-multi-tenant-ruby-on-rails-app-with-subdomains.html https://blog.appsignal.com/2020/12/02/building-a-multi-tenan...
wking 4y ago
What kind of services do you like to manage all of these "optimizations"? Most developers would get into more trouble trying to setup something like is talked about here, so I assume most people's database provider helps?
winslett 4y ago
Protip: in CRUD applications, most performance issues from the data layer have nothing to do with the database. Typically, the performance issues are at the ActiveRecord level, when it's initialization 100s of instances of an object to render a table. With Rails, the #1 optimization technique is to use `ActiveRecord::Base.connection.exec_query ` to return a hash instead of initialization objects. Side note: I still love Rails.
- sigmonsays 4y ago
  from my experience, every performance issue is the database when your programming language makes better choices.
- rubyist5eva 4y ago
  Another trap people fall into is doing joins in their application instead of in the database. I see this all the time where people will query some table with a condition, map the ids from the result and then query another table (which also joins the first table) and then filters by the id of the first query. Or just doing massive aggregates in ruby instead of having the database do it and just spitting out the result. A little bit of knowledge about SQL and how a relational database works goes a long way.
- Rapzid 4y ago
  It also wraps every query in an explicit transaction unnecessarily causing locks to be held while waiting for the commit from the server which got held up behind some cpu intensive block in separate thread (prob some other request deserializing and hydrating those 100s of entities you mentioned). Side note: I really dislike Rails.
  - jemmyw 4y ago
    > It also wraps every query in an explicit transaction unnecessarily causing locks to be held while waiting for the commit from the server no it doesn't. It only wraps updates in transactions, and a single line update would have an implicit transaction at the database anyway.

83 points by Liriel 4y ago

WolfOliver 4y ago

I'm impressed how Rails seems not to become obsolete.

brightball 4y ago
There’s always going to be a demand for pure productivity.
revskill 4y ago
Boring technologies is just "boring, not obsolete.
marginalia_nu 4y ago
Java, PHP, RoR, C, they're all here to stay. Even Perl is still kicking, and it'll be decades still before the last line of COBOL has been written. I think there is a general misconception that if a language is not new and growing, then it is somehow going away. If anything, it's probably the opposite. Something like inertia is a much better predictor for language lifetime than than growth or features. In practice, the longer a language has been around and the more likely it is to stick around.
- KronisLV 4y ago
  > In practice, the longer a language has been around and the more likely it is to stick around. I think this is called the Lindy effect: https://en.wikipedia.org/wiki/Lindy_effect https://en.wikipedia.org/wiki/Lindy_effect > The Lindy effect is a theorized phenomenon by which the future life expectancy of some non-perishable things, like a technology or an idea, is proportional to their current age. Thus, the Lindy effect proposes the longer a period something has survived to exist or be used in the present, the longer its remaining life expectancy.
pyinstallwoes 4y ago
Nothing matches it to this day really.
JamesSwift 4y ago
What would have replaced it?
Wonnk13 4y ago
There's a lot of selection bias wrt who submits posts to HN and the types of content they submit. It creates a sort of perception that tech/libraries/languages are moving near the speed of light and anyone who isn't treading along the bleeding edge is obsolete. As someone who doesn't write Ruby, even I can grok RoR in a few hours. And even though it's boring and not trendy, it gets the job done for a lot of use cases.
pdntspa 4y ago
Rails is fantastic! Especially with the new Hotwire/Turbo way of doing things. On a new personal project so far I haven't had to so much as TOUCH front-end JS!
sytse 4y ago
It is alive and kicking. Open Core Ventures had to decide on a tech stack and I think Rails is still the best option https://boards.greenhouse.io/opencoreventures/jobs/4751957004 https://boards.greenhouse.io/opencoreventures/jobs/475195700...

awesomegoat_com 4y ago

There is so much stuff happening in the ruby on rails. I follow the developments on YJIT and hotwire and absolutely love the communities there. The design is neat and that gives me confidence the stuff will be maintainable for decades to come.

rubyfan 4y ago

It goes without saying that there are a lot of things one would do prior to some of these steps. Because of the complexity these sort of configuration bring into your app one should weigh carefully whether they need these approaches. Database tuning and caching are preferable first steps in an app faced with performance challenges before fancy database routing. As with caching the app awareness of these approaches leaks into your code and can introduce testing challenges and misses in your process. Maybe I missed it but I’m surprised they didn’t mention Multi-Tenancy with Multi-Database since they have otherwise written about it*. For many apps this is not only a way to keep data isolated but is also a way to mitigate performance challenges that might come with a single large DB. https://blog.appsignal.com/2020/12/02/building-a-multi-tenant-ruby-on-rails-app-with-subdomains.html https://blog.appsignal.com/2020/12/02/building-a-multi-tenan...

wking 4y ago

What kind of services do you like to manage all of these "optimizations"? Most developers would get into more trouble trying to setup something like is talked about here, so I assume most people's database provider helps?

winslett 4y ago

Protip: in CRUD applications, most performance issues from the data layer have nothing to do with the database. Typically, the performance issues are at the ActiveRecord level, when it's initialization 100s of instances of an object to render a table. With Rails, the #1 optimization technique is to use `ActiveRecord::Base.connection.exec_query ` to return a hash instead of initialization objects. Side note: I still love Rails.

sigmonsays 4y ago
from my experience, every performance issue is the database when your programming language makes better choices.
rubyist5eva 4y ago
Another trap people fall into is doing joins in their application instead of in the database. I see this all the time where people will query some table with a condition, map the ids from the result and then query another table (which also joins the first table) and then filters by the id of the first query. Or just doing massive aggregates in ruby instead of having the database do it and just spitting out the result. A little bit of knowledge about SQL and how a relational database works goes a long way.
Rapzid 4y ago
It also wraps every query in an explicit transaction unnecessarily causing locks to be held while waiting for the commit from the server which got held up behind some cpu intensive block in separate thread (prob some other request deserializing and hydrating those 100s of entities you mentioned). Side note: I really dislike Rails.
- jemmyw 4y ago
  > It also wraps every query in an explicit transaction unnecessarily causing locks to be held while waiting for the commit from the server no it doesn't. It only wraps updates in transactions, and a single line update would have an implicit transaction at the database anyway.