21 ms·

Building GitHub with Ruby on Rails

holistio 4y ago
We are also running on Rails - and loving it, too. However, incorporating this methodology into our workflow would be both a lot of work to set up and also a lot of work to keep running. There certainly is a size threshold under which this is clearly an overkill and we are under that threshold. Yet, if this could be turned into a product, a clean integration, a command I need to run... would definitely use it (and pay for it).
- nevir 4y ago
  The main cost is maintaining high test coverage so that you can be confident in your automation
- New_California 4y ago
  On the contrary, it is so much easier to upgrade a small app every week or so because you have little to test and probability of breaking changes affecting you is minimal. You should be upgrading all the time since day one, adding necessary infrastructure gradually as your app grows.
  - ravloony 4y ago
    Absolutely. I was leading a team a while ago and I instituted this practice to good effect. Conversely, I was called by a company that I had built an app for previously. They had not upgraded the framework it was built with (Laravel), and ended up offering me consulting days to jump several versions. The irony is that the job ended up being quick and easy to do.
    - jonwinstanley 4y ago
      Yes, Laravel does a good job of making upgrades relatively simple
    - yurishimo 4y ago
      Laravel is relatively painless to upgrade as long as you have the autonomy to get the work done in a timely manner. I've seen upgrade projects drag on for weeks causing issues upstream since most of the team was still working on the product or fixing bugs while one developer was tasked with the upgrade. Did you use a tool like Shift to help with the upgrade? What about frontend dependencies? The recent move from Mix to Vite is great, but if you have a large frontend, it can be a major PITA to update. More so if you have any sort of custom webpack configuration.
  - riffraff 4y ago
    You need to have the right infrastructure/organizational maturity tho. For example in many organizations partial deployments don't exist and rollback is not easy, so having a "once a quarter we test everything and bump" is less traumatic and expensive than handling minor breakages every week. You can invest into that infrastructure/maturity but it may not be the best use of your time and money if you don't even know that your company will exist next year.
- tetha 4y ago
  I've been similar discussions with our security guy and a few of our build infrastructure guy, and yeah. I think the pattern to consider is: (i) Yes, this would improve our deployment speed/upgrade speed/security posture, possibly by a huge amount. (ii) We have much bigger problems with higher impact. Like, at work, we could spend a month or so to setup something like dependabot for our private stuff and I'm pretty sure we could get to a point of deploying these dependency updates quickly - or, for less critical systems, automatically even. And it would be cool. But that won't help us with some of the flagship products in the company that have C++ dependencies on EOL windows components and no automated deployments. We'd rather have the capable guys working on these nasty issues, since these upgrades for the modern products can usually be done by a junior dev in a few days for all of these smaller and well-controlled systems.
upmostly 4y ago
I absolutely love Rails. I'll always remember back in 2010 catching the train to Waterloo station in London and seeing a huge sign overlooking the train tracks that read something like "We Need Rails Developers". Rails was such a huge part of my professional career. Now, 13 years later and I'm deep in the JavaScript ecosystem and have been for 8 years. The most exciting thing to come out of this ecosystem recently is RedwoodJS, because it takes a lot of inspiration from Rails.
- jstummbillig 4y ago
  Have a look at AdonisJS. It's in a somewhat weird spot: On the one hand it is mature and has amazing DX/UX going for it, with a lot of very thoughtful tooling. On the other, for some reason, it has always remained niche with just a couple of core developers, even though it's now nearing version 6 and at least 8 years of releases. I do not know why that is the case. It's a beautifully written full stack framework, taking having inspiration from both Rails and Laravel and in the world of web TS, it should get a lot more attention than it does. Maybe it's just the weird name :^)
  - Gigachad 4y ago
    I used AdonisJS. Reminds me a lot of rails but with the added safety of typescript.
    - himhckr 4y ago
      Totally agree! I have used many frameworks (even written my own) but nothing gets close to Laravel for PHP and AdonisJS for TypeScript in terms of features out of the box and DX.
      - 0xblinq 4y ago
        Adonis is inspired from Laravel but it’s far from being comparable. Features, community, “battle proveness”, third party packages, etc are just not there yet.
  - 4y ago
Scarbutt 4y ago
Is that two millions LOC for a single Rails application or across various Rails applications?
- deleted 4y ago
  [deleted]
- New_California 4y ago
  They keep saying "monolith".
- jaredsohn 4y ago
  They call it a monolith and the way they describe it I think they are using that term for the application rather than the repo.
- nitinreddy88 4y ago
  It's one single application. Most of the secondary services are powered by go/ruby, running independently in non-monolithic way. Ex-Hubber
- drdaeman 4y ago
  One thing I just cannot wrap my head around is Ruby/Rails metaprogramming and attribution of things (as in "where this method/class/macro comes from?"), especially with so many authors trying to do something to the pre-existing stuff, including the standard library bits. Like, a lot of times I see some baz.foobarify() I have really hard time understanding where the heck that comes from. RubyMine makes wonders and control-clicking gets me there about half of time, but the rest of it it's either "search in all files, hoping that `def foobarify` is unique enough" or "this is crazy, but I'm gonna set a breakpoint, run it and see where this rabbit hole goes". It feels kinda like Scala (or, to lesser degree, Haskell) magic operators problem, but worse. I consider myself proficient with a decent number of languages and frameworks, but Ruby/Rails is one arcane mystery that just never clicked. So I really wonder what's the trick to make metaprogramming shenanigans manageable at this scale?
  - ptoomey3 4y ago
samwillis 4y ago
GitHub running off the main branch is fascinating, and initially sounds mad, but makes so much sense. Assuming they have very high test coverage, running against mainline Rails isn't really any different to having the fork they had before, but they have more influence on future development. It must also be a massive boon for the Rails ecosystem to have such a large property running off the head. Doesn't anyone know of any Django shops that do the same, running off mainline?
- winrid 4y ago
  I imagine it wouldn't be too hard, since Django only has one dependency - Django itself (NodeJS developers weep). Is rails the same way dependency wise?
  - procinct 4y ago
    Wait is this true? Doesn’t it also use SQLAlchemy at least? Which then likely has its own dependencies? I’d be really surprised if Django had no dependencies at all.
    - rochak 4y ago
      I don’t think Django uses SQLAlchemy. It has its own ORM which is damn good, but you can also use SWLAlchemy instead of that if you want.
    - fernandotakai 4y ago
      django 4.2 has two dependencies: asgiref and sqlparse. other than that, none. django does not use sqlalchemy as its ORM, it has its own system (which i prefer!).
      - KyeRussell 4y ago
        Amen. Until recently, I was exclusively working with Django for 5+ years. I definitely fell into the trap of making the ORM for granted. Had a brief foray into the JS world and despite plenty of slick-looking projects with fancy pants websites, nothing remotely compares.
Intrepidd 4y ago
Working with Ruby and Rails since 2010 and absolutely loving it. Never ever looking back.
MiraiY 4y ago
meanwhile, i have to work on a rails 3.2 app with ruby 2.2
- WJW 4y ago
  What's preventing you from upgrading?
  - MiraiY 4y ago
    the mindset "if it works, leave it there" edit: also it's a big project without specs, good luck at finding broken stuff...
    - weatherlight 4y ago
      Going to sound nuts but perhaps you can leverage ChatGPT4 to start generating some tests, based on some prompts for like 20% of your app. you just have to be pretty disciplined about how you write your specs.
      - MiraiY 4y ago
        actually, maybe not a bad idea
- awesomegoat_com 4y ago
  That's battle tested by now. Call it enterprise.
- caseyf 4y ago
  nothing wrong with Rails 3.2 :) get it on Ruby 3.1 if you can - https://railslts.com https://railslts.com
- mattbrewsbytes 4y ago
  Those versions are so far removed from receiving any security patches I'd probably want management to sign off that they heard me state this and they take full responsibility if shit hits the fan. Also, are you at least a little concerned about your career using a tech stack that old?
Zanfa 4y ago
It's been interesting looking into their "leaked" source to see how the application is architected. It's surprisingly accessible given the overall size.
- jack_riminton 4y ago
  Do you have a link? Sounds interesting
  - charcircuit 4y ago
    He's probably just referring to the self hosted version of Github. https://enterprise.github.com/trial https://enterprise.github.com/trial
    - Zanfa 4y ago
      Correct! It was reported a few years back that Github's source code was leaked and it just turned out somebody deobfuscated the source shipped as part of Github Enterprise's publicly-available VM image. I don't have a direct link, but it should still be possible to run the latest image through a script to get the source.
deleted 4y ago
[deleted]
politelemon 4y ago
Good post and an interesting glimpse into the behind-the-scenes efforts that goes into maintaining Github. I cannot agree with the "Should I do it too?" section. It probably works very well for an org as large and dedicated as Github, it very likely makes a lot of sense for what they do at the scale they do it at. For most of us that are consumers of technologies and frameworks, treating the framework as an extension of an application stack, as the blog has outlined, is a terrible idea; it requires devoting time and effort towards its upkeep, and that is not our core business, nor should it be. That does not, however, mean it's being treated as an unimportant part of the application stack, and implying so is judgmental. It's importance is that you need to be careful about the tech choices you make and understand what the implications will be down the line.
- awesomegoat_com 4y ago
  > it requires devoting time and effort towards its upkeep, and that is not our core business, nor should it be. I would try to argue that this in fact may be subjective. For me, updating all the dependencies in my stack every morning is a productivity routine that gets me started. If I spend a 30 minutes studying changelog of a dependency and linked github issues, I don't see that time as wasted even though it often does not contribute to the business. I think it has three benefits. - easy morning start routine to get going - developer education, expanding horizons - code climate; having warn feeling in the gut of not falling behind, wanting to avoid dirty plugs and monkey patches, ability to work on the HEAD, should an issue arise.
  - ysavir 4y ago
    I think the parent comment is referring to the act of submitting code changes to Rails directly whenever behavior is needed. Staying up to date with what your dependencies are doing and why makes sense for any project, but actively contributing to the dependencies isn't necessarily a great investment when it doesn't benefit your bottom line in a direct and timely matter. Github has a large enough staff that it can dedicate time to open PRs to the Rails repo without giving up its own productivity, but if you're looking at a team of 10 developers, taking time away from your own product to improve Rails can be a significant time cost.
pvsukale3 4y ago
It is so nice to have the latest releases of both Ruby and Rails run in production as soon as they are released. Huge for indie developers and small teams that use Rails. Kudos to Shopify too, who are working on improving Ruby tooling a lot recently. Exciting time to be a Rails dev.
suralind 4y ago
This make so much sense, I think every company should update deps frequently and Dependabot and other services like that can help with that. It probably would be too costly to attempt what GH is doing for smaller orgs.
snemvalts 4y ago
Github is one of the few webapps where I can feel daily that the framework used isn't enough. So many things get out of sync/not up to date, which are fixed by refreshing the page.
- hakre 4y ago
  Yes, just seen it yesterday. However, this is perhaps not of Github origins. Even when you use the recommended interface to git - the command line - it's only as up-to date as you've entered the last command for - or pressing F5 in gitk.
- Gasp0de 4y ago
  If anything that's a fault in the Frontend though, which is not what the article is about, right?
- amrocha 4y ago
  Very true, but that's a frontend problem, not a rails problem.
  - jeremyjh 4y ago
    These problems are always more common in server-rendered apps though, because front-end state is always a patchwork. And the Rails developers and community have a strong preference for this architecture.
    - VWWHFSfQ 4y ago
      I observe the exact opposite. Server rendered UIs are far more often times up to date than client rendered apps.
      - amrocha 4y ago
        I can't think of a single website that's real time and server rendered
mo_po2 4y ago
Going a bit of topic, Arch Linux does mostly the same, and thanks to it all of its users are direct testers/reporters of unpatched projects, with the exception of a few minor patches related to chore in projects that don't allow to customize it (mostly paths). I guess that this has probably improved the overall health of Linux beyond Arch itself during the last 2 decades for similar reasons to the ones provided in this article.
- jeremyjh 4y ago
  Yes it’s great people use it and report issues. I found and reported a years old bug in Erlang that was exposed by a zlib update. Arch makes it easy to isolate and rollback dependencies which was helpful to isolate the change. On the other hand, Debian/RHEL users never had to know about this complete showstopper bug since they never ran a system that had the new zlib and the old Erlang at the same time.
vrglvrglvrgl 4y ago
[dead]
hakre 4y ago
Some of their points which reminded me I consider working well, also for smaller teams. * Parallel builds/CI to look into the near future: current + next. When time is due current and next need to become green and the next alpha becomes future, which is allowed to fail (3 parallel tracks: current, next and future). Keeps all your build tools en par early, too, so there is less rot, and one team can concentrate on the migration while other teams aren't interrupted by that, but also for a single or only a handful of developers, you can have better change management (at the cost of the extra computing power). * It feels less a monolith, if its in a dynamic language (not compiled / transpilled / linked) and in many files (there is little to build and deployments are smaller and faster). In such projects, increments are possible across multiple axis in a synchronized dance. * Pipeline everything and continuously shift left. Identify the most important improvements and if they can step-break the process well, apply the earliest one coming from the developer perspective. Any such change will speed up any change after that step. * Implement fast turn-around with the parts that change often so you can change them often well. Cooperate well with upstream, this is most often a key to ongoing success. * Distributing traffic over multiple application servers and being able to deploy different configurations / revisions and directing part of the traffic to it is invaluable. Have your tools/systems configured well to make it easy and comfortable to do this way. Production must not feel like an all or nothing game any longer, but serves as experimental playground.
neals 4y ago
[flagged]
- that_guy_iain 4y ago
  > As someone who has experience working with large codebases, I am thoroughly impressed by GitHub's approach to upgrading their Ruby on Rails monolith. It is clear that the GitHub team has invested a significant amount of time and resources to ensure that their application runs on the latest version of Rails and Ruby, and this has resulted in real tangible benefits such as better database connection handling, faster view rendering, and improved security posture. To be fair, they did spend years not adding features. (Only really starting to add features when the Dear GitHub letter came out) It seems like they spent that time wisely just investing in stuff like this. GitHub are in a unique position where they spent so long ignoring features while having a large team. I feel like for the majority of teams these sort of investments are rarely possible because feature work is required.
  - irjustin 4y ago
    I down voted because it just seems like you wanted to being up the dear GitHub. It's been years... 7? with great features that go well beyond the letter. Yes gh was stagnant but this is pretty well in the past and your points have almost no current relevance.
    - that_guy_iain 4y ago
      > I down voted because it just seems like you wanted to being up the dear GitHub. I only brought up dear GitHub because I am sure people wouldn't have had an idea what I was talking about. > Yes gh was stagnant but this is pretty well in the past and your points have almost no current relevance. When my point is they used a bunch of time that other companies wouldn't give to build a solid foundation it seems very much that the past, that being the foundation, is very relevant. My entire point, is that they're in a very unique position a position that others can't really fairly expect since very few companies are able to ignore features for such a long time. This isn't to bash github. This is to put into persective what they've done and if others can do it. It removes the whole "GitHub did it so..." thing that many will suggest down the line.
dagorenouf 4y ago
I built my startup 4 years ago with a combination of react + aws + gatsby + hasura. I thought this would be great for performance and scale. Fast forward to today, I spend at least 2x as much time to code a feature than if I had just stuck with a simple rails stack, and the scale I imagined never happened. Now rebuilding everything with rails so I can ship faster and focus on growing the product, not making engineering prowesses.
- faeyanpiraat 4y ago
  Considering most of the times the startup at hand will fail anyway and the planned scale will not happen, building the initial architecture to be "web scale" is a perfect example of YAGNI. I had a similar experience, and it was a great lesson.
  - papito 4y ago
    We used to write software for pretty high-traffic systems using what is now underpowered CPUs, less RAM, and antiquated databases. It should be easier now, NOT harder. Most of development I see today is resume-driven and not based in any pragmatism.
- AeroNotix 4y ago
  Building for scale from day 0 is a recipe for disaster. You should've seen that coming.
  - Alifatisk 4y ago
    You suggest people should refactor later on when needed?
    - shortrounddev 4y ago
      There comes an inflection point in a startup when you have to move from MVP to scale, and there are two different kinds of tech and two kinds of people for each stage
    - 4y ago
rethab 4y ago
> scheduled GitHub Action workflow ..sounds like they're not dogfooding dependabot? curious if anybody knows more/why
- heartbreak 4y ago
  Dependabot is for security issues, not this?
  - rethab 4y ago
    dependabot can also be used for regular version updates
- deleted 4y ago
  [deleted]
- djm_ 4y ago
  Dependabot is based on releases from the various package repositories; running of main is pre-release - hence they’re probably using GitHub Actions to pin their Gemfile-defined Rails version to a commit hash.
qntmfred 4y ago
Every time it's pointed out that GitHub is a rails app I am reminded of one of my favorite security exploits ever https://news.ycombinator.com/item?id=3663197 https://news.ycombinator.com/item?id=3663197
peter_retief 4y ago
Another big rails application was twitter, it has been rewritten but the concept fitted well with rails. Twitter currently uses some java framework as far as I know. Question, does twitter still have some RoR deployed?
mrwnmonm 4y ago
All the cool kids left Ruby for Go, why are they still in that land?
- rst 4y ago
  Because they care more about getting things done than whether clout-chasers think they're "cool".
  - throwaway445232 4y ago
    [flagged]
  - mrwnmonm 4y ago
    I get things done just fine with Go. And all the code is very clear. With ruby it is not, and a lot of ruby programmers may have a feel about how things work, while they are not really. They just enjoy the syntax. The book "The Ruby Programming Language" which written by the creator or ruby (I forgot his name) is full of "unfortunately", unfortunately this works this way now, unfortunately that works that way now. And people definitly chose ruby because it was cool. So what you said should be applied to Ruby, not Go.
- elif 4y ago
  Since they left ruby (2.0) raw rails performance improved by 75% (3.0) https://www.fastruby.io/blog/assets/images/RRBPerfHistory_720.png https://www.fastruby.io/blog/assets/images/RRBPerfHistory_72... Then in 3.2, ruby released YJIT which improved median rails response time by about 100% https://raw.githubusercontent.com/easydatawarehousing/ruby_measure_responsetime/main/data/rails_devise/plots/rails_devise_0_overview.png https://raw.githubusercontent.com/easydatawarehousing/ruby_m... It's kind of a golden era for rubyists that stuck it out. Beautiful maintainable code that hits C performance for many tasks.
  - Alifatisk 4y ago
    "... hits C performance for many tasks" Really?
qrush 4y ago
Absolutely love this approach and I think it makes a ton of sense for an open source focused company - however I haven't seen this approach in any company I've worked for in 15 years (where I have helped with most of the Rails upgrades!). There's a ton of merit to being on the latest release or close to it so you can get the latest security patches easier. The diff between that and the latest `main` branch seems like it has diminishing returns for most orgs.
compumike 4y ago
Even as a much smaller team, building Heii On-Call [0] as a lightweight alerting/monitoring/on-call rotations SaaS based on Ruby on Rails has basically been a pleasure! And as the article highlights, perhaps the key reason for smooth deployments and upgrades is that the CI testing story is so, so good: RSpec [1] plus Capybara [2] for us. That means we have decently extensive tests of just about all behavior. The few small Rails and Ruby upgrades we've done have gone quite smoothly and confidently, with usually just a few non-Rails gem dependencies needing to be manually updated as well. The "microservices" story is where we've pulled in the Crystal programming language [3] to great effect. After dabbling with Go and Rust, we've found that Crystal is truly a breath of fresh air. Crystal powers the parts of Heii On-Call that need to be fast and low-RAM, specifically the inbound API https://api.heiioncall.com/ https://api.heiioncall.com/ and the outbound HTTP(S) prober background processes. I've ported some shared utility classes from Ruby to Crystal almost completely by just copy-and-pasting ___.rb to ___.cr; porting the tests for those classes was far more onerous than porting the class code itself. (Perhaps another point of evidence toward the superiority of RoR's testing story...) The front-end story is nice but just a bit weaker. Using Hotwire / Turbo successfully, but I have an open PR to fix a fairly obvious stale cache bug in Turbo [4] that has been sitting unloved for nearly a month, despite other users reporting the same issue. I'm hopeful that it will get merged in the next release, but definitely less active than the backend side. For me, the key conclusion is that the excellent Ruby on Rails testing story is what enables everything to go a lot more smoothly and have such a strong foundation. I'd be curious if any GitHubbers can talk more about whether they too are using Rspec+Capybara or something else? Are there internal guidelines for test coverage? [0] https://heiioncall.com/ https://heiioncall.com/ [1] https://rspec.info/ https://rspec.info/ [2] https://github.com/teamcapybara/capybara https://github.com/teamcapybara/capybara [3] https://crystal-lang.org/ https://crystal-lang.org/ [4] https://github.com/hotwired/turbo/pull/895 https://github.com/hotwired/turbo/pull/895
- jack_riminton 4y ago
  My startup is using minitest (mainly for running tests in parallel I believe) and capybara. Crystal looks great, are you using it mainly for type checking? If so why not Sorbet?
elif 4y ago
I wonder how much they end up having to unit test gems they use, and how much they have to maintain gem compatibility for those maintainers.. I have to imagine this leads to either GitHub being the defacto lead maintainers for those gems, or GitHub removing gems from their stack and writing their own code.
cutler 4y ago
I dunno. I lived through Internet Explorer, SCO and Linux patent lawsuit days so I reckon it's only a matter of time before Github goes ASP.Net.
- nobleach 4y ago
  I lived through Microsoft purchasing Hotmail and trying repeatedly to migrate it from FreeBSD to IIS/Windows2000. We may have a good 20 years.
- hocuspocus 4y ago
  A friend at GH was already writing mostly C# and Go before he moved to engineer manager 2+ years ago, so I believe it's already ongoing. Also he absolutely hated having to jump into components in RoR before Codespaces was available internally.
  - dewski 4y ago
    C# is used for Actions: https://github.com/actions/runner https://github.com/actions/runner, and Go is used a lot for internal services. There is no traction rewriting our monolith in C#.
thih9 4y ago
> Every Monday a scheduled GitHub Action workflow triggers an automated pull request, which bumps our Rails version to the latest commit on the Rails main branch for that day. So uploading code to a github repo makes github server execute it later on. I wonder if this would qualify for a bug bounty... /s Edit: I guess not, since this is just a pull request and still requires approval before merging to main. I assume.
- georgyo 4y ago
  Don't work for GitHub, but it would not. First, it opens a Pull Request, not an automatic merge, so hopefully the code is reviewed before merge. To exploit this, you would first need to get malicious code merged into rails master which has many eyes, and then get passed more eyes when it gets reviewed by GitHub. Not impossible, but if you got your code merged into rails master, you have wiggled your way into many more environments than just GitHub.
- nicce 4y ago
  Even if it was just not pull-request, highly unlikely that it would qualify. We can assume that only trusted maintainers can put code into the Ruby main branch. Similarly, every build system depends on many many signed packages. It is not much different than that.
uptownJimmy 4y ago
I like GitHub, fwiw. I am not someone who requires everything to be perfect, and I've learned to be tolerant of our human reality, where imperfection is the norm. But GitHub is not a great Web app. It is frequently/constantly out of sync with the latest data/status. You quickly develop the habit of manually refreshing the page every time you are preparing to do anything with a PR, and that's not something that should be necessary these days. It surprises me to see a loud and proud blog post detailing GitHub's process of staying so relentlessly up to date with the latest and greatest version of Rails: the app is not properly responsive, for whatever reason, and to a degree that would not be tolerated where I have worked. I would rather read about how they are trying to fix this issue.
- ysavir 4y ago
  I agree with this, but that seems to be a problem with Github's frontend code, not their server code. I don't think that detracts from this blog post's message.
  - uptownJimmy 4y ago
    I guess my implied question should be stated more explicitly: is GitHub's front end code entirely separate fro the Rails code? That's not how Rails apps usually work, I thought. Admittedly, it's been a long time since I looked at Rails code, and I don't have the slightest idea how GitHub is actually architected. But I don't remember the "front end code" in Rails being a separate thing from the server code, typically: the whole point was server-side rendering. Is GitHub using one of the JS/TS frameworks for the front end? I do remember that being a developing trend, a few years back.
    - danjac 4y ago
      Rails can just serve JSON or GraphQL from an API to an SPA frontend, or you can do full server-rendered, or Hotwire to do HTML fragment updates, or any combination thereof. IIRC Github used something like Hotwire but home-grown. I've not done Rails development for a while so not sure what the state of things is for web sockets, but I would think that's not a problem for the framework. Point is, there's nothing about Rails in particular that would prevent fixing these issues, that's probably the result of development and business priorities, legacy code, etc. that would be an issue with any tech stack.
exabrial 4y ago
I think the big lesson here is writing excellent tests and making them easy to execute makes your code easier to change fear of breaking something. Unfortunately I’m fairly certain the message that will be heard is ‘We should totally change something around on the customer every week, that’s what GitHub does’
surteen 4y ago
Are there other languages where the dominant web framework changes so much? Will Rails ever be "done"?
- no_wizard 4y ago
  .NET and Django are The pillars of backward compat and stability IMO. Change is somewhat inevitable though
- donapieppo 4y ago
  I hope not.
- ativzzz 4y ago
  It can never be done as long as the web keeps evolving and the web shows zero signs of slowing down evolution And those changes are the reason rails is still really good, modern, and not considered legacy software almost 20 years later
- throwaway445232 4y ago
  [flagged]
alberth 4y ago
> Every Monday a scheduled GitHub Action workflow triggers an automated pull request, which bumps our Rails version to the latest commit on the Rails main branch for that day. That’s a bold move to do as opposed to being end of week or weekend.
- prepend 4y ago
  I like Monday releases as if something goes wrong everyone is around to fix it. If something breaks on Friday it ruins weekends. I think Monday requires more maturity and more successes as it prioritizes dev time over downtime. Saturday outages affect fewer customers but are hard on staff.
  - swat535 4y ago
    We don't even release on Mondays, its too rushed. We ship between Tuesday and Thursday instead and only during productive hours (e.g 10AM-3PM). We want the engineers the be alert and calm.
- lcnPylGDnU4H9OF 4y ago
  I sure don’t want to be called on Saturday to be told that there’s something wrong with Friday’s build.
  - alberth 4y ago
    I’m sure paying customers don’t want outages during their prime usage days.
    - javier2 4y ago
      How do we achieve that? Releasing often and in smaller increments. A release causing an outage should be easier to test than the subtle bugs that come creeping one, two, three days later. Either way, if there is a catastrophic failure, its better to have everyone readily available.
- mousetree 4y ago
TheRealDunkirk 4y ago
I've been using Rails and GitHub since the first 2.x days. I have been expecting an announcement that they are switching to .NET since Microsoft bought them. I'm guessing it's too big of a bite, even for Microsoft, thank Ada. I love being able to point recalcitrant policy makers at my company at GitHub as an example of a top web app -- the context for which they already know very well -- being a Rails monolith. I'd love to see their Gemfile.
- mdaniel 4y ago
  > I'd love to see their Gemfile. I would bet it's present in GHE's OVA (https://enterprise.github.com/releases/3.8.1/download https://enterprise.github.com/releases/3.8.1/download)
nomilk 4y ago
> Instead of telling your team you found something in Rails that will be fixed in the next release, you can work on something in Rails and see it the following week! That's awesome. Not only fixing it for your team but the entire rails world.
- silverwind 4y ago
  > Not only fixing it for your team but the entire rails world Or breaking it for some.
danforddotdev 4y ago
I'm curious what their data access layer looks like underneath that monolith. Is the Rails piece mostly now just a frontend for dozens of other services, properly owned and maintained by other teams? I don't mean to trivialize something that's obviously huge and complex as "just a frontend", but IME one of the biggest things that breaks down in a Rails monolith as it scales is heavy, direct usage of ActiveRecord. Either there's lots of DB migrations happening since there's so many different developers working on different features, which makes development with a shared DB tricky, or the scale makes DB performance problems hard to diagnose since they cut across many teams or tables in complicated ways.
- withinboredom 4y ago
  I've heard this several times over the last few months. Like what makes several devs working in the same area of the code "hard"? In my experience whomever is lucky enough to commit first gets the easiest of it, everyone else just rebases and resolves their conflicts. Maybe if you don't rebase and merge instead? I've seen some screwed up stuff happen from bad merges... like entire lines of code vanish. But generally, I've worked with hundreds of devs in the same code base without issue. So, why do people ask this?
  - danforddotdev 4y ago
    Well, I gave a specific example in Rails, using DB migrations against a shared DB. It’s not an unsolvable problem, and of course each dev can have their own DB, but if this is poorly managed it’s easy to become unwieldy. Outside of that, if many devs are constantly making dependency changes such that every time you “git pull” you have to rebuild environments, etc. Maybe devs are adding features but not prepopulating dev environments with sensible test data so your dev environment gets horked. Etc etc. Its not usually about merging the code itself.
- jakemoffatt2 4y ago
  Can't speak to how GitHub does it, but in every Rails org I've been at, eventually we create secondary services that own a specific domain, and the primary app becomes a gateway that clients use to talk to those services. It's rare to spin up a new service in a new language/framework. A typical pattern is to migrate the service within the monolith into a Rails engine, and then move that to a new app where the app mounts only that engine, or one can find a way to make the monolith deployable with only certain engines mounted, like with an env var. A Rails engine is basically a self contained Rails app, including routes, which you can mount inside of a host Rails app at any route if your choosing. They're usually used to build reusable libraries, but this use case also works very well.
ksec 4y ago
And as I have said before but worth repeating again, Rails is perhaps the only open source framework that is being battle tested in development at scale. It may not be the fastest ( or in fact quite slow ), but I dont think you could find similar testing being done and Deployed at the scale of Github on any other framework. I wonder if Eileen Uchitelle will bring this practice to Shopify as well? Edit: It seems [1] Shopify is also running on latest Ruby and Rails version as well. [1] https://news.ycombinator.com/item?id=35481352 https://news.ycombinator.com/item?id=35481352
- hocuspocus 4y ago
  > but I dont think you could find similar testing being done and Deployed at the scale of Github on any other framework. ?! There's at least Spring.
  - ksec 4y ago
    It's been a very very long time I touched anything Java so excuse my ignorance. I know there are many Enterprise, internal Web Apps that uses Spring. But are there any web companies built their SaaS or Consumer Web App on Spring, that is at the scale of Shopify and Github?
    - hocuspocus 4y ago
      Most companies at that scale don't use a single framework or even language but there's probably more Spring at Amazon than RoR at GitHub, Stripe and Shopify combined.
      - throwaway2847 4y ago
        Stripe doesn't use Ruby on Rails, they have a framework developed in house
    - lenkite 4y ago
      Aliexpress and Udemy come right at the top of my mind. But there are quite a few more. (several German web-sites like Trivago). Spring Boot is used more outside the U.S. - Java is respected more in the rest of the world.
trevor-e 4y ago
Does anyone know if Github uses Sorbet for their Ruby and Rails code?
- dewski 4y ago
  We do.
didip 4y ago
I must be the only person on Hacker News who dislike Ruby. It simply has too much magic and doesn't offer enough abilities (unlike Elixir or any Scheme variant) to justify the dynamic type nature. Go and Rust are perfect for me (and maybe Zig?). They covered everything I need.
- hocuspocus 4y ago
  No, you aren't the only one. I used RoR at its peak hype in 2009 and I wasn't impressed. Even super boilerplaty alternatives at the time (like Spring 2.x and iBatis) were a lot more manageable in my opinion. And the few good ideas from frameworks like RoR and Django have long been stolen by all ecosystems, so I don't really see the point anymore.
- bastardoperator 4y ago
  None of the languages you listed/prefer existed when GitHub was launched, lol.
- sosodev 4y ago
  I love Go but Rails is just unmatched in terms of development velocity. You're right that Go eschews magic and embraces verbosity but in doing so it makes common tasks, like web development, take 10x longer than languages that do the opposite.
- vc8f6vVV 4y ago
  It's only magic if you don't know what's going on under the hood. I felt this way for the first couple of years until I figured it out (experience matters as with every other language/platform). Then there was no magic, just conventions. You can do crazy things with Ruby and people tend to do them thinking it's "cool". Just don't. Rails doesn't have many and they are pretty well documented.
insane_dreamer 4y ago
Oh, and here I was told that RoR is only for quick-and-dirty prototyping and that "real" web applications are written in ___ (fill in blank)
larusso 4y ago
> Ultimately, if more companies treated the framework as an extension of the application, it would result in higher resilience and stability. Investment in Rails ensures your foundation will not crumble under the weight of your application. Treating it as an unimportant part of your application is a mistake and many, many leaders make this mistake I really like the sentiment of this quote but that is an easy thing to say for a behemoth like GitHub. I find these blog posts from google, meta, aws etc super awesome since they run and solve problems smaller companies simply don’t have. And they shape how smaller companies solve similar issue at smaller scale. But doing a setup to be practically on bleeding edge rails for example is not something every company can afford. They need LTS releases and the sorts. Still awesome that they managed to achieve this.
stevebmark 4y ago
This is a very interesting cautionary tail for those who recommend Rails because Github uses it. It's hard to imagine another core framework where the "best" way to use it is to dedicate an entire department to continuously working off the bleeding edge. And if you read between the lines, changes that Github needs to make are within Rails itself, and as we know there are core Ruby and Rails maintainers working at Github. This is a cautionary tale because of how much overhead Github needs to make this work. If your company has the klout to hire core Rails maintainers and the staff to be able to focus on working off the main branch of one of your framework dependencies, then go for it! You should also read about Github's history with "Rails" and how challenging it was for them to make it fast enough for them and upgrade it. It's pretty interesting they got around that "challenge" by throwing a LOT more resources at it. It's interesting because this isn't usually a challenge with other major framework upgrades. It makes sense you would need this much investment given the hyper-dynamic dangers of Ruby (not just the language, but the ecosystem), making it difficult and risky to upgrade. TL;DR the advice in this blog post does not seem applicable to most companies.
aledalgrande 4y ago
This is awesome, now I just wish they spruced up their UI with Hotwire :p How many times have you clicked the back button and found the same notifications unread?
benatkin 4y ago
I've been using Codeberg, which uses Forgejo which is written in Go and is fast and light. Fantastic that GitHub has managed to wrangle so many lines of code in a language I don't care very much for, but my Samsung A53's browser is snappier without it :) Edit: to be fair, GitHub's has a hamburger menu that morphs to an X that I dearly miss. JK :P
- Alacart 4y ago
  Codeberg runs go on the frontend?
  - deleted 4y ago
    [deleted]
funnyfoobar 4y ago
How does the current Architecture of Github looks like, Apart from Rails, I have heard that they use Go as well.
hmontazeri 4y ago
Ruby and Rails are the single reasons why my previous startup worked out. We had a small engineering team and had to move incredibly fast. Without the magic we wouldn’t have been able to ship at insane speed. I tattooed the ruby logo on my arm as this was an absolute life changer for me. Having financial freedom now lets me spend so much time with my kids. I’ll be forever grateful.
- herbst 4y ago
  I totally feel you. Ruby & Rails are part of the reason I work way less than most of my peers for the same amount of money or more. Being able to built publish ready things, alone, within only weeks instead of months is a total life changer if you like building things

We are also running on Rails - and loving it, too. However, incorporating this methodology into our workflow would be both a lot of work to set up and also a lot of work to keep running. There certainly is a size threshold under which this is clearly an overkill and we are under that threshold. Yet, if this could be turned into a product, a clean integration, a command I need to run... would definitely use it (and pay for it).

nevir 4y ago
The main cost is maintaining high test coverage so that you can be confident in your automation
New_California 4y ago
On the contrary, it is so much easier to upgrade a small app every week or so because you have little to test and probability of breaking changes affecting you is minimal. You should be upgrading all the time since day one, adding necessary infrastructure gradually as your app grows.
- ravloony 4y ago
  Absolutely. I was leading a team a while ago and I instituted this practice to good effect. Conversely, I was called by a company that I had built an app for previously. They had not upgraded the framework it was built with (Laravel), and ended up offering me consulting days to jump several versions. The irony is that the job ended up being quick and easy to do.
  - jonwinstanley 4y ago
    Yes, Laravel does a good job of making upgrades relatively simple
  - yurishimo 4y ago
    Laravel is relatively painless to upgrade as long as you have the autonomy to get the work done in a timely manner. I've seen upgrade projects drag on for weeks causing issues upstream since most of the team was still working on the product or fixing bugs while one developer was tasked with the upgrade. Did you use a tool like Shift to help with the upgrade? What about frontend dependencies? The recent move from Mix to Vite is great, but if you have a large frontend, it can be a major PITA to update. More so if you have any sort of custom webpack configuration.
- riffraff 4y ago
  You need to have the right infrastructure/organizational maturity tho. For example in many organizations partial deployments don't exist and rollback is not easy, so having a "once a quarter we test everything and bump" is less traumatic and expensive than handling minor breakages every week. You can invest into that infrastructure/maturity but it may not be the best use of your time and money if you don't even know that your company will exist next year.
tetha 4y ago
I've been similar discussions with our security guy and a few of our build infrastructure guy, and yeah. I think the pattern to consider is: (i) Yes, this would improve our deployment speed/upgrade speed/security posture, possibly by a huge amount. (ii) We have much bigger problems with higher impact. Like, at work, we could spend a month or so to setup something like dependabot for our private stuff and I'm pretty sure we could get to a point of deploying these dependency updates quickly - or, for less critical systems, automatically even. And it would be cool. But that won't help us with some of the flagship products in the company that have C++ dependencies on EOL windows components and no automated deployments. We'd rather have the capable guys working on these nasty issues, since these upgrades for the modern products can usually be done by a junior dev in a few days for all of these smaller and well-controlled systems.

upmostly 4y ago

I absolutely love Rails. I'll always remember back in 2010 catching the train to Waterloo station in London and seeing a huge sign overlooking the train tracks that read something like "We Need Rails Developers". Rails was such a huge part of my professional career. Now, 13 years later and I'm deep in the JavaScript ecosystem and have been for 8 years. The most exciting thing to come out of this ecosystem recently is RedwoodJS, because it takes a lot of inspiration from Rails.

jstummbillig 4y ago
Have a look at AdonisJS. It's in a somewhat weird spot: On the one hand it is mature and has amazing DX/UX going for it, with a lot of very thoughtful tooling. On the other, for some reason, it has always remained niche with just a couple of core developers, even though it's now nearing version 6 and at least 8 years of releases. I do not know why that is the case. It's a beautifully written full stack framework, taking having inspiration from both Rails and Laravel and in the world of web TS, it should get a lot more attention than it does. Maybe it's just the weird name :^)
- Gigachad 4y ago
  I used AdonisJS. Reminds me a lot of rails but with the added safety of typescript.
  - himhckr 4y ago
    Totally agree! I have used many frameworks (even written my own) but nothing gets close to Laravel for PHP and AdonisJS for TypeScript in terms of features out of the box and DX.
    - 0xblinq 4y ago
      Adonis is inspired from Laravel but it’s far from being comparable. Features, community, “battle proveness”, third party packages, etc are just not there yet.
- 4y ago

Scarbutt 4y ago

Is that two millions LOC for a single Rails application or across various Rails applications?

deleted 4y ago
[deleted]
New_California 4y ago
They keep saying "monolith".
jaredsohn 4y ago
They call it a monolith and the way they describe it I think they are using that term for the application rather than the repo.
nitinreddy88 4y ago
It's one single application. Most of the secondary services are powered by go/ruby, running independently in non-monolithic way. Ex-Hubber
drdaeman 4y ago
One thing I just cannot wrap my head around is Ruby/Rails metaprogramming and attribution of things (as in "where this method/class/macro comes from?"), especially with so many authors trying to do something to the pre-existing stuff, including the standard library bits. Like, a lot of times I see some baz.foobarify() I have really hard time understanding where the heck that comes from. RubyMine makes wonders and control-clicking gets me there about half of time, but the rest of it it's either "search in all files, hoping that `def foobarify` is unique enough" or "this is crazy, but I'm gonna set a breakpoint, run it and see where this rabbit hole goes". It feels kinda like Scala (or, to lesser degree, Haskell) magic operators problem, but worse. I consider myself proficient with a decent number of languages and frameworks, but Ruby/Rails is one arcane mystery that just never clicked. So I really wonder what's the trick to make metaprogramming shenanigans manageable at this scale?
- ptoomey3 4y ago

samwillis 4y ago

GitHub running off the main branch is fascinating, and initially sounds mad, but makes so much sense. Assuming they have very high test coverage, running against mainline Rails isn't really any different to having the fork they had before, but they have more influence on future development. It must also be a massive boon for the Rails ecosystem to have such a large property running off the head. Doesn't anyone know of any Django shops that do the same, running off mainline?

winrid 4y ago
I imagine it wouldn't be too hard, since Django only has one dependency - Django itself (NodeJS developers weep). Is rails the same way dependency wise?
- procinct 4y ago
  Wait is this true? Doesn’t it also use SQLAlchemy at least? Which then likely has its own dependencies? I’d be really surprised if Django had no dependencies at all.
  - rochak 4y ago
    I don’t think Django uses SQLAlchemy. It has its own ORM which is damn good, but you can also use SWLAlchemy instead of that if you want.
  - fernandotakai 4y ago
    django 4.2 has two dependencies: asgiref and sqlparse. other than that, none. django does not use sqlalchemy as its ORM, it has its own system (which i prefer!).
    - KyeRussell 4y ago
      Amen. Until recently, I was exclusively working with Django for 5+ years. I definitely fell into the trap of making the ORM for granted. Had a brief foray into the JS world and despite plenty of slick-looking projects with fancy pants websites, nothing remotely compares.

Intrepidd 4y ago

Working with Ruby and Rails since 2010 and absolutely loving it. Never ever looking back.

MiraiY 4y ago

meanwhile, i have to work on a rails 3.2 app with ruby 2.2

WJW 4y ago
What's preventing you from upgrading?
- MiraiY 4y ago
  the mindset "if it works, leave it there" edit: also it's a big project without specs, good luck at finding broken stuff...
  - weatherlight 4y ago
    Going to sound nuts but perhaps you can leverage ChatGPT4 to start generating some tests, based on some prompts for like 20% of your app. you just have to be pretty disciplined about how you write your specs.
    - MiraiY 4y ago
      actually, maybe not a bad idea
awesomegoat_com 4y ago
That's battle tested by now. Call it enterprise.
caseyf 4y ago
nothing wrong with Rails 3.2 :) get it on Ruby 3.1 if you can - https://railslts.com https://railslts.com
mattbrewsbytes 4y ago
Those versions are so far removed from receiving any security patches I'd probably want management to sign off that they heard me state this and they take full responsibility if shit hits the fan. Also, are you at least a little concerned about your career using a tech stack that old?

Zanfa 4y ago

It's been interesting looking into their "leaked" source to see how the application is architected. It's surprisingly accessible given the overall size.

jack_riminton 4y ago
Do you have a link? Sounds interesting
- charcircuit 4y ago
  He's probably just referring to the self hosted version of Github. https://enterprise.github.com/trial https://enterprise.github.com/trial
  - Zanfa 4y ago
    Correct! It was reported a few years back that Github's source code was leaked and it just turned out somebody deobfuscated the source shipped as part of Github Enterprise's publicly-available VM image. I don't have a direct link, but it should still be possible to run the latest image through a script to get the source.

deleted 4y ago

[deleted]

politelemon 4y ago

Good post and an interesting glimpse into the behind-the-scenes efforts that goes into maintaining Github. I cannot agree with the "Should I do it too?" section. It probably works very well for an org as large and dedicated as Github, it very likely makes a lot of sense for what they do at the scale they do it at. For most of us that are consumers of technologies and frameworks, treating the framework as an extension of an application stack, as the blog has outlined, is a terrible idea; it requires devoting time and effort towards its upkeep, and that is not our core business, nor should it be. That does not, however, mean it's being treated as an unimportant part of the application stack, and implying so is judgmental. It's importance is that you need to be careful about the tech choices you make and understand what the implications will be down the line.

awesomegoat_com 4y ago
> it requires devoting time and effort towards its upkeep, and that is not our core business, nor should it be. I would try to argue that this in fact may be subjective. For me, updating all the dependencies in my stack every morning is a productivity routine that gets me started. If I spend a 30 minutes studying changelog of a dependency and linked github issues, I don't see that time as wasted even though it often does not contribute to the business. I think it has three benefits. - easy morning start routine to get going - developer education, expanding horizons - code climate; having warn feeling in the gut of not falling behind, wanting to avoid dirty plugs and monkey patches, ability to work on the HEAD, should an issue arise.
- ysavir 4y ago
  I think the parent comment is referring to the act of submitting code changes to Rails directly whenever behavior is needed. Staying up to date with what your dependencies are doing and why makes sense for any project, but actively contributing to the dependencies isn't necessarily a great investment when it doesn't benefit your bottom line in a direct and timely matter. Github has a large enough staff that it can dedicate time to open PRs to the Rails repo without giving up its own productivity, but if you're looking at a team of 10 developers, taking time away from your own product to improve Rails can be a significant time cost.

pvsukale3 4y ago

It is so nice to have the latest releases of both Ruby and Rails run in production as soon as they are released. Huge for indie developers and small teams that use Rails. Kudos to Shopify too, who are working on improving Ruby tooling a lot recently. Exciting time to be a Rails dev.

suralind 4y ago

This make so much sense, I think every company should update deps frequently and Dependabot and other services like that can help with that. It probably would be too costly to attempt what GH is doing for smaller orgs.

snemvalts 4y ago

Github is one of the few webapps where I can feel daily that the framework used isn't enough. So many things get out of sync/not up to date, which are fixed by refreshing the page.

hakre 4y ago
Yes, just seen it yesterday. However, this is perhaps not of Github origins. Even when you use the recommended interface to git - the command line - it's only as up-to date as you've entered the last command for - or pressing F5 in gitk.
Gasp0de 4y ago
If anything that's a fault in the Frontend though, which is not what the article is about, right?
amrocha 4y ago
Very true, but that's a frontend problem, not a rails problem.
- jeremyjh 4y ago
  These problems are always more common in server-rendered apps though, because front-end state is always a patchwork. And the Rails developers and community have a strong preference for this architecture.
  - VWWHFSfQ 4y ago
    I observe the exact opposite. Server rendered UIs are far more often times up to date than client rendered apps.
    - amrocha 4y ago
      I can't think of a single website that's real time and server rendered

mo_po2 4y ago

Going a bit of topic, Arch Linux does mostly the same, and thanks to it all of its users are direct testers/reporters of unpatched projects, with the exception of a few minor patches related to chore in projects that don't allow to customize it (mostly paths). I guess that this has probably improved the overall health of Linux beyond Arch itself during the last 2 decades for similar reasons to the ones provided in this article.

jeremyjh 4y ago
Yes it’s great people use it and report issues. I found and reported a years old bug in Erlang that was exposed by a zlib update. Arch makes it easy to isolate and rollback dependencies which was helpful to isolate the change. On the other hand, Debian/RHEL users never had to know about this complete showstopper bug since they never ran a system that had the new zlib and the old Erlang at the same time.

vrglvrglvrgl 4y ago

[dead]

hakre 4y ago

Some of their points which reminded me I consider working well, also for smaller teams. * Parallel builds/CI to look into the near future: current + next. When time is due current and next need to become green and the next alpha becomes future, which is allowed to fail (3 parallel tracks: current, next and future). Keeps all your build tools en par early, too, so there is less rot, and one team can concentrate on the migration while other teams aren't interrupted by that, but also for a single or only a handful of developers, you can have better change management (at the cost of the extra computing power). * It feels less a monolith, if its in a dynamic language (not compiled / transpilled / linked) and in many files (there is little to build and deployments are smaller and faster). In such projects, increments are possible across multiple axis in a synchronized dance. * Pipeline everything and continuously shift left. Identify the most important improvements and if they can step-break the process well, apply the earliest one coming from the developer perspective. Any such change will speed up any change after that step. * Implement fast turn-around with the parts that change often so you can change them often well. Cooperate well with upstream, this is most often a key to ongoing success. * Distributing traffic over multiple application servers and being able to deploy different configurations / revisions and directing part of the traffic to it is invaluable. Have your tools/systems configured well to make it easy and comfortable to do this way. Production must not feel like an all or nothing game any longer, but serves as experimental playground.

neals 4y ago

[flagged]

that_guy_iain 4y ago
> As someone who has experience working with large codebases, I am thoroughly impressed by GitHub's approach to upgrading their Ruby on Rails monolith. It is clear that the GitHub team has invested a significant amount of time and resources to ensure that their application runs on the latest version of Rails and Ruby, and this has resulted in real tangible benefits such as better database connection handling, faster view rendering, and improved security posture. To be fair, they did spend years not adding features. (Only really starting to add features when the Dear GitHub letter came out) It seems like they spent that time wisely just investing in stuff like this. GitHub are in a unique position where they spent so long ignoring features while having a large team. I feel like for the majority of teams these sort of investments are rarely possible because feature work is required.
- irjustin 4y ago
  I down voted because it just seems like you wanted to being up the dear GitHub. It's been years... 7? with great features that go well beyond the letter. Yes gh was stagnant but this is pretty well in the past and your points have almost no current relevance.
  - that_guy_iain 4y ago
    > I down voted because it just seems like you wanted to being up the dear GitHub. I only brought up dear GitHub because I am sure people wouldn't have had an idea what I was talking about. > Yes gh was stagnant but this is pretty well in the past and your points have almost no current relevance. When my point is they used a bunch of time that other companies wouldn't give to build a solid foundation it seems very much that the past, that being the foundation, is very relevant. My entire point, is that they're in a very unique position a position that others can't really fairly expect since very few companies are able to ignore features for such a long time. This isn't to bash github. This is to put into persective what they've done and if others can do it. It removes the whole "GitHub did it so..." thing that many will suggest down the line.

dagorenouf 4y ago

I built my startup 4 years ago with a combination of react + aws + gatsby + hasura. I thought this would be great for performance and scale. Fast forward to today, I spend at least 2x as much time to code a feature than if I had just stuck with a simple rails stack, and the scale I imagined never happened. Now rebuilding everything with rails so I can ship faster and focus on growing the product, not making engineering prowesses.

faeyanpiraat 4y ago
Considering most of the times the startup at hand will fail anyway and the planned scale will not happen, building the initial architecture to be "web scale" is a perfect example of YAGNI. I had a similar experience, and it was a great lesson.
- papito 4y ago
  We used to write software for pretty high-traffic systems using what is now underpowered CPUs, less RAM, and antiquated databases. It should be easier now, NOT harder. Most of development I see today is resume-driven and not based in any pragmatism.
AeroNotix 4y ago
Building for scale from day 0 is a recipe for disaster. You should've seen that coming.
- Alifatisk 4y ago
  You suggest people should refactor later on when needed?
  - shortrounddev 4y ago
    There comes an inflection point in a startup when you have to move from MVP to scale, and there are two different kinds of tech and two kinds of people for each stage
  - 4y ago

rethab 4y ago

> scheduled GitHub Action workflow ..sounds like they're not dogfooding dependabot? curious if anybody knows more/why

heartbreak 4y ago
Dependabot is for security issues, not this?
- rethab 4y ago
  dependabot can also be used for regular version updates
deleted 4y ago
[deleted]
djm_ 4y ago
Dependabot is based on releases from the various package repositories; running of main is pre-release - hence they’re probably using GitHub Actions to pin their Gemfile-defined Rails version to a commit hash.

qntmfred 4y ago

Every time it's pointed out that GitHub is a rails app I am reminded of one of my favorite security exploits ever https://news.ycombinator.com/item?id=3663197 https://news.ycombinator.com/item?id=3663197

peter_retief 4y ago

Another big rails application was twitter, it has been rewritten but the concept fitted well with rails. Twitter currently uses some java framework as far as I know. Question, does twitter still have some RoR deployed?

mrwnmonm 4y ago

All the cool kids left Ruby for Go, why are they still in that land?

rst 4y ago
Because they care more about getting things done than whether clout-chasers think they're "cool".
- throwaway445232 4y ago
  [flagged]
- mrwnmonm 4y ago
  I get things done just fine with Go. And all the code is very clear. With ruby it is not, and a lot of ruby programmers may have a feel about how things work, while they are not really. They just enjoy the syntax. The book "The Ruby Programming Language" which written by the creator or ruby (I forgot his name) is full of "unfortunately", unfortunately this works this way now, unfortunately that works that way now. And people definitly chose ruby because it was cool. So what you said should be applied to Ruby, not Go.
elif 4y ago
Since they left ruby (2.0) raw rails performance improved by 75% (3.0) https://www.fastruby.io/blog/assets/images/RRBPerfHistory_720.png https://www.fastruby.io/blog/assets/images/RRBPerfHistory_72... Then in 3.2, ruby released YJIT which improved median rails response time by about 100% https://raw.githubusercontent.com/easydatawarehousing/ruby_measure_responsetime/main/data/rails_devise/plots/rails_devise_0_overview.png https://raw.githubusercontent.com/easydatawarehousing/ruby_m... It's kind of a golden era for rubyists that stuck it out. Beautiful maintainable code that hits C performance for many tasks.
- Alifatisk 4y ago
  "... hits C performance for many tasks" Really?

qrush 4y ago

Absolutely love this approach and I think it makes a ton of sense for an open source focused company - however I haven't seen this approach in any company I've worked for in 15 years (where I have helped with most of the Rails upgrades!). There's a ton of merit to being on the latest release or close to it so you can get the latest security patches easier. The diff between that and the latest `main` branch seems like it has diminishing returns for most orgs.

compumike 4y ago

Even as a much smaller team, building Heii On-Call [0] as a lightweight alerting/monitoring/on-call rotations SaaS based on Ruby on Rails has basically been a pleasure! And as the article highlights, perhaps the key reason for smooth deployments and upgrades is that the CI testing story is so, so good: RSpec [1] plus Capybara [2] for us. That means we have decently extensive tests of just about all behavior. The few small Rails and Ruby upgrades we've done have gone quite smoothly and confidently, with usually just a few non-Rails gem dependencies needing to be manually updated as well. The "microservices" story is where we've pulled in the Crystal programming language [3] to great effect. After dabbling with Go and Rust, we've found that Crystal is truly a breath of fresh air. Crystal powers the parts of Heii On-Call that need to be fast and low-RAM, specifically the inbound API https://api.heiioncall.com/ https://api.heiioncall.com/ and the outbound HTTP(S) prober background processes. I've ported some shared utility classes from Ruby to Crystal almost completely by just copy-and-pasting ___.rb to ___.cr; porting the tests for those classes was far more onerous than porting the class code itself. (Perhaps another point of evidence toward the superiority of RoR's testing story...) The front-end story is nice but just a bit weaker. Using Hotwire / Turbo successfully, but I have an open PR to fix a fairly obvious stale cache bug in Turbo [4] that has been sitting unloved for nearly a month, despite other users reporting the same issue. I'm hopeful that it will get merged in the next release, but definitely less active than the backend side. For me, the key conclusion is that the excellent Ruby on Rails testing story is what enables everything to go a lot more smoothly and have such a strong foundation. I'd be curious if any GitHubbers can talk more about whether they too are using Rspec+Capybara or something else? Are there internal guidelines for test coverage? [0] https://heiioncall.com/ https://heiioncall.com/ [1] https://rspec.info/ https://rspec.info/ [2] https://github.com/teamcapybara/capybara https://github.com/teamcapybara/capybara [3] https://crystal-lang.org/ https://crystal-lang.org/ [4] https://github.com/hotwired/turbo/pull/895 https://github.com/hotwired/turbo/pull/895

jack_riminton 4y ago
My startup is using minitest (mainly for running tests in parallel I believe) and capybara. Crystal looks great, are you using it mainly for type checking? If so why not Sorbet?

elif 4y ago

I wonder how much they end up having to unit test gems they use, and how much they have to maintain gem compatibility for those maintainers.. I have to imagine this leads to either GitHub being the defacto lead maintainers for those gems, or GitHub removing gems from their stack and writing their own code.

cutler 4y ago

I dunno. I lived through Internet Explorer, SCO and Linux patent lawsuit days so I reckon it's only a matter of time before Github goes ASP.Net.

nobleach 4y ago
I lived through Microsoft purchasing Hotmail and trying repeatedly to migrate it from FreeBSD to IIS/Windows2000. We may have a good 20 years.
hocuspocus 4y ago
A friend at GH was already writing mostly C# and Go before he moved to engineer manager 2+ years ago, so I believe it's already ongoing. Also he absolutely hated having to jump into components in RoR before Codespaces was available internally.
- dewski 4y ago
  C# is used for Actions: https://github.com/actions/runner https://github.com/actions/runner, and Go is used a lot for internal services. There is no traction rewriting our monolith in C#.

thih9 4y ago

> Every Monday a scheduled GitHub Action workflow triggers an automated pull request, which bumps our Rails version to the latest commit on the Rails main branch for that day. So uploading code to a github repo makes github server execute it later on. I wonder if this would qualify for a bug bounty... /s Edit: I guess not, since this is just a pull request and still requires approval before merging to main. I assume.

georgyo 4y ago
Don't work for GitHub, but it would not. First, it opens a Pull Request, not an automatic merge, so hopefully the code is reviewed before merge. To exploit this, you would first need to get malicious code merged into rails master which has many eyes, and then get passed more eyes when it gets reviewed by GitHub. Not impossible, but if you got your code merged into rails master, you have wiggled your way into many more environments than just GitHub.
nicce 4y ago
Even if it was just not pull-request, highly unlikely that it would qualify. We can assume that only trusted maintainers can put code into the Ruby main branch. Similarly, every build system depends on many many signed packages. It is not much different than that.

uptownJimmy 4y ago

I like GitHub, fwiw. I am not someone who requires everything to be perfect, and I've learned to be tolerant of our human reality, where imperfection is the norm. But GitHub is not a great Web app. It is frequently/constantly out of sync with the latest data/status. You quickly develop the habit of manually refreshing the page every time you are preparing to do anything with a PR, and that's not something that should be necessary these days. It surprises me to see a loud and proud blog post detailing GitHub's process of staying so relentlessly up to date with the latest and greatest version of Rails: the app is not properly responsive, for whatever reason, and to a degree that would not be tolerated where I have worked. I would rather read about how they are trying to fix this issue.

ysavir 4y ago
I agree with this, but that seems to be a problem with Github's frontend code, not their server code. I don't think that detracts from this blog post's message.
- uptownJimmy 4y ago
  I guess my implied question should be stated more explicitly: is GitHub's front end code entirely separate fro the Rails code? That's not how Rails apps usually work, I thought. Admittedly, it's been a long time since I looked at Rails code, and I don't have the slightest idea how GitHub is actually architected. But I don't remember the "front end code" in Rails being a separate thing from the server code, typically: the whole point was server-side rendering. Is GitHub using one of the JS/TS frameworks for the front end? I do remember that being a developing trend, a few years back.
  - danjac 4y ago
    Rails can just serve JSON or GraphQL from an API to an SPA frontend, or you can do full server-rendered, or Hotwire to do HTML fragment updates, or any combination thereof. IIRC Github used something like Hotwire but home-grown. I've not done Rails development for a while so not sure what the state of things is for web sockets, but I would think that's not a problem for the framework. Point is, there's nothing about Rails in particular that would prevent fixing these issues, that's probably the result of development and business priorities, legacy code, etc. that would be an issue with any tech stack.

exabrial 4y ago

I think the big lesson here is writing excellent tests and making them easy to execute makes your code easier to change fear of breaking something. Unfortunately I’m fairly certain the message that will be heard is ‘We should totally change something around on the customer every week, that’s what GitHub does’

surteen 4y ago

Are there other languages where the dominant web framework changes so much? Will Rails ever be "done"?

no_wizard 4y ago
.NET and Django are The pillars of backward compat and stability IMO. Change is somewhat inevitable though
donapieppo 4y ago
I hope not.
ativzzz 4y ago
It can never be done as long as the web keeps evolving and the web shows zero signs of slowing down evolution And those changes are the reason rails is still really good, modern, and not considered legacy software almost 20 years later
throwaway445232 4y ago
[flagged]

alberth 4y ago

> Every Monday a scheduled GitHub Action workflow triggers an automated pull request, which bumps our Rails version to the latest commit on the Rails main branch for that day. That’s a bold move to do as opposed to being end of week or weekend.

prepend 4y ago
I like Monday releases as if something goes wrong everyone is around to fix it. If something breaks on Friday it ruins weekends. I think Monday requires more maturity and more successes as it prioritizes dev time over downtime. Saturday outages affect fewer customers but are hard on staff.
- swat535 4y ago
  We don't even release on Mondays, its too rushed. We ship between Tuesday and Thursday instead and only during productive hours (e.g 10AM-3PM). We want the engineers the be alert and calm.
lcnPylGDnU4H9OF 4y ago
I sure don’t want to be called on Saturday to be told that there’s something wrong with Friday’s build.
- alberth 4y ago
  I’m sure paying customers don’t want outages during their prime usage days.
  - javier2 4y ago
    How do we achieve that? Releasing often and in smaller increments. A release causing an outage should be easier to test than the subtle bugs that come creeping one, two, three days later. Either way, if there is a catastrophic failure, its better to have everyone readily available.
mousetree 4y ago

TheRealDunkirk 4y ago

I've been using Rails and GitHub since the first 2.x days. I have been expecting an announcement that they are switching to .NET since Microsoft bought them. I'm guessing it's too big of a bite, even for Microsoft, thank Ada. I love being able to point recalcitrant policy makers at my company at GitHub as an example of a top web app -- the context for which they already know very well -- being a Rails monolith. I'd love to see their Gemfile.

mdaniel 4y ago
> I'd love to see their Gemfile. I would bet it's present in GHE's OVA (https://enterprise.github.com/releases/3.8.1/download https://enterprise.github.com/releases/3.8.1/download)

nomilk 4y ago

> Instead of telling your team you found something in Rails that will be fixed in the next release, you can work on something in Rails and see it the following week! That's awesome. Not only fixing it for your team but the entire rails world.

silverwind 4y ago
> Not only fixing it for your team but the entire rails world Or breaking it for some.

danforddotdev 4y ago

I'm curious what their data access layer looks like underneath that monolith. Is the Rails piece mostly now just a frontend for dozens of other services, properly owned and maintained by other teams? I don't mean to trivialize something that's obviously huge and complex as "just a frontend", but IME one of the biggest things that breaks down in a Rails monolith as it scales is heavy, direct usage of ActiveRecord. Either there's lots of DB migrations happening since there's so many different developers working on different features, which makes development with a shared DB tricky, or the scale makes DB performance problems hard to diagnose since they cut across many teams or tables in complicated ways.

withinboredom 4y ago
I've heard this several times over the last few months. Like what makes several devs working in the same area of the code "hard"? In my experience whomever is lucky enough to commit first gets the easiest of it, everyone else just rebases and resolves their conflicts. Maybe if you don't rebase and merge instead? I've seen some screwed up stuff happen from bad merges... like entire lines of code vanish. But generally, I've worked with hundreds of devs in the same code base without issue. So, why do people ask this?
- danforddotdev 4y ago
  Well, I gave a specific example in Rails, using DB migrations against a shared DB. It’s not an unsolvable problem, and of course each dev can have their own DB, but if this is poorly managed it’s easy to become unwieldy. Outside of that, if many devs are constantly making dependency changes such that every time you “git pull” you have to rebuild environments, etc. Maybe devs are adding features but not prepopulating dev environments with sensible test data so your dev environment gets horked. Etc etc. Its not usually about merging the code itself.
jakemoffatt2 4y ago
Can't speak to how GitHub does it, but in every Rails org I've been at, eventually we create secondary services that own a specific domain, and the primary app becomes a gateway that clients use to talk to those services. It's rare to spin up a new service in a new language/framework. A typical pattern is to migrate the service within the monolith into a Rails engine, and then move that to a new app where the app mounts only that engine, or one can find a way to make the monolith deployable with only certain engines mounted, like with an env var. A Rails engine is basically a self contained Rails app, including routes, which you can mount inside of a host Rails app at any route if your choosing. They're usually used to build reusable libraries, but this use case also works very well.

ksec 4y ago

And as I have said before but worth repeating again, Rails is perhaps the only open source framework that is being battle tested in development at scale. It may not be the fastest ( or in fact quite slow ), but I dont think you could find similar testing being done and Deployed at the scale of Github on any other framework. I wonder if Eileen Uchitelle will bring this practice to Shopify as well? Edit: It seems [1] Shopify is also running on latest Ruby and Rails version as well. [1] https://news.ycombinator.com/item?id=35481352 https://news.ycombinator.com/item?id=35481352

hocuspocus 4y ago
> but I dont think you could find similar testing being done and Deployed at the scale of Github on any other framework. ?! There's at least Spring.
- ksec 4y ago
  It's been a very very long time I touched anything Java so excuse my ignorance. I know there are many Enterprise, internal Web Apps that uses Spring. But are there any web companies built their SaaS or Consumer Web App on Spring, that is at the scale of Shopify and Github?
  - hocuspocus 4y ago
    Most companies at that scale don't use a single framework or even language but there's probably more Spring at Amazon than RoR at GitHub, Stripe and Shopify combined.
    - throwaway2847 4y ago
      Stripe doesn't use Ruby on Rails, they have a framework developed in house
  - lenkite 4y ago
    Aliexpress and Udemy come right at the top of my mind. But there are quite a few more. (several German web-sites like Trivago). Spring Boot is used more outside the U.S. - Java is respected more in the rest of the world.

trevor-e 4y ago

Does anyone know if Github uses Sorbet for their Ruby and Rails code?

dewski 4y ago
We do.

didip 4y ago

I must be the only person on Hacker News who dislike Ruby. It simply has too much magic and doesn't offer enough abilities (unlike Elixir or any Scheme variant) to justify the dynamic type nature. Go and Rust are perfect for me (and maybe Zig?). They covered everything I need.

hocuspocus 4y ago
No, you aren't the only one. I used RoR at its peak hype in 2009 and I wasn't impressed. Even super boilerplaty alternatives at the time (like Spring 2.x and iBatis) were a lot more manageable in my opinion. And the few good ideas from frameworks like RoR and Django have long been stolen by all ecosystems, so I don't really see the point anymore.
bastardoperator 4y ago
None of the languages you listed/prefer existed when GitHub was launched, lol.
sosodev 4y ago
I love Go but Rails is just unmatched in terms of development velocity. You're right that Go eschews magic and embraces verbosity but in doing so it makes common tasks, like web development, take 10x longer than languages that do the opposite.
vc8f6vVV 4y ago
It's only magic if you don't know what's going on under the hood. I felt this way for the first couple of years until I figured it out (experience matters as with every other language/platform). Then there was no magic, just conventions. You can do crazy things with Ruby and people tend to do them thinking it's "cool". Just don't. Rails doesn't have many and they are pretty well documented.

insane_dreamer 4y ago

Oh, and here I was told that RoR is only for quick-and-dirty prototyping and that "real" web applications are written in ___ (fill in blank)

larusso 4y ago

> Ultimately, if more companies treated the framework as an extension of the application, it would result in higher resilience and stability. Investment in Rails ensures your foundation will not crumble under the weight of your application. Treating it as an unimportant part of your application is a mistake and many, many leaders make this mistake I really like the sentiment of this quote but that is an easy thing to say for a behemoth like GitHub. I find these blog posts from google, meta, aws etc super awesome since they run and solve problems smaller companies simply don’t have. And they shape how smaller companies solve similar issue at smaller scale. But doing a setup to be practically on bleeding edge rails for example is not something every company can afford. They need LTS releases and the sorts. Still awesome that they managed to achieve this.

stevebmark 4y ago

This is a very interesting cautionary tail for those who recommend Rails because Github uses it. It's hard to imagine another core framework where the "best" way to use it is to dedicate an entire department to continuously working off the bleeding edge. And if you read between the lines, changes that Github needs to make are within Rails itself, and as we know there are core Ruby and Rails maintainers working at Github. This is a cautionary tale because of how much overhead Github needs to make this work. If your company has the klout to hire core Rails maintainers and the staff to be able to focus on working off the main branch of one of your framework dependencies, then go for it! You should also read about Github's history with "Rails" and how challenging it was for them to make it fast enough for them and upgrade it. It's pretty interesting they got around that "challenge" by throwing a LOT more resources at it. It's interesting because this isn't usually a challenge with other major framework upgrades. It makes sense you would need this much investment given the hyper-dynamic dangers of Ruby (not just the language, but the ecosystem), making it difficult and risky to upgrade. TL;DR the advice in this blog post does not seem applicable to most companies.

aledalgrande 4y ago

This is awesome, now I just wish they spruced up their UI with Hotwire :p How many times have you clicked the back button and found the same notifications unread?

benatkin 4y ago

I've been using Codeberg, which uses Forgejo which is written in Go and is fast and light. Fantastic that GitHub has managed to wrangle so many lines of code in a language I don't care very much for, but my Samsung A53's browser is snappier without it :) Edit: to be fair, GitHub's has a hamburger menu that morphs to an X that I dearly miss. JK :P

Alacart 4y ago
Codeberg runs go on the frontend?
- deleted 4y ago
  [deleted]

funnyfoobar 4y ago

How does the current Architecture of Github looks like, Apart from Rails, I have heard that they use Go as well.

hmontazeri 4y ago

Ruby and Rails are the single reasons why my previous startup worked out. We had a small engineering team and had to move incredibly fast. Without the magic we wouldn’t have been able to ship at insane speed. I tattooed the ruby logo on my arm as this was an absolute life changer for me. Having financial freedom now lets me spend so much time with my kids. I’ll be forever grateful.

herbst 4y ago
I totally feel you. Ruby & Rails are part of the reason I work way less than most of my peers for the same amount of money or more. Being able to built publish ready things, alone, within only weeks instead of months is a total life changer if you like building things