9 ms·

GitHub running off the main branch is fascinating, and initially sounds mad, but makes so much sense. Assuming they have very high test coverage, running agains

GitHub running off the main branch is fascinating, and initially sounds mad, but makes so much sense. Assuming they have very high test coverage, running against mainline Rails isn't really any different to having the fork they had before, but they have more influence on future development. It must also be a massive boon for the Rails ecosystem to have such a large property running off the head. Doesn't anyone know of any Django shops that do the same, running off mainline?

winrid 4y ago
I imagine it wouldn't be too hard, since Django only has one dependency - Django itself (NodeJS developers weep). Is rails the same way dependency wise?
- procinct 4y ago
  Wait is this true? Doesn’t it also use SQLAlchemy at least? Which then likely has its own dependencies? I’d be really surprised if Django had no dependencies at all.
  - rochak 4y ago
    I don’t think Django uses SQLAlchemy. It has its own ORM which is damn good, but you can also use SWLAlchemy instead of that if you want.
  - fernandotakai 4y ago
    django 4.2 has two dependencies: asgiref and sqlparse. other than that, none. django does not use sqlalchemy as its ORM, it has its own system (which i prefer!).
    - KyeRussell 4y ago
      Amen. Until recently, I was exclusively working with Django for 5+ years. I definitely fell into the trap of making the ORM for granted. Had a brief foray into the JS world and despite plenty of slick-looking projects with fancy pants websites, nothing remotely compares.
      - rajamaka 4y ago
        What JS based ORMs did you use that you weren't a fan of?
        AeroNotix 4y ago
        All of them are bad in their own ways. They all fall down in key features, or the non-standard SQL features (hooks, after-save, automatic transactions, etc) all have edge-cases and surprising behaviour. I have tried a lot and the least worst is Zapatos (and it's not really an ORM) because it at least tries to not paper over SQL and instead just creates a type-safe API for using SQL.
        ackfoobar 4y ago
        Slightly related, I'm not used to using ORMs. Spoiled by jooq from the JVM world, writing SQL with the query builder in TypeORM was a terrible experience.
        winrid 4y ago
        The thing is - even if someone creates an amazing ORM for Node, I doubt I'd use it. I am tired and done with async everything. No more NodeJS for business logic. Now, if someone could create a Django-like ORM for Java or Rust, then we're talking. Hibernate and Diesel are nowhere near Django's ORM in terms of productivity and "it just works" factor. Go's GORM looks pretty good but haven't tried it. jooq does not look as easy to use or setup, and the workflow is entirely different.
      - kdazzle 4y ago
        Same here, I came into a project that had been started months before and found that they were spending weeks reimplementing stuff that comes out of the box in Django. But not as well.
  - black3r 4y ago
    Django doesn't use SQLALchemy, they have their own ORM system. And it's true that Django used to have 0 real dependencies (now it has 2), but even now many of its "optional builtins" do require some additional external packages, and some of them are really useful. For example you need an external package to use builtin postgresql, mysql or oracledb support. You need tblib to run tests in parallel using the built-in test runner. You need external packages to use argon2 or bcrypt as password hashers in the builtin auth system, and I'm sure there are others, since Django is very much batteries included, but modular. The modularity also means you can also use any 3rd party database, use pytest to run tests, make your password hashing on your own, ... so I can understand that these are not listed as dependencies on pypi, but I don't really like that I have to list packages I don't directly import in my code as direct dependencies...
  - simonw 4y ago
    Django had zero external dependencies for a long time, which was a factor of its history. The first release of Django was in 2005. Back then, the Python Package Index didn't exist yet. Installing Python dependencies was really hard - you pretty much had to grab a copy of the code for each one and put it on your "sys.path" somehow. So Django avoided the issue entirely by bundling everything you needed to build a web application in a single package. That's why Django has "django.contrib" - in a time before pip dependencies, it was a way to separate out things like GeoDjango which weren't exactly part of the "core" framework but could be distributed along with it.
- Aeolun 4y ago
  Ah, NodeJs developers are fine with a single dependency too. As long as that dependency is NPM.
- wg0 4y ago
  That's not correct. Django isn't independent, depends on lot many packages but that's not the issue. Issue is the frameworks own Interface: Existing Interface getting removed OR behavior or defaults changing for an already existing Interface. That's usually the upgrades are about.
  - winrid 4y ago
    Yes the framework's interface can change. But this is usually a piece of cake compared to hundreds of disjoint libraries and leaky abstractions that would be in use in the Node world.
  - Kwpolska 4y ago
    > Django isn't independent, depends on lot many packages It is quite independent. There are between two and four dependencies: asgiref, sqlparse, tzdata on Windows only [0], and typing_extensions on <3.11 [1]. There are some optional dependencies (argon2-cffi, bcrypt, and a database library like psycopg2), but they are small and mostly self-contained. [0] https://github.com/django/django/blob/main/setup.cfg#L39-L42 https://github.com/django/django/blob/main/setup.cfg#L39-L42 [1] https://github.com/django/asgiref/blob/main/setup.cfg#L34-L35 https://github.com/django/asgiref/blob/main/setup.cfg#L34-L3...
- danielheath 4y ago
  Rails has dependencies, but not like node apps; last time I counted create-react-app vs a new rails app, there was nearly 10x as many distinct maintainers with access to push new releases to the default dependency tree. I consider that a better measure of dependency risk than absolute count, since different ecosystems have different ideas about how large a library should be.
  - nicoburns 4y ago
    Create-react-app isn't really a fair comparison. Create-react-app has ridiculous dependency bloat compared to the norm in the JS ecosystem because it includes every possible option rather than just picking one option in each category. Most serious project using React aren't using create-react-app.
    - danielheath 4y ago
      I’m not really familiar enough to contradict you, beyond saying that - as an outsider - it has by far the most visibility in its space.
      - iudqnolq 4y ago
        Funnily it's now outdated. The brand new react docs don't even mention it: https://react.dev/learn/start-a-new-react-project https://react.dev/learn/start-a-new-react-project > We are currently leaning towards Option 5 ("Turn Create React App into a launcher"). The original goal of Create React App was to provide the best way to start a new React web app for the majority of React users. We like that repurposing it as a launcher explicitly communicates a shift in what we think is best for most new web apps, while at the same time leaving an escape hatch for the older workflows. Unlike Option 3, it avoids the perception that "creating a React app" is somehow deprecated. It acknowledges the reality that a bit of choice is needed, but there are now really great options to choose from. https://github.com/reactjs/react.dev/pull/5487#issuecomment-1409720741 https://github.com/reactjs/react.dev/pull/5487#issuecomment-...
jack_riminton 4y ago
Exactly, it’s an investment in Rails by tying yourself so closely with it. Contrast to the JS ecosystem where as soon as there’s a disagreement or new idea a new framework is born
- z3t4 4y ago
  Except for the JS language itself where every proposed suggestion is implemented.
  - pjmlp 4y ago
    Errr, tail calls.
    - roguas 4y ago
      Every reasonable. Maybe I am wrong, but tco usually means no more stacktraces?
      - pjmlp 4y ago
        No, only dumb implementations, Scheme and other languages that require TCO have no issues mapping the debugging metadata to proper calls.
      - deleted 4y ago
        [deleted]
      - AlchemistCamp 4y ago
        Not at all! Elixir stacktraces are much nicer than JavaScript ones.
    - anonymoushn 4y ago
      stackful coroutines, an "await" keyword that only awaits one layer of awaitables instead of infinitely many layers, etc.
    - Jarred 4y ago
wg0 4y ago
I have yet to see a tech stack that's not locked up into a framework version. That's more because of our engineering culture. The cost of NOT upgrading outweighs by a huge margin than keeping building on top. And yes, have seen Django shops locked into 0.9x release patched right into the core and running for a very long time, impossible to upgrade and all the horror stories. EDIT: Added Django
- ecshafer 4y ago
  At Shopify we also are basically always on the bleeding edge of all Rails and Ruby versions. By always moving forward it makes each improvement much smaller
  - sjs 4y ago
    Plus Shopify and GitHub must drive most development of Rails so it’s basically an in-house dependency to some extent. I know that’s simplifying things.
- wlll 4y ago
  As a counterpoint I have yet to see one that is locked in my 15 years working with Rails.
  - pmontra 4y ago
    I have customers with a mix of small Rails projects in version 6 or 7. I told one this week that we could use AR's encrypts :field to encrypt some fields in the database, but that project is Rails 6 and that feature is added by Rails 7. We could add the code to handle it because it's small, but customer decided not to encrypt the field and wait until (if!) the project moves to Rails 7.
    - wlll 4y ago
      I wouldn't say that's locked though. The place I currently work is on 6.1. We're not locked to it, we're going to upgrade, but in a few weeks or so. I can understand the reason to put off encrypting the field too. Upgrades, or backporting, are just work like any other project and I guess that other things for now take preference. After all, before field encryption came along the risk balance calculation had already been done (even if only implicitly) that storing the field unencrypted in the database was safe enough. The availability of encrypted fields doesn't change that, though making some things safer doesn't necessarily fall neatly into the "requirements" bucket despite being a good idea.
danjac 4y ago
I remember working on projects that did this in the early days of Django, particularly when there were badly needed features that took too long to land in the stable release. Nowadays most projects I've seen use the latest stable release or at least the latest LTS - maybe some legacy projects lagging behind.
tiffanyh 4y ago
> “It must also be a massive boon for the Rails ecosystem to have such a large property running off the head.” I would have expected Microsoft to focus on developer efforts into speeding up Ruby as a language given they are one of a small few large companies that have deep language/compiler expertise.
- wlll 4y ago
  Much as I'd like Ruby to be faster it's rarely the bottleneck in most Rails applications, besides, Ruby is becoming faster, it's a lot better than it used to be.
- weatherlight 4y ago
  It's usually I/O and dDB calls that are the bottle neck, not the speed of the language. Although, Ruby and be a bit of a memory hog,Ii did hear though, that this got better with the new YJIT in Ruby 3.x.
  - Rapzid 4y ago
    > It's usually I/O and dDB calls that are the bottle neck Not in my experience.
    - weatherlight 4y ago
      In a Ruby on Rails app?
- ksec 4y ago
  Unfortunately Github is the only product that uses Ruby inside Microsoft. So their incentive is pretty small. I do wish someone inside Github could push M$ to spend some money and collaborate with Shopify on their JIT effort though.
  - kovacs 4y ago
    That's actually not true. There's at least one other product at MS that uses ruby... Yammer. Although I'm not sure how much more incentive that would add for MS.
WirelessGigabit 4y ago
You MUST have an insane amount of test coverage to trust something like this to Ruby. I'm new to Ruby, with 13 yoe as a SW engineer. Personally, I find it a very hard language to master. Writing tests often feels like I'm settings variables left and right without seeing them being used in the current context. But that then happens to be part of the let() way in rspec. Now you might say: why use rspec? I inherited this codebase, so gotta do what you gotta do. I do really miss my compiler. I am not a fan of writing a block somewhere that can be invoked 2 weeks later for the first time and then fail, because someone passed in a number where a string was expected. I went from PHP to Java to C# to F# to JS to Rust. F# and Rust stand out in terms of hardest to write, but easiest to trust. I don't have that feeling with Ruby, and RoR. But, again, my personal opinion. Good friend of mine started with Ruby, and he loves it. He says that he accepts the magic things for what they are, and uses them. My brain doesn't allow that. I have to understand.
- winrid 4y ago
  Maybe some day Ruby will get type hinting. I'm loving it in Python. In terms of the "I wrote a block somewhere and it broke when executed for the first time two weeks later" - don't do that in a scripting language. Use the REPL to build/test the block. That's the trade off - instead of a compiler you get a fast REPL - use it! :)
  - ok123456 4y ago
    it already does
    - willlma 4y ago
      Please say more. What's your setup?
      - teaearlgraycold 4y ago
        Sorbet exists, but it’s worse than what Python offers, which is dramatically worse than what TypeScript did to JS.
aledalgrande 4y ago
Shopify also runs off main, with a bigger codebase and more devs, and employs some Ruby core/Rails core devs too. It is the right choice IMO. Once you decide on your main framework as a big company, you should invest as much as you can in it and its community, it will pay off in the long term. https://shopify.engineering/shopify-monolith https://shopify.engineering/shopify-monolith

winrid 4y ago

I imagine it wouldn't be too hard, since Django only has one dependency - Django itself (NodeJS developers weep). Is rails the same way dependency wise?

procinct 4y ago
Wait is this true? Doesn’t it also use SQLAlchemy at least? Which then likely has its own dependencies? I’d be really surprised if Django had no dependencies at all.
- rochak 4y ago
  I don’t think Django uses SQLAlchemy. It has its own ORM which is damn good, but you can also use SWLAlchemy instead of that if you want.
- fernandotakai 4y ago
  django 4.2 has two dependencies: asgiref and sqlparse. other than that, none. django does not use sqlalchemy as its ORM, it has its own system (which i prefer!).
  - KyeRussell 4y ago
    Amen. Until recently, I was exclusively working with Django for 5+ years. I definitely fell into the trap of making the ORM for granted. Had a brief foray into the JS world and despite plenty of slick-looking projects with fancy pants websites, nothing remotely compares.
    - rajamaka 4y ago
      What JS based ORMs did you use that you weren't a fan of?
      - AeroNotix 4y ago
        All of them are bad in their own ways. They all fall down in key features, or the non-standard SQL features (hooks, after-save, automatic transactions, etc) all have edge-cases and surprising behaviour. I have tried a lot and the least worst is Zapatos (and it's not really an ORM) because it at least tries to not paper over SQL and instead just creates a type-safe API for using SQL.
      - ackfoobar 4y ago
        Slightly related, I'm not used to using ORMs. Spoiled by jooq from the JVM world, writing SQL with the query builder in TypeORM was a terrible experience.
      - winrid 4y ago
        The thing is - even if someone creates an amazing ORM for Node, I doubt I'd use it. I am tired and done with async everything. No more NodeJS for business logic. Now, if someone could create a Django-like ORM for Java or Rust, then we're talking. Hibernate and Diesel are nowhere near Django's ORM in terms of productivity and "it just works" factor. Go's GORM looks pretty good but haven't tried it. jooq does not look as easy to use or setup, and the workflow is entirely different.
    - kdazzle 4y ago
      Same here, I came into a project that had been started months before and found that they were spending weeks reimplementing stuff that comes out of the box in Django. But not as well.
- black3r 4y ago
  Django doesn't use SQLALchemy, they have their own ORM system. And it's true that Django used to have 0 real dependencies (now it has 2), but even now many of its "optional builtins" do require some additional external packages, and some of them are really useful. For example you need an external package to use builtin postgresql, mysql or oracledb support. You need tblib to run tests in parallel using the built-in test runner. You need external packages to use argon2 or bcrypt as password hashers in the builtin auth system, and I'm sure there are others, since Django is very much batteries included, but modular. The modularity also means you can also use any 3rd party database, use pytest to run tests, make your password hashing on your own, ... so I can understand that these are not listed as dependencies on pypi, but I don't really like that I have to list packages I don't directly import in my code as direct dependencies...
- simonw 4y ago
  Django had zero external dependencies for a long time, which was a factor of its history. The first release of Django was in 2005. Back then, the Python Package Index didn't exist yet. Installing Python dependencies was really hard - you pretty much had to grab a copy of the code for each one and put it on your "sys.path" somehow. So Django avoided the issue entirely by bundling everything you needed to build a web application in a single package. That's why Django has "django.contrib" - in a time before pip dependencies, it was a way to separate out things like GeoDjango which weren't exactly part of the "core" framework but could be distributed along with it.
Aeolun 4y ago
Ah, NodeJs developers are fine with a single dependency too. As long as that dependency is NPM.
wg0 4y ago
That's not correct. Django isn't independent, depends on lot many packages but that's not the issue. Issue is the frameworks own Interface: Existing Interface getting removed OR behavior or defaults changing for an already existing Interface. That's usually the upgrades are about.
- winrid 4y ago
  Yes the framework's interface can change. But this is usually a piece of cake compared to hundreds of disjoint libraries and leaky abstractions that would be in use in the Node world.
- Kwpolska 4y ago
  > Django isn't independent, depends on lot many packages It is quite independent. There are between two and four dependencies: asgiref, sqlparse, tzdata on Windows only [0], and typing_extensions on <3.11 [1]. There are some optional dependencies (argon2-cffi, bcrypt, and a database library like psycopg2), but they are small and mostly self-contained. [0] https://github.com/django/django/blob/main/setup.cfg#L39-L42 https://github.com/django/django/blob/main/setup.cfg#L39-L42 [1] https://github.com/django/asgiref/blob/main/setup.cfg#L34-L35 https://github.com/django/asgiref/blob/main/setup.cfg#L34-L3...
danielheath 4y ago
Rails has dependencies, but not like node apps; last time I counted create-react-app vs a new rails app, there was nearly 10x as many distinct maintainers with access to push new releases to the default dependency tree. I consider that a better measure of dependency risk than absolute count, since different ecosystems have different ideas about how large a library should be.
- nicoburns 4y ago
  Create-react-app isn't really a fair comparison. Create-react-app has ridiculous dependency bloat compared to the norm in the JS ecosystem because it includes every possible option rather than just picking one option in each category. Most serious project using React aren't using create-react-app.
  - danielheath 4y ago
    I’m not really familiar enough to contradict you, beyond saying that - as an outsider - it has by far the most visibility in its space.
    - iudqnolq 4y ago
      Funnily it's now outdated. The brand new react docs don't even mention it: https://react.dev/learn/start-a-new-react-project https://react.dev/learn/start-a-new-react-project > We are currently leaning towards Option 5 ("Turn Create React App into a launcher"). The original goal of Create React App was to provide the best way to start a new React web app for the majority of React users. We like that repurposing it as a launcher explicitly communicates a shift in what we think is best for most new web apps, while at the same time leaving an escape hatch for the older workflows. Unlike Option 3, it avoids the perception that "creating a React app" is somehow deprecated. It acknowledges the reality that a bit of choice is needed, but there are now really great options to choose from. https://github.com/reactjs/react.dev/pull/5487#issuecomment-1409720741 https://github.com/reactjs/react.dev/pull/5487#issuecomment-...

jack_riminton 4y ago

Exactly, it’s an investment in Rails by tying yourself so closely with it. Contrast to the JS ecosystem where as soon as there’s a disagreement or new idea a new framework is born

z3t4 4y ago
Except for the JS language itself where every proposed suggestion is implemented.
- pjmlp 4y ago
  Errr, tail calls.
  - roguas 4y ago
    Every reasonable. Maybe I am wrong, but tco usually means no more stacktraces?
    - pjmlp 4y ago
      No, only dumb implementations, Scheme and other languages that require TCO have no issues mapping the debugging metadata to proper calls.
    - deleted 4y ago
      [deleted]
    - AlchemistCamp 4y ago
      Not at all! Elixir stacktraces are much nicer than JavaScript ones.
  - anonymoushn 4y ago
    stackful coroutines, an "await" keyword that only awaits one layer of awaitables instead of infinitely many layers, etc.
  - Jarred 4y ago

wg0 4y ago

I have yet to see a tech stack that's not locked up into a framework version. That's more because of our engineering culture. The cost of NOT upgrading outweighs by a huge margin than keeping building on top. And yes, have seen Django shops locked into 0.9x release patched right into the core and running for a very long time, impossible to upgrade and all the horror stories. EDIT: Added Django

ecshafer 4y ago
At Shopify we also are basically always on the bleeding edge of all Rails and Ruby versions. By always moving forward it makes each improvement much smaller
- sjs 4y ago
  Plus Shopify and GitHub must drive most development of Rails so it’s basically an in-house dependency to some extent. I know that’s simplifying things.
wlll 4y ago
As a counterpoint I have yet to see one that is locked in my 15 years working with Rails.
- pmontra 4y ago
  I have customers with a mix of small Rails projects in version 6 or 7. I told one this week that we could use AR's encrypts :field to encrypt some fields in the database, but that project is Rails 6 and that feature is added by Rails 7. We could add the code to handle it because it's small, but customer decided not to encrypt the field and wait until (if!) the project moves to Rails 7.
  - wlll 4y ago
    I wouldn't say that's locked though. The place I currently work is on 6.1. We're not locked to it, we're going to upgrade, but in a few weeks or so. I can understand the reason to put off encrypting the field too. Upgrades, or backporting, are just work like any other project and I guess that other things for now take preference. After all, before field encryption came along the risk balance calculation had already been done (even if only implicitly) that storing the field unencrypted in the database was safe enough. The availability of encrypted fields doesn't change that, though making some things safer doesn't necessarily fall neatly into the "requirements" bucket despite being a good idea.

danjac 4y ago

I remember working on projects that did this in the early days of Django, particularly when there were badly needed features that took too long to land in the stable release. Nowadays most projects I've seen use the latest stable release or at least the latest LTS - maybe some legacy projects lagging behind.

tiffanyh 4y ago

> “It must also be a massive boon for the Rails ecosystem to have such a large property running off the head.” I would have expected Microsoft to focus on developer efforts into speeding up Ruby as a language given they are one of a small few large companies that have deep language/compiler expertise.

wlll 4y ago
Much as I'd like Ruby to be faster it's rarely the bottleneck in most Rails applications, besides, Ruby is becoming faster, it's a lot better than it used to be.
weatherlight 4y ago
It's usually I/O and dDB calls that are the bottle neck, not the speed of the language. Although, Ruby and be a bit of a memory hog,Ii did hear though, that this got better with the new YJIT in Ruby 3.x.
- Rapzid 4y ago
  > It's usually I/O and dDB calls that are the bottle neck Not in my experience.
  - weatherlight 4y ago
    In a Ruby on Rails app?
ksec 4y ago
Unfortunately Github is the only product that uses Ruby inside Microsoft. So their incentive is pretty small. I do wish someone inside Github could push M$ to spend some money and collaborate with Shopify on their JIT effort though.
- kovacs 4y ago
  That's actually not true. There's at least one other product at MS that uses ruby... Yammer. Although I'm not sure how much more incentive that would add for MS.

WirelessGigabit 4y ago

You MUST have an insane amount of test coverage to trust something like this to Ruby. I'm new to Ruby, with 13 yoe as a SW engineer. Personally, I find it a very hard language to master. Writing tests often feels like I'm settings variables left and right without seeing them being used in the current context. But that then happens to be part of the let() way in rspec. Now you might say: why use rspec? I inherited this codebase, so gotta do what you gotta do. I do really miss my compiler. I am not a fan of writing a block somewhere that can be invoked 2 weeks later for the first time and then fail, because someone passed in a number where a string was expected. I went from PHP to Java to C# to F# to JS to Rust. F# and Rust stand out in terms of hardest to write, but easiest to trust. I don't have that feeling with Ruby, and RoR. But, again, my personal opinion. Good friend of mine started with Ruby, and he loves it. He says that he accepts the magic things for what they are, and uses them. My brain doesn't allow that. I have to understand.

winrid 4y ago
Maybe some day Ruby will get type hinting. I'm loving it in Python. In terms of the "I wrote a block somewhere and it broke when executed for the first time two weeks later" - don't do that in a scripting language. Use the REPL to build/test the block. That's the trade off - instead of a compiler you get a fast REPL - use it! :)
- ok123456 4y ago
  it already does
  - willlma 4y ago
    Please say more. What's your setup?
    - teaearlgraycold 4y ago
      Sorbet exists, but it’s worse than what Python offers, which is dramatically worse than what TypeScript did to JS.

aledalgrande 4y ago

Shopify also runs off main, with a bigger codebase and more devs, and employs some Ruby core/Rails core devs too. It is the right choice IMO. Once you decide on your main framework as a big company, you should invest as much as you can in it and its community, it will pay off in the long term. https://shopify.engineering/shopify-monolith https://shopify.engineering/shopify-monolith