4 ms·

I'm not hugely familiar with the Linux release cycle, can anyone explain why such rudimentary support should be merged in now, instead of waiting until it's rea

I'm not hugely familiar with the Linux release cycle, can anyone explain why such rudimentary support should be merged in now, instead of waiting until it's ready?

bbojan 5y ago
Release early, release often?
gregkh 5y ago
Merge the portions that you know are correct and will have no affect on anyone else now, which makes future work easier as you do not have to keep those "working" commits up to date. We do this all the time with kernel development, and is one reason why breaking changes up into tiny pieces is so powerful. We can take the pieces that make sense now, and allow the developer to redo the portions that are not ready yet, instead of having to reject the whole thing if it were done in one single "chunk." Also note that the TTY/serial portions of this hardware support was already merged through the serial tree because they were independent and didn't affect anyone else.
- 600frogs 5y ago
  Interesting, thanks for the explanation. Has this style of development ever come back to bite you?
  - nousermane 5y ago
    Not GP, but... In the context of recent event, [0] where not reviewing thoroughly enough some tiny patches had a major come-back-to-bite fallout, I can't help but wonder: How, exactly are you expecting an increase in average patch size to help? [0] https://news.ycombinator.com/item?id=26887670 https://news.ycombinator.com/item?id=26887670
    - 600frogs 5y ago
      I did read through this debacle when it came out actually, I'm thoroughly on team Greg. I suppose my question was separate from malicious patches - I was interested in knowing if this incremental "merge tiny patches as and when they're ready" mode of development has ever caused issues with half-baked solutions affecting other parts of the kernel where perhaps it wouldn't have otherwise done so if the release was given more time for polishing and testing.
  - gregkh 5y ago
    Off the top of my head, no. The big "downside" is that it takes more work on the patch submitter side. But the benefits in the end are almost always more than worth it (easier reviewer time, easier time to track down problems, better development cycle as feedback can be more specific, easier evolution of changes, etc.) I wrote a whole chapter in the book "Beautiful Code" about how this development model can help create an end result that is almost always better than the initial "huge" submission model. Check it out if you are interested, it should be free online somewhere...
    - 600frogs 5y ago
      My instinct would have been that it's easier for the submitter (as they have less to polish and test) and more irksome for the reviewer as they have to go through multiple rounds of submissions, but naturally I'll take your word for it! This kind of discussion is always of interest to me, I'll check out the book, thank you.
      - Filligree 5y ago
        Reviewing three changelists, which individually do only a single thing each, is in my experience much easier than reviewing a single changelist bundling the changes from all three. This is true even if the same lines are changed multiple times. It's something you'll learn with experience, but it's also not even close. Break your patches up as much as possible, and everyone will be happier.
    - joseluisq 5y ago
      I found it, Greg's chapter `The Linux Kernel Driver Model: The Benefits of Working Together` on page 267 - https://www.oreilly.com/library/view/beautiful-code/9780596510046/ https://www.oreilly.com/library/view/beautiful-code/97805965... - https://github.com/stormtrooper96/books/blob/master/software-development/Beautiful%20Code.pdf https://github.com/stormtrooper96/books/blob/master/software... So I'll definitely give it a read. Thanks!
richardanaya 5y ago
One idea might be building a relationship of trust around a certain architecture in the Kernel, another idea is that it makes future work faster to merge in if the basics are already in.
soneil 5y ago
There's a relatively low cost to having this merged as it becomes available. It's not like x86 distros will be building it into their kernel.
regularfry 5y ago
It lets others build on it.
gsnedders 5y ago
Obviously you should listen to Greg and not me, but in many ways you can summarise it as "are the changes correct and do they work?". That they don't amount to end-user useful support is a very separate matter. What's the benefit of delaying submission of correct and working branches?
- aquadrop 5y ago
  If you plan for such model beforehand you already might be lowering overall efficiency. You know that you can't get the whole thing you want as one piece, but you plan "one third now, another third in a couple of months and the last third in more couple of months". Overall you might end up with twice the effort, since you had to account for that split, but in the context of kernel it still makes sense because of all the complexity and many people working on it simultaneously. And of course it also depends on "splitability" of the thing you're working on.
  - marcan_42 5y ago
    It's actually more efficient to do it this way. When you develop in a fork, you end up having to both keep rebasing on mainline, and then on submission, you might find that large parts of the code are the wrong approach or do not meet upstream standards, and need to be rewritten. By planning for incremental merges, you ensure that your foundation is solid and acceptable and avoid wasted work.
trissylegs 5y ago
It seems like it's ready to go in mailine. Linux doesn't really do "Big-bang" Everything is done releases of new features. Support is added incrementally. Here it says USB, PCIe, IOMMU, NVMe. Haven't been finished. But given those features are spread occross difference subsystems and have different maintainers it'll be easier to work on those once the platform is in Mainline. (You won't be able to expect all the PCIe developers to have the special M1 mac branch ready for testing)
- 600frogs 5y ago
  So it seems to be it's primarily solving branching issues rather than providing any benefit of the incremental support, is that correct?
  - viraptor 5y ago
    Depends what you mean by incremental support. It would definitely help someone to base new work on what's already approved, merged and supported without the need to hunt for the latest updated external branch and reconciling it with their local version. (But maybe that's what you're calling branching issues)
  - cestith 5y ago
    I'd rather spend time writing new branches against approved, merged upstream rather than submitting to an upstream branch that may never get reviewed and merged. Keep in mind, too, that the longer a branch lives the more rebasing it goes through. Also, the bigger the branch is, the more it is to review when finally considering whether to merge it. I think giving people a foundation on which to build is a good way to prevent a lot of extraneous work and also to build confidence in the direction of development.
Rhedox 5y ago
Easier to address feedback on the initial code now than when you have hundreds of lines building on top of it.
jononor 5y ago
This is an example of "integrate early, integrate often" which is a core part of Continuous Integration (CI). These days people tend to focus on the automation aspect of CI, but getting code into mainline early is also key.

I'm not hugely familiar with the Linux release cycle, can anyone explain why such rudimentary support should be merged in now, instead of waiting until it's ready?

bbojan 5y ago

Release early, release often?

gregkh 5y ago

Merge the portions that you know are correct and will have no affect on anyone else now, which makes future work easier as you do not have to keep those "working" commits up to date. We do this all the time with kernel development, and is one reason why breaking changes up into tiny pieces is so powerful. We can take the pieces that make sense now, and allow the developer to redo the portions that are not ready yet, instead of having to reject the whole thing if it were done in one single "chunk." Also note that the TTY/serial portions of this hardware support was already merged through the serial tree because they were independent and didn't affect anyone else.

600frogs 5y ago
Interesting, thanks for the explanation. Has this style of development ever come back to bite you?
- nousermane 5y ago
  Not GP, but... In the context of recent event, [0] where not reviewing thoroughly enough some tiny patches had a major come-back-to-bite fallout, I can't help but wonder: How, exactly are you expecting an increase in average patch size to help? [0] https://news.ycombinator.com/item?id=26887670 https://news.ycombinator.com/item?id=26887670
  - 600frogs 5y ago
    I did read through this debacle when it came out actually, I'm thoroughly on team Greg. I suppose my question was separate from malicious patches - I was interested in knowing if this incremental "merge tiny patches as and when they're ready" mode of development has ever caused issues with half-baked solutions affecting other parts of the kernel where perhaps it wouldn't have otherwise done so if the release was given more time for polishing and testing.
- gregkh 5y ago
  Off the top of my head, no. The big "downside" is that it takes more work on the patch submitter side. But the benefits in the end are almost always more than worth it (easier reviewer time, easier time to track down problems, better development cycle as feedback can be more specific, easier evolution of changes, etc.) I wrote a whole chapter in the book "Beautiful Code" about how this development model can help create an end result that is almost always better than the initial "huge" submission model. Check it out if you are interested, it should be free online somewhere...
  - 600frogs 5y ago
    My instinct would have been that it's easier for the submitter (as they have less to polish and test) and more irksome for the reviewer as they have to go through multiple rounds of submissions, but naturally I'll take your word for it! This kind of discussion is always of interest to me, I'll check out the book, thank you.
    - Filligree 5y ago
      Reviewing three changelists, which individually do only a single thing each, is in my experience much easier than reviewing a single changelist bundling the changes from all three. This is true even if the same lines are changed multiple times. It's something you'll learn with experience, but it's also not even close. Break your patches up as much as possible, and everyone will be happier.
  - joseluisq 5y ago
    I found it, Greg's chapter `The Linux Kernel Driver Model: The Benefits of Working Together` on page 267 - https://www.oreilly.com/library/view/beautiful-code/9780596510046/ https://www.oreilly.com/library/view/beautiful-code/97805965... - https://github.com/stormtrooper96/books/blob/master/software-development/Beautiful%20Code.pdf https://github.com/stormtrooper96/books/blob/master/software... So I'll definitely give it a read. Thanks!

richardanaya 5y ago

One idea might be building a relationship of trust around a certain architecture in the Kernel, another idea is that it makes future work faster to merge in if the basics are already in.

soneil 5y ago

There's a relatively low cost to having this merged as it becomes available. It's not like x86 distros will be building it into their kernel.

regularfry 5y ago

It lets others build on it.

gsnedders 5y ago

Obviously you should listen to Greg and not me, but in many ways you can summarise it as "are the changes correct and do they work?". That they don't amount to end-user useful support is a very separate matter. What's the benefit of delaying submission of correct and working branches?

aquadrop 5y ago
If you plan for such model beforehand you already might be lowering overall efficiency. You know that you can't get the whole thing you want as one piece, but you plan "one third now, another third in a couple of months and the last third in more couple of months". Overall you might end up with twice the effort, since you had to account for that split, but in the context of kernel it still makes sense because of all the complexity and many people working on it simultaneously. And of course it also depends on "splitability" of the thing you're working on.
- marcan_42 5y ago
  It's actually more efficient to do it this way. When you develop in a fork, you end up having to both keep rebasing on mainline, and then on submission, you might find that large parts of the code are the wrong approach or do not meet upstream standards, and need to be rewritten. By planning for incremental merges, you ensure that your foundation is solid and acceptable and avoid wasted work.

trissylegs 5y ago

It seems like it's ready to go in mailine. Linux doesn't really do "Big-bang" Everything is done releases of new features. Support is added incrementally. Here it says USB, PCIe, IOMMU, NVMe. Haven't been finished. But given those features are spread occross difference subsystems and have different maintainers it'll be easier to work on those once the platform is in Mainline. (You won't be able to expect all the PCIe developers to have the special M1 mac branch ready for testing)

600frogs 5y ago
So it seems to be it's primarily solving branching issues rather than providing any benefit of the incremental support, is that correct?
- viraptor 5y ago
  Depends what you mean by incremental support. It would definitely help someone to base new work on what's already approved, merged and supported without the need to hunt for the latest updated external branch and reconciling it with their local version. (But maybe that's what you're calling branching issues)
- cestith 5y ago
  I'd rather spend time writing new branches against approved, merged upstream rather than submitting to an upstream branch that may never get reviewed and merged. Keep in mind, too, that the longer a branch lives the more rebasing it goes through. Also, the bigger the branch is, the more it is to review when finally considering whether to merge it. I think giving people a foundation on which to build is a good way to prevent a lot of extraneous work and also to build confidence in the direction of development.

Rhedox 5y ago

Easier to address feedback on the initial code now than when you have hundreds of lines building on top of it.

jononor 5y ago

This is an example of "integrate early, integrate often" which is a core part of Continuous Integration (CI). These days people tend to focus on the automation aspect of CI, but getting code into mainline early is also key.