7 ms·

If all the world were a monorepo

277 points by sebg 1y ago

esafak 1y ago
> In what other ecosystem would a top package introduce itself using an eight-variable equation? That's the objective function of Hastie et al's GLM. I had a good chuckle when I realized the author's last name is Tibshirani. If you know you know.
- bryanrasmussen 1y ago
  Robert Tibshirani has a daughter named Julie.
- david_draco 1y ago
  And if I don't know, can I know?
  - esafak 1y ago
    Hastie and Tibshirani wrote a famous book on ML (https://hastie.su.domains/ElemStatLearn/ https://hastie.su.domains/ElemStatLearn/), and extended GLMs into GAMs: https://en.wikipedia.org/wiki/Generalized_additive_model https://en.wikipedia.org/wiki/Generalized_additive_model
- jordigh 1y ago
  Peeved that it isn't an equation. There's no equals signs!
  - croemer 1y ago
    Objective function or expression might have been more precise, indeed.
  - jubileet 1y ago
    I'm the author. Thanks for the copy-edit, I should have indeed said "expression".
kazinator 1y ago
> But… CRAN had also rerun the tests for all packages that depend on mine, even if they don’t belong to me! When you propose a change to something that other things depend on, it makes sense to test those dependents for a regression; this is not earth shattering. If you want to change something which breaks them, you have to then do it in a different way. First provide a new way of doing something. Then get all the dependencies that use the old way to migrate to the new way. Then when the dependents are no longer relying on the old way, you can push out a change which removes it.
- constantcrying 1y ago
  >If you want to change something which breaks them, you have to then do it in a different way. Almost every other package repo works differently and publishing packages which would break other packages is more than common, it is the standards way to publish updates. >Then get all the dependencies that use the old way to migrate to the new way. Then when the dependents are no longer relying on the old way, you can push out a change which removes it. This is actually how no software repo works as it is actually insane.
  - kazinator 1y ago
    Nope! - Breaking things obviously insane rather than not breaking things. - Staged obsolescence is sane compared to breaking things, and compared to installations carrying multiple versions of the same package. There being single version of every package in any given installation, and every package carefully managed for backwards compatibility, removing features only when everything has migrated off them, is utterly sane. There may be exceptions. (The idea that there are never exceptions is insane). Suppose that it is discovered that there is no secure way of using some API, and there is a real threat: it has to be removed. I think that a good way forward would be to identify the packages which use the API, and contact all the maintainers for an emergency conference. The package system should also be capable of applying its own patches to packages; if something must break for backwards compatibility, the package system should provide patches to fix the broken packages until the upstreams develops their own fixes.
maxbond 1y ago
> When declaring dependencies, most packages don’t specify any version requirements, and if they do, it’s usually just a lower bound like ‘grf >= 1.0’. I like the perspective presented in this article, I think CRAN is taking an interesting approach. But this is nuts and bolts. Explicitly saying you're compatible with any future breaking changes!? You can't possibly know that! I get that a lot of R programmers might be data scientists first and programmers second, so many of them probably don't know semver, but I feel like the language should guide them to a safe choice here. If CRAN is going to email you about reverse dependencies, maybe publishing a package with a crazy semver expression should also trigger an email.
- remus 1y ago
  > Explicitly saying you're compatible with any future breaking changes!? You can't possibly know that! I kind of like it in a way. In a lot of eco systems it's easy for package publishers to be a bit lazy with compatibility which can push a huge amount of work on package consumers. R seems similar to go in this regard, where there is a big focus on not breaking compatibility which then means they are conservative about adding new stuff until they're happy to support it for a long time.
  - maxbond 1y ago
    I guess it wouldn't bother me if it weren't a semver expression. As a semver expression it's ridiculous on it's face, a breaking release will break your code until proven otherwise. "foo >= 2024R1", well, I'm not entirely comfortable with it but if you've got a comprehensive plan to address the potential dangers (as CRAN appears to), godspeed.
derefr 1y ago
CRAN’s approach here sounds like it has all the disadvantages of a monorepo without any of the advantages. In a true monorepo — the one for the FreeBSD base system, say — if you make a PR that updates some low-level code, then the expectation is that you 1. compile the tree and run all the tests (so far so good), 2. update the high-level code so the tests pass (hmm), and 3. include those updates in your PR. In a true centralized monorepo, a single atomic commit can affect vertical-slice change through a dependency and all of its transitive dependents. I don’t know what the equivalent would be in distributed “meta-monorepo” development ala CRAN, but it’s not what they’re currently doing. (One hypothetical approach I could imagine, is that a dependency major-version release of a package can ship with AST-rewriting-algorithm code migrations, which automatically push both “dependency-computed” PRs to the dependents’ repos, while also pushing those same patches as temporary forced overlays onto releases of dependent packages until such time as the related PRs get merged. So your dependents’ tests still have to pass before you can release your package — but you can iteratively update things on your end until those tests do pass, and then trigger a simultaneous release of your package and your dependent packages. It’s then in your dependents’ court to modify + merge your PR to undo the forced overlay, asynchronously, as they wish.)
- joek1301 1y ago
  > One hypothetical approach I could imagine, is that a dependency major-version release of a package can ship with AST-rewriting-algorithm code migrations Jane Street has something similar called a "tree smash" [1]. When someone makes a breaking change to their internal dialect of OCaml, they also push a commit updating the entire company monorepo. It's not explicitly stated whether such migrations happen via AST rewrites, but one can imagine leveraging the existing compiler infrastructure to do that. [1]: https://signalsandthreads.com/future-of-programming/#3535 https://signalsandthreads.com/future-of-programming/#3535
  - swiftcoder 1y ago
    This is more of less how Facebook developed PHP -> Hack on the fly. Each new language feature would be patched in, and at the same time, a whole-monorepo transform would be run to adopt the feature. Pretty neat, if a logistical nightmare
jiggawatts 1y ago
This (with some tweaks) is what I envision the future of NPM, Cargo, and NuGet should look like. Automated tests, compilation by the package publisher, and enforcement of portability flags and SemVer semantics.
haberman 1y ago
This was an interesting article, but it made me even more interested in the author's larger take on R as a language: > In the years since, my discomfort has given away to fascination. I’ve come to respect R’s bold choices, its clarity of focus, and the R community’s continued confidence to ‘do their own thing’. I would love to see a follow-up article about the key insights that the author took away from diving more deeply into R.
ants_everywhere 1y ago
I genuinely enjoy R. I use it for calculations daily. In comparison using Python feels tedious and clunky even though I know it better. > CRAN had also rerun the tests for all packages that depend on mine, even if they don’t belong to me! Another way to frame this is these are the customers of your package's API. If you broke them you are required to ship a fix. I see why this isn't the default (e.g. on GitHub you have no idea how many people depend on you). But the developer experience is much nicer like this. Google, for example, makes this promise with some of their public tools. Outside the word of professional software developers, R is used by many academics in statistics, economics, social sciences etc. This rule makes it less likely that their research breaks because of some obscure dependency they don't understand.
- mhh__ 1y ago
  I've never written a line of R but it seems slightly underrated from what I've seen. Maybe there are some massive footguns I'm not aware of but python is mostly oriented around variables rather than pipelines so it never seems to flow as well as R
0xbadcafebee 1y ago
The author is a little confused. A system that blocks releases on defects and doesn't pin versions is continuous integration, not a monorepo. The two are not synonymous. Monorepos often use continuous integration to ensure their integrity, but you can use continuous integration without a monorepo, and monorepos can be used without continuous integration. > But the migration had a steep cost: over 6 years later, there are thousands of projects still stuck on an older version. This is a feature, not a bug. The pinning of versions allows systems to independently maintain their own dependency trees. This is how your Linux distribution actually remains stable (or used to, before the onslaught of "rolling release" distributions, and the infection of the "automatically updating application" into product development culture, which constantly leaves me with non-functional Mobile applications whereupon I am forced to update them once a week). You set the versions, and nothing changes, so you can keep using the same software, and it doesn't break. Until you choose to upgrade it and deal with all the breaking shit. Every decision in life is a tradeoff. Do you go with no version numbers at all, always updating, always fixing things? Or do you always require version numbers, keeping things stable, but having difficulty updating because of a lack of compatible versions? Or do you find some middle ground? There are pros and cons to all these decisions. There is no one best way, only different ways.
- summis 1y ago
  For me the comparison to monorepo made a lot sense. One of the main features of monorepo is maintaining a DAG of dependencies and use that to decide which tests to run given a code change. CRAN package publishing seems to follow same idea.
  - procaryote 1y ago
    That's something you can do just as well with multiple repos though What a monorepo gives you on top of that is that you can change the dependents in the same PR
  - malkia 1y ago
    for me too - in a way a "virtual" monorepo - as if all these packages belong in some ideal monorepo, even though they don't.
cortesoft 1y ago
I feel like if more package repositories did this, you would end up just finding more and more workarounds and alternative distribution methods. I mean, just look at how many projects use “curl and bash” as their distribution method even though the project repositories they could use instead don’t even require anything nearly as onerous as the reverse dependency checks described in this article. If the minimal requirements the current repos have are enough to push projects to alternate distribution, I can’t imagine what would happen if it was added.
pabs3 1y ago
Debian is kind of like that, except packages broken by upgrades are mostly just removed.
- dima55 1y ago
  Eventually, yes I guess. But long before that the breaker and breakee both are notified, and the breakage hopefully is fixed. As it should be. I would hope the other aspirational software distribution systems (pip, npm, et al) ALSO do that, but according to this article, I guess they don't? Not shocked , to be honest
  - askonomm 1y ago
    But how would that work? Say I have software written that runs just fine, but has not been updated to the latest runtime of Python or Node (as per your example). Perhaps a dependency I use has a broken recent version, but the old version I use works fine. You remove the package, now it breaks my software. This would effectively make it so that all libraries / dependencies that are "abandoned" by the author or inactive, would be deleted, which then results in all the software that used them to also break. Unless I misunderstood something?
- malkia 1y ago
  I've been using it for so many years, and now it makes complete sense now that you mentioned that! THanks!
didip 1y ago
Damn. Well, time to fork everything and keep internal patches internal. This system is unworkable.
- gorset 1y ago
  That’s the default in the monorepos I’ve worked on. When a third party dep is broken or needs a workaround, just include a patch in the build (or fork). Then those patches can be upstreamed asynchronously without slowing down development.
quelsolaar 1y ago
This is awesome. I run a team that uses software I produce and i have a rule that i can’t deliver breaking changes, and i cant force migrations. I can do the migration myself, or i have to emulate the old behavior next to the new. It makes you think really hard about releasing new APIs. I wish this was standard practice.
- zelphirkalt 1y ago
  Sounds like an easy workaround would be versioned APIs then. Missing that, it sounds like the API will forever be stuck, or add-only, creating a mess. That is, if it is not already very stable.
croemer 1y ago
Might be useful to add "R" somewhere to the title to make it clearer what this article is about.
noname123 1y ago
To be honest, I don't know what is worse. Installing a R library that require re-installing a bunch of updates, and being stuck in R installation hell or exerpiencing conda install that is stuck in "Resolving Dependencies" hell. The only thing I've learned to mitigate both is just containerize everything.
croemer 1y ago
One workaround that isn't mentioned is that one could just release a new package entirely for each blocked release. grf1, grf2, grf3... The downside is that dependees have to manually change their dependency and you get proliferation of packages with informal relationships.
jurschreuder 1y ago
[flagged]
- tomhow 1y ago
  Please don't do this here.
  - jurschreuder 1y ago
    I know I know, I would never make it in a corporate environment I would get an appointment with HR the first day to explain my "funny" jokes XD
constantcrying 1y ago
What people have to understand is that this is not a repository for software developers. It is a repository for people needing tools for statistical analysis and scientific computing. The way a software developer thinks about a package is totally different to the way someone trying to perform statistical analysis thinks about packages. This is the same for CTAN, the name is no coincidence. The packages are for users and not developers.
- 3eb7988a1663 1y ago
  As an admitted R hater, this is exceptionally good framing. Reproducible science has only ever had limited buy-in, but R was extra late to the party with packrat and renv. Enforcing code consistency at this level has probably done wonders to reducing some amount of churn.
nurettin 1y ago
I just put my own library dependencies into submodules. They act like local copies, so you can develop them while developing the main repo. That essentially makes the high level project a monorepo while giving you the option to work on the submodule on its own.
__MatrixMan__ 1y ago
I think there's tooling for doing something like this reverse dependency trick in nixpkgs. I made a change to pre-commit and somebody more in-the-know than I stopped by the PR and pointed out the two python packages that my changes broke. Zero wouldn't have been surprising to me, nor would several hundred, but two... what a conveniently actionable number. It has me wanting to give names to some of my hacks and publish them as packages so that people are more more aware when their changes are breaking changes. On the other hand, if I do something weird, I don't necessity want to burden others with maintaining it. Tradeoffs...
mycall 1y ago
> Taking advantage of the major version bump, we had snuck in a small API change. This change then caused a test failure in policytree Wait a second. Another package failed your MAJOR version upgrade because you changed your API? Not following semver is crazy for any package manager to enforce.
catchcatchcatch 1y ago
[dead]
tanepiper 1y ago
My wife is having to learn R as part of her Masters - she's not got a technical background and my impression is it's throwing people into programming at the deep end, and likely after the course she will never actually use it. Meanwhile I read the material and t absolutely feels like a cult - "R is fun" is like something they say to persuade themselves they are not in a cult
alexpotato 1y ago
I recently started using python packages for some statistical work. I then discovered that there are often bugs with many of the python stats packages. Many python numerical packages also have the reputation of changing how things work "under the hood" from version to version. This means that you can't trust your output after a version change. Given all of the above, I can see why "serious" data scientists stick with R and this article is just another reason why.

277 points by sebg 1y ago

> In what other ecosystem would a top package introduce itself using an eight-variable equation? That's the objective function of Hastie et al's GLM. I had a good chuckle when I realized the author's last name is Tibshirani. If you know you know.

bryanrasmussen 1y ago
Robert Tibshirani has a daughter named Julie.
david_draco 1y ago
And if I don't know, can I know?
- esafak 1y ago
  Hastie and Tibshirani wrote a famous book on ML (https://hastie.su.domains/ElemStatLearn/ https://hastie.su.domains/ElemStatLearn/), and extended GLMs into GAMs: https://en.wikipedia.org/wiki/Generalized_additive_model https://en.wikipedia.org/wiki/Generalized_additive_model
jordigh 1y ago
Peeved that it isn't an equation. There's no equals signs!
- croemer 1y ago
  Objective function or expression might have been more precise, indeed.
- jubileet 1y ago
  I'm the author. Thanks for the copy-edit, I should have indeed said "expression".

kazinator 1y ago

> But… CRAN had also rerun the tests for all packages that depend on mine, even if they don’t belong to me! When you propose a change to something that other things depend on, it makes sense to test those dependents for a regression; this is not earth shattering. If you want to change something which breaks them, you have to then do it in a different way. First provide a new way of doing something. Then get all the dependencies that use the old way to migrate to the new way. Then when the dependents are no longer relying on the old way, you can push out a change which removes it.

constantcrying 1y ago
>If you want to change something which breaks them, you have to then do it in a different way. Almost every other package repo works differently and publishing packages which would break other packages is more than common, it is the standards way to publish updates. >Then get all the dependencies that use the old way to migrate to the new way. Then when the dependents are no longer relying on the old way, you can push out a change which removes it. This is actually how no software repo works as it is actually insane.
- kazinator 1y ago
  Nope! - Breaking things obviously insane rather than not breaking things. - Staged obsolescence is sane compared to breaking things, and compared to installations carrying multiple versions of the same package. There being single version of every package in any given installation, and every package carefully managed for backwards compatibility, removing features only when everything has migrated off them, is utterly sane. There may be exceptions. (The idea that there are never exceptions is insane). Suppose that it is discovered that there is no secure way of using some API, and there is a real threat: it has to be removed. I think that a good way forward would be to identify the packages which use the API, and contact all the maintainers for an emergency conference. The package system should also be capable of applying its own patches to packages; if something must break for backwards compatibility, the package system should provide patches to fix the broken packages until the upstreams develops their own fixes.

maxbond 1y ago

> When declaring dependencies, most packages don’t specify any version requirements, and if they do, it’s usually just a lower bound like ‘grf >= 1.0’. I like the perspective presented in this article, I think CRAN is taking an interesting approach. But this is nuts and bolts. Explicitly saying you're compatible with any future breaking changes!? You can't possibly know that! I get that a lot of R programmers might be data scientists first and programmers second, so many of them probably don't know semver, but I feel like the language should guide them to a safe choice here. If CRAN is going to email you about reverse dependencies, maybe publishing a package with a crazy semver expression should also trigger an email.

remus 1y ago
> Explicitly saying you're compatible with any future breaking changes!? You can't possibly know that! I kind of like it in a way. In a lot of eco systems it's easy for package publishers to be a bit lazy with compatibility which can push a huge amount of work on package consumers. R seems similar to go in this regard, where there is a big focus on not breaking compatibility which then means they are conservative about adding new stuff until they're happy to support it for a long time.
- maxbond 1y ago
  I guess it wouldn't bother me if it weren't a semver expression. As a semver expression it's ridiculous on it's face, a breaking release will break your code until proven otherwise. "foo >= 2024R1", well, I'm not entirely comfortable with it but if you've got a comprehensive plan to address the potential dangers (as CRAN appears to), godspeed.

derefr 1y ago

CRAN’s approach here sounds like it has all the disadvantages of a monorepo without any of the advantages. In a true monorepo — the one for the FreeBSD base system, say — if you make a PR that updates some low-level code, then the expectation is that you 1. compile the tree and run all the tests (so far so good), 2. update the high-level code so the tests pass (hmm), and 3. include those updates in your PR. In a true centralized monorepo, a single atomic commit can affect vertical-slice change through a dependency and all of its transitive dependents. I don’t know what the equivalent would be in distributed “meta-monorepo” development ala CRAN, but it’s not what they’re currently doing. (One hypothetical approach I could imagine, is that a dependency major-version release of a package can ship with AST-rewriting-algorithm code migrations, which automatically push both “dependency-computed” PRs to the dependents’ repos, while also pushing those same patches as temporary forced overlays onto releases of dependent packages until such time as the related PRs get merged. So your dependents’ tests still have to pass before you can release your package — but you can iteratively update things on your end until those tests do pass, and then trigger a simultaneous release of your package and your dependent packages. It’s then in your dependents’ court to modify + merge your PR to undo the forced overlay, asynchronously, as they wish.)

joek1301 1y ago
> One hypothetical approach I could imagine, is that a dependency major-version release of a package can ship with AST-rewriting-algorithm code migrations Jane Street has something similar called a "tree smash" [1]. When someone makes a breaking change to their internal dialect of OCaml, they also push a commit updating the entire company monorepo. It's not explicitly stated whether such migrations happen via AST rewrites, but one can imagine leveraging the existing compiler infrastructure to do that. [1]: https://signalsandthreads.com/future-of-programming/#3535 https://signalsandthreads.com/future-of-programming/#3535
- swiftcoder 1y ago
  This is more of less how Facebook developed PHP -> Hack on the fly. Each new language feature would be patched in, and at the same time, a whole-monorepo transform would be run to adopt the feature. Pretty neat, if a logistical nightmare

jiggawatts 1y ago

This (with some tweaks) is what I envision the future of NPM, Cargo, and NuGet should look like. Automated tests, compilation by the package publisher, and enforcement of portability flags and SemVer semantics.

haberman 1y ago

This was an interesting article, but it made me even more interested in the author's larger take on R as a language: > In the years since, my discomfort has given away to fascination. I’ve come to respect R’s bold choices, its clarity of focus, and the R community’s continued confidence to ‘do their own thing’. I would love to see a follow-up article about the key insights that the author took away from diving more deeply into R.

ants_everywhere 1y ago

I genuinely enjoy R. I use it for calculations daily. In comparison using Python feels tedious and clunky even though I know it better. > CRAN had also rerun the tests for all packages that depend on mine, even if they don’t belong to me! Another way to frame this is these are the customers of your package's API. If you broke them you are required to ship a fix. I see why this isn't the default (e.g. on GitHub you have no idea how many people depend on you). But the developer experience is much nicer like this. Google, for example, makes this promise with some of their public tools. Outside the word of professional software developers, R is used by many academics in statistics, economics, social sciences etc. This rule makes it less likely that their research breaks because of some obscure dependency they don't understand.

mhh__ 1y ago
I've never written a line of R but it seems slightly underrated from what I've seen. Maybe there are some massive footguns I'm not aware of but python is mostly oriented around variables rather than pipelines so it never seems to flow as well as R

0xbadcafebee 1y ago

The author is a little confused. A system that blocks releases on defects and doesn't pin versions is continuous integration, not a monorepo. The two are not synonymous. Monorepos often use continuous integration to ensure their integrity, but you can use continuous integration without a monorepo, and monorepos can be used without continuous integration. > But the migration had a steep cost: over 6 years later, there are thousands of projects still stuck on an older version. This is a feature, not a bug. The pinning of versions allows systems to independently maintain their own dependency trees. This is how your Linux distribution actually remains stable (or used to, before the onslaught of "rolling release" distributions, and the infection of the "automatically updating application" into product development culture, which constantly leaves me with non-functional Mobile applications whereupon I am forced to update them once a week). You set the versions, and nothing changes, so you can keep using the same software, and it doesn't break. Until you choose to upgrade it and deal with all the breaking shit. Every decision in life is a tradeoff. Do you go with no version numbers at all, always updating, always fixing things? Or do you always require version numbers, keeping things stable, but having difficulty updating because of a lack of compatible versions? Or do you find some middle ground? There are pros and cons to all these decisions. There is no one best way, only different ways.

summis 1y ago
For me the comparison to monorepo made a lot sense. One of the main features of monorepo is maintaining a DAG of dependencies and use that to decide which tests to run given a code change. CRAN package publishing seems to follow same idea.
- procaryote 1y ago
  That's something you can do just as well with multiple repos though What a monorepo gives you on top of that is that you can change the dependents in the same PR
- malkia 1y ago
  for me too - in a way a "virtual" monorepo - as if all these packages belong in some ideal monorepo, even though they don't.

cortesoft 1y ago

I feel like if more package repositories did this, you would end up just finding more and more workarounds and alternative distribution methods. I mean, just look at how many projects use “curl and bash” as their distribution method even though the project repositories they could use instead don’t even require anything nearly as onerous as the reverse dependency checks described in this article. If the minimal requirements the current repos have are enough to push projects to alternate distribution, I can’t imagine what would happen if it was added.

pabs3 1y ago

Debian is kind of like that, except packages broken by upgrades are mostly just removed.

dima55 1y ago
Eventually, yes I guess. But long before that the breaker and breakee both are notified, and the breakage hopefully is fixed. As it should be. I would hope the other aspirational software distribution systems (pip, npm, et al) ALSO do that, but according to this article, I guess they don't? Not shocked , to be honest
- askonomm 1y ago
  But how would that work? Say I have software written that runs just fine, but has not been updated to the latest runtime of Python or Node (as per your example). Perhaps a dependency I use has a broken recent version, but the old version I use works fine. You remove the package, now it breaks my software. This would effectively make it so that all libraries / dependencies that are "abandoned" by the author or inactive, would be deleted, which then results in all the software that used them to also break. Unless I misunderstood something?
malkia 1y ago
I've been using it for so many years, and now it makes complete sense now that you mentioned that! THanks!

didip 1y ago

Damn. Well, time to fork everything and keep internal patches internal. This system is unworkable.

gorset 1y ago
That’s the default in the monorepos I’ve worked on. When a third party dep is broken or needs a workaround, just include a patch in the build (or fork). Then those patches can be upstreamed asynchronously without slowing down development.

quelsolaar 1y ago

This is awesome. I run a team that uses software I produce and i have a rule that i can’t deliver breaking changes, and i cant force migrations. I can do the migration myself, or i have to emulate the old behavior next to the new. It makes you think really hard about releasing new APIs. I wish this was standard practice.

zelphirkalt 1y ago
Sounds like an easy workaround would be versioned APIs then. Missing that, it sounds like the API will forever be stuck, or add-only, creating a mess. That is, if it is not already very stable.

croemer 1y ago

Might be useful to add "R" somewhere to the title to make it clearer what this article is about.

noname123 1y ago

To be honest, I don't know what is worse. Installing a R library that require re-installing a bunch of updates, and being stuck in R installation hell or exerpiencing conda install that is stuck in "Resolving Dependencies" hell. The only thing I've learned to mitigate both is just containerize everything.

croemer 1y ago

One workaround that isn't mentioned is that one could just release a new package entirely for each blocked release. grf1, grf2, grf3... The downside is that dependees have to manually change their dependency and you get proliferation of packages with informal relationships.

jurschreuder 1y ago

[flagged]

tomhow 1y ago
Please don't do this here.
- jurschreuder 1y ago
  I know I know, I would never make it in a corporate environment I would get an appointment with HR the first day to explain my "funny" jokes XD

constantcrying 1y ago

What people have to understand is that this is not a repository for software developers. It is a repository for people needing tools for statistical analysis and scientific computing. The way a software developer thinks about a package is totally different to the way someone trying to perform statistical analysis thinks about packages. This is the same for CTAN, the name is no coincidence. The packages are for users and not developers.

3eb7988a1663 1y ago
As an admitted R hater, this is exceptionally good framing. Reproducible science has only ever had limited buy-in, but R was extra late to the party with packrat and renv. Enforcing code consistency at this level has probably done wonders to reducing some amount of churn.

nurettin 1y ago

I just put my own library dependencies into submodules. They act like local copies, so you can develop them while developing the main repo. That essentially makes the high level project a monorepo while giving you the option to work on the submodule on its own.

__MatrixMan__ 1y ago

I think there's tooling for doing something like this reverse dependency trick in nixpkgs. I made a change to pre-commit and somebody more in-the-know than I stopped by the PR and pointed out the two python packages that my changes broke. Zero wouldn't have been surprising to me, nor would several hundred, but two... what a conveniently actionable number. It has me wanting to give names to some of my hacks and publish them as packages so that people are more more aware when their changes are breaking changes. On the other hand, if I do something weird, I don't necessity want to burden others with maintaining it. Tradeoffs...

mycall 1y ago

> Taking advantage of the major version bump, we had snuck in a small API change. This change then caused a test failure in policytree Wait a second. Another package failed your MAJOR version upgrade because you changed your API? Not following semver is crazy for any package manager to enforce.

catchcatchcatch 1y ago

[dead]

tanepiper 1y ago

My wife is having to learn R as part of her Masters - she's not got a technical background and my impression is it's throwing people into programming at the deep end, and likely after the course she will never actually use it. Meanwhile I read the material and t absolutely feels like a cult - "R is fun" is like something they say to persuade themselves they are not in a cult

alexpotato 1y ago

I recently started using python packages for some statistical work. I then discovered that there are often bugs with many of the python stats packages. Many python numerical packages also have the reputation of changing how things work "under the hood" from version to version. This means that you can't trust your output after a version change. Given all of the above, I can see why "serious" data scientists stick with R and this article is just another reason why.