4 ms·

I get where he’s coming from. I’m guessing everyone here has had a script they wrote for five people balloon into a multi-team project. People wonder why you, t

I get where he’s coming from. I’m guessing everyone here has had a script they wrote for five people balloon into a multi-team project. People wonder why you, the original author, were so lazy with your architecture even though you thought it wasn’t going to survive more than a week. The problem is that people aren’t willing to start over and rewrite software so you end up trying to bolt on features to an architecture that can’t handle them and the complexity becomes necessary for the software to function. C++ is the poster child for this, where the syntax for pure virtual functions is asinine but necessary because there was no other good way to do it at the time. If they had just rewritten C++ (and called it, I dunno, D for example) then they would avoid the complexity entirely. I respect Ryan but he comes across as someone ignorant of the realities of real-world SWE.

sseagull 5y ago
> The problem is that people aren’t willing to start over and rewrite software If I had to continuously re-write projects in new languages, I would never get anything done. There are a lot of projects that we all depend on that are pretty much one-person projects, and not everyone has time to learn a new language + ecosystem, then re-write, introduce new bugs, fix those, etc. Then of course, the rewrite doesn't always have exactly the same interface (API or user interface). So now you foist your new preference on everyone else. Many of them will stay on the old version, thus fragmenting everything even further. Look how long python 2 to 3 took (and is still not 100% complete). I come from scientific computing, and I have a rant bubbling up about how modern software development is being pushed into science, and probably slowing it down as a result. I should make a blog post about it, but I guess I would need to make a blog first :)
- sitkack 5y ago
  > I should make a blog post about it, but I guess I would need to make a blog first :) Jonathan, you can use a google doc, or a gist, or bug in a github project to get your message out. I recommend you should. I am a failed scientist that programmer in tech. I have helped grad students train up on unit testing, version control, api design, etc. I think modern software development needs to be brought into science so that science is reproducible. Without reproduction, we have nuthing. So I disagree, but I would love to hear what you have to say. Probably that our definitions of modern don't match and that you are drowing in sprint planning and story points, 1 beaker to 3 beakers.
  - sseagull 5y ago
    Short version of rant, unrefined and unpolished :) I basically agree about bringing modern software development into science, and I teach grad students and postdocs all over the country as well. But I'm looking for a better way. A major problem is constant churn. We used to teach Travis CI. That seems to have fallen out of favor, so now we teach GHA. But what about all those projects still using Travis? What happens if/when Github or GHA fall out of favor? Same with python packaging. setup.py vs. setup.cfg, pypi vs. conda vs. poetry vs. whatever else. Various ways of specifying requirements. Plus docker on top for some people. How to build/host docs is yet another thing to figure out. GH Pages, netlify, or RTD? Depends on the language? Now lots of projects are both C++ and Python, and cross language documentation is hacky at best. And what about git? Lots of non-science software developers have issues understanding that, what chance do informally-trained scientists have? And now we are starting to see people wanting to use Julia and Rust. Or to program web apis and web pages (html/css/js). Each of these issues may not be too bad on its own, but they all add up. When you already know about all these things, you drastically underestimate how difficult they can be to understand for someone new. Tech companies and startups can keep up with this stuff. I'm not sure scientists' time should be spent doing this. [PS: Not sure why you think my name is Jonathan, unless I said that somewhere. If I did, I was lying :) ]
    - alex_smart 5y ago
      > Not sure why you think my name is Jonathan Pretty sure that is a reference to the book by Richard Bach.
    - sitkack 5y ago
      I agree with everything you wrote. Software development is a mess. To git specifically, git is hard for everyone, anyone that says otherwise is either lying or doesn't know what they don't know. I said as much at the software carpentry classes I helped with. The lead scientist trainer didn't really like me saying that, but it is the truth. Humans have a fixed decision budget, we can only handle so much complexity before our brain turns off. Do we want to spend that solving domain problems or on huge software stacks? I guess this is where the folks like us could do a better job to make simple, obvious ways to do things. There is a lot of value in standardization and a reduced amount of choice. Hobgoblins and all. I knew there was a lot of complexity in our tech stacks, trying to bootstrap smart people from basically nothing made it even more apparent. But I don't think the answer is to get scientists out of writing code. Code is the mirror to science as science is the mirror to our understanding. It rigorously and formally, when used correctly, closes the loop. For all the problems that Python notebooks have, I think they make an excellent way to convey ideas, but they need history and a way to make them reproducible. Apologies for the mistaken identity, I thought you were a different seagull.
- kmstout 5y ago
  > If I had to continuously re-write projects in new languages, I would never get anything done. Among Common Lisp's features is that, while the ecosystem has evolved, the standard has been stable since the '90s.
- exdsq 5y ago
  Please do, I'd love to read it
CoderPuppy 5y ago
I had an idea a little while ago: constantly rewriting code. It was inspired by the idea of doing something repeatedly allows you to improve. So constantly rewriting the same bit of code (with changes) would allow you to get better at writing that sort of code (or solving that sort of problem). Every time you'd end up doing it slightly differently and learning something. I also came up with a different idea along the same lines while writing this comment. Weakening it to rewriting code regularly (instead of constantly) and adding a (rough) time limit. A reasonable time limit could be anywhere from a day to a week. I think it'd work like this: you'd build a project for the first time (in a few days) and use it for things. Later when you find a problem with it, you'd return and rewrite the whole thing (again in just a few days). I think the combination of rewriting from scratch and a time limit would limit the complexity of each project. One thing I haven't fleshed out yet is how to limit the depth/complexity of dependency chains.

sseagull 5y ago

> The problem is that people aren’t willing to start over and rewrite software If I had to continuously re-write projects in new languages, I would never get anything done. There are a lot of projects that we all depend on that are pretty much one-person projects, and not everyone has time to learn a new language + ecosystem, then re-write, introduce new bugs, fix those, etc. Then of course, the rewrite doesn't always have exactly the same interface (API or user interface). So now you foist your new preference on everyone else. Many of them will stay on the old version, thus fragmenting everything even further. Look how long python 2 to 3 took (and is still not 100% complete). I come from scientific computing, and I have a rant bubbling up about how modern software development is being pushed into science, and probably slowing it down as a result. I should make a blog post about it, but I guess I would need to make a blog first :)

sitkack 5y ago
> I should make a blog post about it, but I guess I would need to make a blog first :) Jonathan, you can use a google doc, or a gist, or bug in a github project to get your message out. I recommend you should. I am a failed scientist that programmer in tech. I have helped grad students train up on unit testing, version control, api design, etc. I think modern software development needs to be brought into science so that science is reproducible. Without reproduction, we have nuthing. So I disagree, but I would love to hear what you have to say. Probably that our definitions of modern don't match and that you are drowing in sprint planning and story points, 1 beaker to 3 beakers.
- sseagull 5y ago
  Short version of rant, unrefined and unpolished :) I basically agree about bringing modern software development into science, and I teach grad students and postdocs all over the country as well. But I'm looking for a better way. A major problem is constant churn. We used to teach Travis CI. That seems to have fallen out of favor, so now we teach GHA. But what about all those projects still using Travis? What happens if/when Github or GHA fall out of favor? Same with python packaging. setup.py vs. setup.cfg, pypi vs. conda vs. poetry vs. whatever else. Various ways of specifying requirements. Plus docker on top for some people. How to build/host docs is yet another thing to figure out. GH Pages, netlify, or RTD? Depends on the language? Now lots of projects are both C++ and Python, and cross language documentation is hacky at best. And what about git? Lots of non-science software developers have issues understanding that, what chance do informally-trained scientists have? And now we are starting to see people wanting to use Julia and Rust. Or to program web apis and web pages (html/css/js). Each of these issues may not be too bad on its own, but they all add up. When you already know about all these things, you drastically underestimate how difficult they can be to understand for someone new. Tech companies and startups can keep up with this stuff. I'm not sure scientists' time should be spent doing this. [PS: Not sure why you think my name is Jonathan, unless I said that somewhere. If I did, I was lying :) ]
  - alex_smart 5y ago
    > Not sure why you think my name is Jonathan Pretty sure that is a reference to the book by Richard Bach.
  - sitkack 5y ago
    I agree with everything you wrote. Software development is a mess. To git specifically, git is hard for everyone, anyone that says otherwise is either lying or doesn't know what they don't know. I said as much at the software carpentry classes I helped with. The lead scientist trainer didn't really like me saying that, but it is the truth. Humans have a fixed decision budget, we can only handle so much complexity before our brain turns off. Do we want to spend that solving domain problems or on huge software stacks? I guess this is where the folks like us could do a better job to make simple, obvious ways to do things. There is a lot of value in standardization and a reduced amount of choice. Hobgoblins and all. I knew there was a lot of complexity in our tech stacks, trying to bootstrap smart people from basically nothing made it even more apparent. But I don't think the answer is to get scientists out of writing code. Code is the mirror to science as science is the mirror to our understanding. It rigorously and formally, when used correctly, closes the loop. For all the problems that Python notebooks have, I think they make an excellent way to convey ideas, but they need history and a way to make them reproducible. Apologies for the mistaken identity, I thought you were a different seagull.
kmstout 5y ago
> If I had to continuously re-write projects in new languages, I would never get anything done. Among Common Lisp's features is that, while the ecosystem has evolved, the standard has been stable since the '90s.
exdsq 5y ago
Please do, I'd love to read it

CoderPuppy 5y ago

I had an idea a little while ago: constantly rewriting code. It was inspired by the idea of doing something repeatedly allows you to improve. So constantly rewriting the same bit of code (with changes) would allow you to get better at writing that sort of code (or solving that sort of problem). Every time you'd end up doing it slightly differently and learning something. I also came up with a different idea along the same lines while writing this comment. Weakening it to rewriting code regularly (instead of constantly) and adding a (rough) time limit. A reasonable time limit could be anywhere from a day to a week. I think it'd work like this: you'd build a project for the first time (in a few days) and use it for things. Later when you find a problem with it, you'd return and rewrite the whole thing (again in just a few days). I think the combination of rewriting from scratch and a time limit would limit the complexity of each project. One thing I haven't fleshed out yet is how to limit the depth/complexity of dependency chains.