5 ms·

First of all, I hate this "Agile" nonsense. I've seen it kill companies. It's truly awful, because it gives legitimacy to the anti-intellectualism that has infe

First of all, I hate this "Agile" nonsense. I've seen it kill companies. It's truly awful, because it gives legitimacy to the anti-intellectualism that has infected this industry. It's that anti-intellectualism that, if you let it, will cause a rot of your mathematical and technical skills. Before you know it, you've spent five years reacting to Scrum tickets and haven't written any serious code, and your math has gone to the birds as well. It's insidious but dangerous, this culture of business-driven engineering mediocrity. I hope that it'll be the fakes and the brogrammers who get flushed out in the next crash. Who knows, though? Obviously I can't predict the future better than anyone else. To me, Python doesn't feel like a "scientific" language. Python's a great exploratory tool, and it's got some great libraries for feeling out a concept or exploring a type of model (e.g. off-the-shelf machine learning tools). That said, science values reproducibility and precision, which brings us around to functional programming and static typing... and suddenly we're at Haskell. (Of course, for a wide variety of purposes, Python is just fine, and may be a better choice because of its library ecosystem.) I do think that, as we use more machine learning, we're going to have a high demand for people who can apply rigor to the sorts of engineering that are currently done very quickly (resulting in "magic" algorithms that seem to work but that no one understands). I also agree that "deep learning" and machine learning in general are carrying some substance, even if 90% of what is being called "data science" is watered-down bullshit. I still don't feel like I know what a "scientific programmer" is, or should be. And I'd love to see the death of business-driven engineering and "Agile" and all the mediocrity of user stories and backlog grooming meetings, but there's nothing yet that has convinced me that it's imminent just yet. Sadly, I think it may be around for a while.

vskarine 12y ago
Curious, what are the software development processes that you do not consider nonsense?
- yuhong 12y ago
  I think it is the fake "Agile" processes that michaelochurch is referring to, not the real one.
- michaelochurch 12y ago
  I'm negative on "Agile" because it's an attempt to patch closed-allocation, business-driven development. If you have a high level of talent and you're solving interesting problems, you can do open allocation. If those aren't the case, and for some reason can't be, then you need to take different approaches entirely (but you should seriously question whether what you're working on is worth doing in the first place).
  - bkeroack 12y ago
    "Agile" in many shops means letting the passengers fly the plane (to borrow a phrase from one of your blog posts).
samdk 12y ago
I think you may be under-weighting the value of Python being a great exploratory tool a bit. For the (relatively small) amount of research I've done, the bulk of the time spent working with code was spent exploring large-ish sets of data to see what results came up. Python is a very nice language for doing this, and also has a great ecosystem of tools and libraries for doing it. I think it's also true that the benefits of FP and static typing are most pronounced on large codebases. If you're writing small specific bits of code you can mostly hold in your head all at once you don't gain nearly as much from a language like Haskell. Which is not to say I wouldn't like to see more people using languages like Haskell (my primary programming language is OCaml now), but I think the bad habits people in scientific fields have around programming are a lot more of a detriment than the specific language they're using.
dllthomas 12y ago
"Python's a great exploratory tool" Is it great at that in ways Haskell isn't? If so, is that something we could capture and host within Haskell meaningfully? I like doing exploratory work in Haskell, but if there's something I'm missing I'd like to capture that too where it's appropriate.

vskarine 12y ago

Curious, what are the software development processes that you do not consider nonsense?

yuhong 12y ago
I think it is the fake "Agile" processes that michaelochurch is referring to, not the real one.
michaelochurch 12y ago
I'm negative on "Agile" because it's an attempt to patch closed-allocation, business-driven development. If you have a high level of talent and you're solving interesting problems, you can do open allocation. If those aren't the case, and for some reason can't be, then you need to take different approaches entirely (but you should seriously question whether what you're working on is worth doing in the first place).
- bkeroack 12y ago
  "Agile" in many shops means letting the passengers fly the plane (to borrow a phrase from one of your blog posts).

samdk 12y ago

I think you may be under-weighting the value of Python being a great exploratory tool a bit. For the (relatively small) amount of research I've done, the bulk of the time spent working with code was spent exploring large-ish sets of data to see what results came up. Python is a very nice language for doing this, and also has a great ecosystem of tools and libraries for doing it. I think it's also true that the benefits of FP and static typing are most pronounced on large codebases. If you're writing small specific bits of code you can mostly hold in your head all at once you don't gain nearly as much from a language like Haskell. Which is not to say I wouldn't like to see more people using languages like Haskell (my primary programming language is OCaml now), but I think the bad habits people in scientific fields have around programming are a lot more of a detriment than the specific language they're using.

dllthomas 12y ago

"Python's a great exploratory tool" Is it great at that in ways Haskell isn't? If so, is that something we could capture and host within Haskell meaningfully? I like doing exploratory work in Haskell, but if there's something I'm missing I'd like to capture that too where it's appropriate.