10 ms·

I stopped reading the post once I got to the drawing of the waterfall diagram. There's not a single service I'm aware of that runs today on a waterfall process

I stopped reading the post once I got to the drawing of the waterfall diagram. There's not a single service I'm aware of that runs today on a waterfall process. Not sure how you can trust anything else in the post with that obvious of a strawman argument. I'm not going to spend the time trying.

sjg007 6y ago
Plenty of waterfalls out there.
- fizixer 6y ago
  This is a nitpick on "... not a single ..." claim by GP. GP is otherwise right. First figure in the article is a joke. And that makes the article a joke. Plenty of waterfalls out there. Even more non-waterfalls, aware of iterative, modern process, who will kick you out of the discussion if you try to sell them waterfall. ML processes need innovation over those of traditional s/w dev. This article is the worst place on the internet to look for that innovation.
  - sjg007 6y ago
    Plenty of waterfalls in ML pipelines as well.
    - fuzzfactor 6y ago
      The circular chart example given for Machine Learning is virtually identical to a cycle of continuous improvement in a service or manufacturing operation, except for one little detail, with Continuous Improvement you don't want the minimum viable product, you want the Maximum Viable Product. Seems like a machine learning outfit would really benefit from a leader who is smarter to begin with, a better learner, and already more skilled in the task that you want the machine to do.
talal7860 6y ago
I kind of did the same.
tachyonbeam 6y ago
I've seen such of the worst throwaway academic garbage code in ML. Maybe the problem is that they completely ignore and look down on software engineering to begin with.
- hinkley 6y ago
  The high concept people never appreciate the grief they're causing for the people who have to implement their vision. The stuff Frank Lloyd Wright didn't know about construction could fill an entire tour. Which I discovered by taking that tour. Boy, did that dude try to ignore physics.
- mhh__ 6y ago
  It's common across all academia i.e. no engineering textbook is complete without 20 pages of raw, uncommented, single-letter variable-ed MATLAB vomited before the index. I despise python, but MATLAB is positively satanic by comparison (from a software quality perspective at least)
carabiner 6y ago
I work at an old school industry with data scientists who use Python, sklearn etc. everyday. Waterfall is everything here. Software is not just at software companies.
pkaye 6y ago
The writer seems to be a recent college graduate. Maybe he doesn't have a lot of software development experience in industry.
- hinkley 6y ago
  Pick a discipline, and you will find a bunch of people who can tell you why the conventional wisdom doesn't apply to them. It's more common in the young, but also has a much better prognosis. In particular I've seen my share of code written by non-developers, and they all make the same classes of mistakes one makes when you have less than a couple thousand hours of focused development work under your belt. Well we can't do that because X, or we don't need that because Y. Oh, okay. Is that why I have to come tow you out of the proverbial mud every couple of months? Could you humor me and try a few of these things out? I think it'll help. What people fear most is automated systems that are constructed on wishful thinking, not healthy skepticism. A lot of our processes are about proving to ourselves what we think we already know. There is plenty of code under the AI umbrella for which very conventional unit tests could be written. If you can't use our practices for ML, you'd better drop what you're doing and start developing parity, instead of bullshitting yourselves and everyone else about your exceptionalism. Code that makes decisions for or about people needs more ceremony and sobriety around it, not less. If you can't do at least as well as a startup (which is a pretty low bar), then we are really in some deep doodoo. But not to worry. The good news is that when the machines take over and we have to hide underground, you'll be the first ones we eat, so you won't suffer long.
  - free_rms 6y ago
    I think the issue here is where you put the ceremony. If you're iterating on your approach to a model, it really is a drag to do a bunch of bullshit in order to run an experiment that's likely to fail and just be thrown out anyways. If you're shipping that model to production and affecting people's lives, it should really be held to a different standard than the "fuck around and see what happens" experiments. And this is before we even get into unexplainability of deep neural nets.
    - 6y ago
ponker 6y ago
Most high reliability software is still waterfall. It's usually not service-delivered software since people who need high reliability software need it to run locally.
dboreham 6y ago
Or ever did.
tonyarkles 6y ago
Not too add too much snark, but the first sentence can also be paraphrased as “ML expert claims ML is critically important” :) I do tend to agree that it’s made some dramatic improvements regarding utility over the last few years, but...
tomc1985 6y ago
What are you talking about? I followed that exact process this week in the process of writing and checking in code, as did nearly every other programmer.

sjg007 6y ago

Plenty of waterfalls out there.

fizixer 6y ago
This is a nitpick on "... not a single ..." claim by GP. GP is otherwise right. First figure in the article is a joke. And that makes the article a joke. Plenty of waterfalls out there. Even more non-waterfalls, aware of iterative, modern process, who will kick you out of the discussion if you try to sell them waterfall. ML processes need innovation over those of traditional s/w dev. This article is the worst place on the internet to look for that innovation.
- sjg007 6y ago
  Plenty of waterfalls in ML pipelines as well.
  - fuzzfactor 6y ago
    The circular chart example given for Machine Learning is virtually identical to a cycle of continuous improvement in a service or manufacturing operation, except for one little detail, with Continuous Improvement you don't want the minimum viable product, you want the Maximum Viable Product. Seems like a machine learning outfit would really benefit from a leader who is smarter to begin with, a better learner, and already more skilled in the task that you want the machine to do.

talal7860 6y ago

I kind of did the same.

tachyonbeam 6y ago

I've seen such of the worst throwaway academic garbage code in ML. Maybe the problem is that they completely ignore and look down on software engineering to begin with.

hinkley 6y ago
The high concept people never appreciate the grief they're causing for the people who have to implement their vision. The stuff Frank Lloyd Wright didn't know about construction could fill an entire tour. Which I discovered by taking that tour. Boy, did that dude try to ignore physics.
mhh__ 6y ago
It's common across all academia i.e. no engineering textbook is complete without 20 pages of raw, uncommented, single-letter variable-ed MATLAB vomited before the index. I despise python, but MATLAB is positively satanic by comparison (from a software quality perspective at least)

carabiner 6y ago

I work at an old school industry with data scientists who use Python, sklearn etc. everyday. Waterfall is everything here. Software is not just at software companies.

pkaye 6y ago

The writer seems to be a recent college graduate. Maybe he doesn't have a lot of software development experience in industry.

hinkley 6y ago
Pick a discipline, and you will find a bunch of people who can tell you why the conventional wisdom doesn't apply to them. It's more common in the young, but also has a much better prognosis. In particular I've seen my share of code written by non-developers, and they all make the same classes of mistakes one makes when you have less than a couple thousand hours of focused development work under your belt. Well we can't do that because X, or we don't need that because Y. Oh, okay. Is that why I have to come tow you out of the proverbial mud every couple of months? Could you humor me and try a few of these things out? I think it'll help. What people fear most is automated systems that are constructed on wishful thinking, not healthy skepticism. A lot of our processes are about proving to ourselves what we think we already know. There is plenty of code under the AI umbrella for which very conventional unit tests could be written. If you can't use our practices for ML, you'd better drop what you're doing and start developing parity, instead of bullshitting yourselves and everyone else about your exceptionalism. Code that makes decisions for or about people needs more ceremony and sobriety around it, not less. If you can't do at least as well as a startup (which is a pretty low bar), then we are really in some deep doodoo. But not to worry. The good news is that when the machines take over and we have to hide underground, you'll be the first ones we eat, so you won't suffer long.
- free_rms 6y ago
  I think the issue here is where you put the ceremony. If you're iterating on your approach to a model, it really is a drag to do a bunch of bullshit in order to run an experiment that's likely to fail and just be thrown out anyways. If you're shipping that model to production and affecting people's lives, it should really be held to a different standard than the "fuck around and see what happens" experiments. And this is before we even get into unexplainability of deep neural nets.
  - 6y ago

ponker 6y ago

Most high reliability software is still waterfall. It's usually not service-delivered software since people who need high reliability software need it to run locally.

dboreham 6y ago

Or ever did.

tonyarkles 6y ago

Not too add too much snark, but the first sentence can also be paraphrased as “ML expert claims ML is critically important” :) I do tend to agree that it’s made some dramatic improvements regarding utility over the last few years, but...

tomc1985 6y ago

What are you talking about? I followed that exact process this week in the process of writing and checking in code, as did nearly every other programmer.