7 ms·

From the article: "Data engineers are the janitors who keep your data clean and flowing." Hm, I wonder why he's having problems hiring janitors.

kafkaesq 10y ago
Data engineers are the janitors who keep your data clean and flowing. In a boldface font, no less. The cockiness behind that language is really quite astounding.
- praccu 10y ago
  It's really true, though. It's brutal, ugly work with no hope of an end. Edit: Favorite paper on the topic: http://research.google.com/pubs/pub43146.html http://research.google.com/pubs/pub43146.html
  - kafkaesq 10y ago
    So is the work that doctors, lawyers, and other highly-skilled people do, by and large. Everyone knows that day-to-day aspects of these jobs are hardly glamorous (or even cerebral), the vast majority of the time. Yet somehow we manage to accord these people with their due degree of respect, and wouldn't think of referring to them as "janitors".
    - wavefunction 10y ago
      I don't see what's so bad about janitors, though. They do very thankless jobs for not much money, whereas doctors and lawyers and other high-skilled individuals are often well renumerated or offered certain social prestige that your post shows is quite lacking when a humble janitor is considered.
      - kafkaesq 10y ago
        A perfectly valid point. But getting back to the original article -- it pretty much takes a SV alpha-nerd (or aspiring CEO seeking to cater to them) to come up with language like that.
      - Grishnakh 10y ago
        There's nothing wrong with janitors, but the work they do can be done by any dumb monkey with almost no training. That's why the pay is so low for the job: anyone can do it, as long as they can lift trash cans and push a vacuum cleaner. Plumbers are entirely different. They have to get their hands dirty working on some awful systems, but they actually have to know what they're doing, get specialized training, etc. Soldering a proper joint with copper pipes isn't that easy, and if you screw it up, it'll leak later and cause a lot of property damage. Knowing which pipes and fittings to use where is specialized knowledge. It's not something you can just grab someone off the street and train them to do in 30 minutes. Of course, plumbers also cost a lot too, and the ones who are self-employed (rather than their assistants) generally do pretty well financially.
pavlov 10y ago
Bizarrely, I remember a recent HN discussion where a poster was arguing that any software developer who is not working in machine learning is like a plumber. I guess this means that the entire profession consists of janitors and plumbers.
- p4wnc6 10y ago
  In the minds of middle management, I think this is precisely correct.
  - triplesec 10y ago
    and as pointed out so much, is entirely why nobody wants to work for them. Respect these very bright people znd you have a starting negotiation position.
- biztos 10y ago
  I recently had a plumber do some work on a >100-year-old apartment. I was lucky: he's a very good plumber. The job didn't involve too many "pipelines" but the knowledge and creativity required to make them work was well above what I see from most software developers. "Plumber" is not the put-down that poster thought it was.
- jballanc 10y ago
  Considering that plumbers and janitors have likely, in the entire history of human civilization, done more for health and longevity than doctors and scientists...I'm kind of ok with this analogy.
  - hga 10y ago
    Doctors, maybe, but it was the scientists who told them about the germ theory of disease, for instance. I've read, but not confirmed for myself, that in the US the biggest gains in health came in the post-Civil War period, when "plumbers and janitors" made the difference. Of course, that's really starting with, after the science, the civil engineers who designed the public works systems that supplied clean water and took away sewage, and let's not forget that politicians and like who found it worthwhile to buy votes that way (now, they take our infrastructure for granted and buy votes more directly...).
jrs235 10y ago
Janitors? They are certainly more than janitors! More like plumbers... getting your data safely from point a to point b without plugging things up while passing through [process] boundary's. How much does a plumber cost? $140 / hr? Sounds about right.
jakestein 10y ago
Hey I'm the author of this blog post and the CEO of the company that did the benchmark report. That was a very poor choice of words on my part, and I appreciate you flagging it. I reworked the paragraph to remove the janitor comment and (hopefully) make it clearer.
- wavefunction 10y ago
  You should also not use "janitor" as a disparaging term. That would be another good takeaway from all of this.
  - jakestein 10y ago
    I agree that it's a bad idea to use "janitor" as a disparaging term, and that was very far away from my intention. If that was what you took away from reading it, then that's more evidence that I didn't do a great job with writing the original draft. Here's the original paragraph for reference: Data engineers are the janitors who keep your data clean and flowing. Insights are great, and you need them. But to deliver insights at scale, you need data infrastructure. That’s delivered by data engineering. It’s not as fun to talk about as D3 visualizations and business intelligence dashboards, but it’s every bit as important.

kafkaesq 10y ago

Data engineers are the janitors who keep your data clean and flowing. In a boldface font, no less. The cockiness behind that language is really quite astounding.

praccu 10y ago
It's really true, though. It's brutal, ugly work with no hope of an end. Edit: Favorite paper on the topic: http://research.google.com/pubs/pub43146.html http://research.google.com/pubs/pub43146.html
- kafkaesq 10y ago
  So is the work that doctors, lawyers, and other highly-skilled people do, by and large. Everyone knows that day-to-day aspects of these jobs are hardly glamorous (or even cerebral), the vast majority of the time. Yet somehow we manage to accord these people with their due degree of respect, and wouldn't think of referring to them as "janitors".
  - wavefunction 10y ago
    I don't see what's so bad about janitors, though. They do very thankless jobs for not much money, whereas doctors and lawyers and other high-skilled individuals are often well renumerated or offered certain social prestige that your post shows is quite lacking when a humble janitor is considered.
    - kafkaesq 10y ago
      A perfectly valid point. But getting back to the original article -- it pretty much takes a SV alpha-nerd (or aspiring CEO seeking to cater to them) to come up with language like that.
    - Grishnakh 10y ago
      There's nothing wrong with janitors, but the work they do can be done by any dumb monkey with almost no training. That's why the pay is so low for the job: anyone can do it, as long as they can lift trash cans and push a vacuum cleaner. Plumbers are entirely different. They have to get their hands dirty working on some awful systems, but they actually have to know what they're doing, get specialized training, etc. Soldering a proper joint with copper pipes isn't that easy, and if you screw it up, it'll leak later and cause a lot of property damage. Knowing which pipes and fittings to use where is specialized knowledge. It's not something you can just grab someone off the street and train them to do in 30 minutes. Of course, plumbers also cost a lot too, and the ones who are self-employed (rather than their assistants) generally do pretty well financially.

pavlov 10y ago

Bizarrely, I remember a recent HN discussion where a poster was arguing that any software developer who is not working in machine learning is like a plumber. I guess this means that the entire profession consists of janitors and plumbers.

p4wnc6 10y ago
In the minds of middle management, I think this is precisely correct.
- triplesec 10y ago
  and as pointed out so much, is entirely why nobody wants to work for them. Respect these very bright people znd you have a starting negotiation position.
biztos 10y ago
I recently had a plumber do some work on a >100-year-old apartment. I was lucky: he's a very good plumber. The job didn't involve too many "pipelines" but the knowledge and creativity required to make them work was well above what I see from most software developers. "Plumber" is not the put-down that poster thought it was.
jballanc 10y ago
Considering that plumbers and janitors have likely, in the entire history of human civilization, done more for health and longevity than doctors and scientists...I'm kind of ok with this analogy.
- hga 10y ago
  Doctors, maybe, but it was the scientists who told them about the germ theory of disease, for instance. I've read, but not confirmed for myself, that in the US the biggest gains in health came in the post-Civil War period, when "plumbers and janitors" made the difference. Of course, that's really starting with, after the science, the civil engineers who designed the public works systems that supplied clean water and took away sewage, and let's not forget that politicians and like who found it worthwhile to buy votes that way (now, they take our infrastructure for granted and buy votes more directly...).

jrs235 10y ago

Janitors? They are certainly more than janitors! More like plumbers... getting your data safely from point a to point b without plugging things up while passing through [process] boundary's. How much does a plumber cost? $140 / hr? Sounds about right.

jakestein 10y ago

Hey I'm the author of this blog post and the CEO of the company that did the benchmark report. That was a very poor choice of words on my part, and I appreciate you flagging it. I reworked the paragraph to remove the janitor comment and (hopefully) make it clearer.

wavefunction 10y ago
You should also not use "janitor" as a disparaging term. That would be another good takeaway from all of this.
- jakestein 10y ago
  I agree that it's a bad idea to use "janitor" as a disparaging term, and that was very far away from my intention. If that was what you took away from reading it, then that's more evidence that I didn't do a great job with writing the original draft. Here's the original paragraph for reference: Data engineers are the janitors who keep your data clean and flowing. Insights are great, and you need them. But to deliver insights at scale, you need data infrastructure. That’s delivered by data engineering. It’s not as fun to talk about as D3 visualizations and business intelligence dashboards, but it’s every bit as important.