9 ms·

The way I look at it is: there are more tools in the toolbox than ever before. Which makes our judgement (the thing they really pay us for) even more important.

The way I look at it is: there are more tools in the toolbox than ever before. Which makes our judgement (the thing they really pay us for) even more important. Kubernetes, for example, is a specific solution to a specific problem. The solution is complex but so is the problem. If k8s give you the right trade-offs for your situation, then it’s not busy work. Of course, there are plenty of project where judgement Is thrown out the window in favor of adding a buzzword to everyone’s resume. I’ve heard it called “promotion-based architecture”, as in you pick the technology most likely to get you promoted. (If that works, it says all sorts of not great things about your organization). Regardless, I don’t think the availability of tools is the root problem. It’s and industry-wide lack of emphasis on identifying and understanding the problem first.

zivkovicp 4y ago
This was very well put. +1
ojciecczas 4y ago
"promotion-based architecture" aka "CV-driven architecture" :)
- marvinblum 4y ago
  No, it's called "résumé-driven design".
kevinmhickey 4y ago
I call it Resume Driven Development
- foofoo4u 4y ago
  Yes, this has been my observation as well. I also see this happening concurrently with what I call the "shiny object" problem. Developers have their eye on a shiny new library, framework, language, etc. and have a seated desire to use it at the expense of it being an inappropriate choice for the company. I believe these two phenomenons are producing a positive feedback loop in the industry. Selected technologies address one challenge, but introduce complexity. The complexity becomes difficult to manage. So other technologies are incorporated to manage the complexity. In the midst of all this, core technologies are replaced, swapped out, or transitioned to under the desires of the dev(s). For example, switching from one js rendering library to another, while preserving the old legacy code. The complexity footprint keeps growing and it doesn't stop because the engineers themselves aren't entirely committed to the project. They can incorporate the new technology to the project, pad their resume, and bail to a new employer if things grow out of wack.
- avgDev 4y ago
  I worked with someone who did this. I'm glad he has moved on. He would not do basic tasks correctly. Had an attention span of a toddler. He was DBA and couldn't google simple problems. I asked him to do something he would say give me 20 minutes. 2 weeks later he would say "IT CANT BE DONE". Then, I would do it myself in a few minutes after googling the error. Instead of applying knowledge, he would bring up all these buzzwords to meetings and not really understand what he was talking about. I get angry just thinking about it.
  - digisign 4y ago
    Hmm, incompetent folks exist and need to be let go. However, I think it is a slightly different category than RDD.
  - arinlen 4y ago
    > I worked with someone who did this. Nothing in your comment points to resume-driven development. You even failed to mention any project or design decision. You just decided to get angry at a coworker because of something.
i_like_waiting 4y ago
Promotion based architecture is self fulfilling prophecy at least in BI/ data world. I see everybody around me moving to cloud, without really good explanation why. Only reasonable thing I can see as an pattern is that cloud experience on top of data things gets paid 30% more. It made me consider cloud a lot. I was considering switching to cloud, just so I can put in my CV "experience with migration to cloud". For next person commenting that it makes sense: it doesn't with 200gb database and super predictable workload, growth and usage.
- morelisp 4y ago
  Hell, I'm working with ~200TB and it still doesn't make sense if you work out the costs. As far as I can tell it only makes sense if you have a ton of data, and only ever use a fraction of it at a time, for one-off jobs, infrequently.
- recursive 4y ago
  There are non-IT perspectives on this as well. My company offers two deployment scenarios: host it yourself and cloud hosted (SaaS). Many of our customers choose the latter because their internal IT systems require more process, and they just want to get something up and running.
  - wil421 4y ago
    Just to add, at my last company it could take up to 6 months to get a VM provisioned on prem. We could provision what we needed in Azure on demand.
- garciasn 4y ago
  >For next person commenting that it makes sense: it doesn't with 200gb database and super predictable workload, growth and usage. Depends on the company. I've been working for marketing agencies for the last 15 years and they're generally staffed by, at most, 1 IT person who is in charge of a third-party vendor relationship that offers managed IT services. Those IT resources (internal or vendor) don't specialize in data and often don't know how to deal with it well, predictable workloads or not, and often offer up solutions which are not appropriate (cost or otherwise) whereas there are managed solutions from cloud vendors that do. BigQuery, for example, handles compute and storage for you and rolls it into one reasonable query price. No need to worry about any management of anything there and just sit Data Studio (included) or any other BI tool (Tableau, etc) on top and you're good to go. I get your skepticism and welcome it, but you're being a little rough. We're cloud native at my current company (I did a partial cloud migration at my last which was completed after I left) and it makes my life leading a Data Engineering and Data Science team MUCH easier without upfront hardware/software costs or long-term contracts, which were STAGGERING and left us with much more difficult to maintain, upgrade, etc hardware that took up most of my job as opposed to almost none of it today. YMMV.
johnflan 4y ago
the auld mortgage driven development strategy
MattGaiser 4y ago
> It’s and industry-wide lack of emphasis on identifying and understanding the problem first. Or just no real incentives of the people involved to do so. As a dev, I don't get any real credit for biz outcomes.
_vdpp 4y ago
I’d argue the quality of tools is the problem. I am starting to view every new tool as just a new set of log messages I have to sort through to figure out why things aren’t working. I spend more time trawling GitHub issues to find workarounds than I do actually using the tools.
wnevets 4y ago
> “promotion-based architecture” That's a great way to describe the phenomenon.

zivkovicp 4y ago

This was very well put. +1

ojciecczas 4y ago

"promotion-based architecture" aka "CV-driven architecture" :)

marvinblum 4y ago
No, it's called "résumé-driven design".

kevinmhickey 4y ago

I call it Resume Driven Development

foofoo4u 4y ago
Yes, this has been my observation as well. I also see this happening concurrently with what I call the "shiny object" problem. Developers have their eye on a shiny new library, framework, language, etc. and have a seated desire to use it at the expense of it being an inappropriate choice for the company. I believe these two phenomenons are producing a positive feedback loop in the industry. Selected technologies address one challenge, but introduce complexity. The complexity becomes difficult to manage. So other technologies are incorporated to manage the complexity. In the midst of all this, core technologies are replaced, swapped out, or transitioned to under the desires of the dev(s). For example, switching from one js rendering library to another, while preserving the old legacy code. The complexity footprint keeps growing and it doesn't stop because the engineers themselves aren't entirely committed to the project. They can incorporate the new technology to the project, pad their resume, and bail to a new employer if things grow out of wack.
avgDev 4y ago
I worked with someone who did this. I'm glad he has moved on. He would not do basic tasks correctly. Had an attention span of a toddler. He was DBA and couldn't google simple problems. I asked him to do something he would say give me 20 minutes. 2 weeks later he would say "IT CANT BE DONE". Then, I would do it myself in a few minutes after googling the error. Instead of applying knowledge, he would bring up all these buzzwords to meetings and not really understand what he was talking about. I get angry just thinking about it.
- digisign 4y ago
  Hmm, incompetent folks exist and need to be let go. However, I think it is a slightly different category than RDD.
- arinlen 4y ago
  > I worked with someone who did this. Nothing in your comment points to resume-driven development. You even failed to mention any project or design decision. You just decided to get angry at a coworker because of something.

i_like_waiting 4y ago

Promotion based architecture is self fulfilling prophecy at least in BI/ data world. I see everybody around me moving to cloud, without really good explanation why. Only reasonable thing I can see as an pattern is that cloud experience on top of data things gets paid 30% more. It made me consider cloud a lot. I was considering switching to cloud, just so I can put in my CV "experience with migration to cloud". For next person commenting that it makes sense: it doesn't with 200gb database and super predictable workload, growth and usage.

morelisp 4y ago
Hell, I'm working with ~200TB and it still doesn't make sense if you work out the costs. As far as I can tell it only makes sense if you have a ton of data, and only ever use a fraction of it at a time, for one-off jobs, infrequently.
recursive 4y ago
There are non-IT perspectives on this as well. My company offers two deployment scenarios: host it yourself and cloud hosted (SaaS). Many of our customers choose the latter because their internal IT systems require more process, and they just want to get something up and running.
- wil421 4y ago
  Just to add, at my last company it could take up to 6 months to get a VM provisioned on prem. We could provision what we needed in Azure on demand.
garciasn 4y ago
>For next person commenting that it makes sense: it doesn't with 200gb database and super predictable workload, growth and usage. Depends on the company. I've been working for marketing agencies for the last 15 years and they're generally staffed by, at most, 1 IT person who is in charge of a third-party vendor relationship that offers managed IT services. Those IT resources (internal or vendor) don't specialize in data and often don't know how to deal with it well, predictable workloads or not, and often offer up solutions which are not appropriate (cost or otherwise) whereas there are managed solutions from cloud vendors that do. BigQuery, for example, handles compute and storage for you and rolls it into one reasonable query price. No need to worry about any management of anything there and just sit Data Studio (included) or any other BI tool (Tableau, etc) on top and you're good to go. I get your skepticism and welcome it, but you're being a little rough. We're cloud native at my current company (I did a partial cloud migration at my last which was completed after I left) and it makes my life leading a Data Engineering and Data Science team MUCH easier without upfront hardware/software costs or long-term contracts, which were STAGGERING and left us with much more difficult to maintain, upgrade, etc hardware that took up most of my job as opposed to almost none of it today. YMMV.

johnflan 4y ago

the auld mortgage driven development strategy

MattGaiser 4y ago

> It’s and industry-wide lack of emphasis on identifying and understanding the problem first. Or just no real incentives of the people involved to do so. As a dev, I don't get any real credit for biz outcomes.

_vdpp 4y ago

I’d argue the quality of tools is the problem. I am starting to view every new tool as just a new set of log messages I have to sort through to figure out why things aren’t working. I spend more time trawling GitHub issues to find workarounds than I do actually using the tools.

wnevets 4y ago

> “promotion-based architecture” That's a great way to describe the phenomenon.