Your characterization of the typical job marketplace in tech jives pretty closely with my experience. I've often said, in public speaking forums no less, that

Your characterization of the typical job marketplace in tech jives pretty closely with my experience. I've often said, in public speaking forums no less, that to me, "The overwhelming majority of the 'Big Data' marketplace seems predicated on selling to the hubris that software engineers want to believe they have problems bigger than they actually are." Despite being a denizen of the whole "NoSQL" thing personally. That said, in my particular case it had a lot less to do with trying to necessarily rebuild HDFS from nothing, and more to do with a mindset of rigor and principles necessary to do so. Because being able to work all the way through that problem domain in both broad strokes and in meticulous detail would hopefully lend itself toward also considering ways to validate and attest the correctness of not only things like HDFS (rather than treating it like a solved problem ready built for use), but also applying that same level of rigor and principle to the stuff we actually do have to build from scratch. Though to your point... a non-trivial amount of this concern and necessity is borne out of the market and regulatory regimes this stuff has to service and abide. That fact that it's not necessary for huge swaths of the marketplace is evidenced by the fact that things like property-based testing, mutation testing, chaos testing, and formal verification are fringe skills (at best) out there... yet the tech world continues to turn out totally awesome cool new stuff with none of that overhead all that time that still transforms all manner of life. I actually think that "developers" and "engineers" are mostly pretty transparent about what sort they are. Or at the very least it's trivially easy to assess within just a few minutes with the right questions and conversation space. The harder part is getting non-technical people to understand that there's a distinction and that technical people aren't all just a fungible commodity. The weirdest part is that they get that on some level, especially when suddenly they're hit by a bus-factor problem, but that realization hasn't seemed to make a big impact in business/hiring process, practices, etc.