4 ms·
Yeah, DE seems to have emerged as the title for those who manage large databases -- who regularly import raw data, cleanse, normalize, update, construct analysi
by randcraw 10y ago
Yeah, DE seems to have emerged as the title for those who manage large databases -- who regularly import raw data, cleanse, normalize, update, construct analysis pipelines, scale up and parallelize processes, and validate results. It lies somewhere between a DBA, sysadmin, and HPC engineer, with an awareness of basic stats -- where mastery of Hadoop's many components might converge.
It seems like a natural evolution of DB admin for very large scale noSQL and DBs, esp. those with unstructured data and often non-commodity architecture. I've seen numerous companies looking for such folks and I suspect demand will rise.
IMO, it's not a job you'd want to outsource. The role is too mission critical, the skills not predictable enough to be a commodity, and the penalty for screwing up is too great.