Depending on how folks are defining data engineering these days, my group may or may not be using Scala for that purpose. There's another group of computational

Depending on how folks are defining data engineering these days, my group may or may not be using Scala for that purpose. There's another group of computational biologists (read: data scientists who know biology) who come up with novel algorithms to gain new insight in our raw data and then our group takes those algorithms (among other things) and gets them working at massive scale in well vetted data pipelines. Our company refers to it as data engineering (as in "software engineering the data science") but I know that's not what everyone means when they say that. The use of Scala predates me, it's been going on about 5 years although there's been a huge shift in that direction over the last year. This time last year I'd say it was about 50/50 Scala and Java and now almost all new code is written in Scala. I've been told that the initial "why" of scala was that it made it easy to write DSLs with which to write pipeline data flows w/o leaving the JVM. We have our own pipeline infrastructure written in house (in scala) although we're starting to move toward more OTS options (also in scala). The surge over the last year has more to do with personnel changes within the group with more people joining who were more pro-Scala (such as myself), mainly for some combination of the functional trappings and the reduction of boilerplate and other nonsense from Java. In terms of the job market, while we'd love to see a candidate who is a Scala wiz, we don't expect them to have an ounce of scala background. Personally I don't care if they have any java background either, although not everyone in my group agrees. I lean very heavily on the side of hiring smart people instead of hiring for buzzwords, assuming that we're not talking about contract positions. FWIW the data scientists in our group are mostly split between R and Python, with some also writing a fair amount of scala. The latter has more to do with some of the in house pipeline management tools than any data science-y reasons.