3 ms·
This isn't interesting or new. When Python and SQL support was added to Spark many years ago, Data Engineers quickly embraced it. It's standard amongst all Da
by threeseed 3y ago
This isn't interesting or new.
When Python and SQL support was added to Spark many years ago, Data Engineers quickly embraced it.
It's standard amongst all Data Science people that you know both. But the people who actually do well in the industry know a lot more such as what the underlying infrastructure e.g. JVM is doing. As well as some other languages like Scala or R for more specific tasks.
Data Engineers in particular are expected to take full ownership over performance and reliability as well. So more than a little bit of cross-over with DevOps and SRE.