3 ms·
I was a data engineer for a couple years at one of the largest websites in Europe. My job was to develop applications to handle our data pipline and deploying t
by quantisan 12y ago
I was a data engineer for a couple years at one of the largest websites in Europe. My job was to develop applications to handle our data pipline and deploying the infrastructure on Amazon EC2, Redshift, EMR, etc.
What is it that you need and what are the problems you're facing?
- joeclark77 12y agoI'm coming at this as an academic, and the problem I'm hearing from companies about the whole idea of a "data scientist" is that it describes a superstar -- someone who knows math, statistics, programming, the business domain, etc, and is self-sufficient. The glowing articles about data scientists focus on the talents and skills they have, and don't tell us much about the operational "work" they do. I can't tell from the literature, for example, what kind of workflow I'd see on a kanban or scrum taskboard for a data science team. Closer to home, I am also very interested in how a university can train students to do this kind of work... so I want to understand the operational processes and not just a list of technologies the students should learn. I'm currently teaching a course on software development which touches on workflows and DevOps, and I'd like to do the same for our new analytics major.
- quantisan 12y agoYou mentioned that you know about general software development devops already, why and what do you think is different for data science workflow?