3 ms·
I think this diagnoses the problem well, but ignores an obvious solution. A team of one data scientist and one engineer, completely responsible for building a
by jimbokun 8y ago
I think this diagnoses the problem well, but ignores an obvious solution.
A team of one data scientist and one engineer, completely responsible for building a model, and seeing it through into production, meeting all applicable SLAs and performance metrics.
Or maybe it's two data scientists and one engineer, or one scientist and two engineers, whatever is required.
The point is to have a small team you can hold completely accountable for their output. They sink or swim together, so there is no debating whether the scientists or engineers get the credit or take the blame. They are assessed by the effectiveness of the end product they produce.
- disposedtrolley 8y agoSmall teams are awesome in so many scenarios. I recently wrapped up a 4-week proof of concept for a client on knowledge management and discovery using NLP. I was able to work with someone apt at machine learning while I focused on building out the UI and backend. We delivered a first release about 3 days after we started, giving ample time to seek feedback and let the users shape the direction.
- tkyjonathan 8y agoIn 4 weeks, I was able to create a data mart that had self-healing (we had issues with Python/events missing data which should have reached the existing data warehouse) and the physical data models in it sped up an existing 6 hour ETL task down to 0.15 seconds AND speed up a production query that took 5 seconds per click down to 0.07 seconds. No team needed or proof of concepts. Actual working data models, up to date tables + ETL code in production. Background is DBA.