4 ms·

Love it; one of the biggest things we've preached to users of our computer vision dev-tool is: it doesn't matter how much time you spend on your model if your d

Love it; one of the biggest things we've preached to users of our computer vision dev-tool is: it doesn't matter how much time you spend on your model if your data is bad. When you're trying to get a first version of your CV project launched, you can usually pretty much ignore the model (and pick one up off the shelf) because the time you spend on improving your data is orders of magnitude higher leverage. Someday the model might become the limiting re-agent but it's much, much later than people assume (if at all). But the model is the "fun" part for technical people to play with (and academia encourages treating your data as an immutable constant) so it gets a lot of attention at the expense of the data.

naijaboiler 4y ago
I think what was surprising for me dealing with junior data scientists was that I had to make this point. Why are you killing yourself trying to get insane accuracy on dirty data? Your energy is better spent getting better data.
- joshspankit 4y ago
  Some possible insight: For some of us there is a particular pull to tasks that are significant, complicated, and seem to have a finite solution. Cleaning/parsing existing data strikes me as one of those.
- ramraj07 4y ago
  It’s not a junior data scientist thing, it’s a bad DS thing. Just shows that they can’t see the forest for the trees and if somehow you make them realize this problem there will be the next problem with the same result.
- robertlagrant 4y ago
  Getting the data better is not the fun bit, I imagine.
  - gunshai 4y ago
    oh but it certainly can be depending on the industry you are in. It can just be frustrating because you want to deliver results and this realization can get you stuck in the mud of "We need better measuring processes." However you also get to advocate for why these measuring practices matter.
  - didgetmaster 4y ago
    Cleaning data is usually very low on the totem pole of priorities for many companies. It is often a very manual and time consuming process. Support from management to fix it can be very poor. Relational tables with millions or billions of rows can contain many different kinds of errors. Finding and fixing them can be like finding a few dozen needles in a field of haystacks. I was working on a tool to make it much easier to find and fix anomalies. I used the Chicago crime data (available for free download on the city's open data portal) to show how easy it was to find and fix problems. Here is a short video of some of the things it found: https://www.youtube.com/watch?v=kqkNeU1LYEQ https://www.youtube.com/watch?v=kqkNeU1LYEQ I contacted the people responsible for managing the data to show them some of the problems and how to fix them. They seemed like they could not be any less interested.
tejtm 4y ago
I have come to see it as a sort of "Maslow's Hierarchy" for Data-needs where the base is getting data and the top is producing "results". Where in academia "results" is publishing papers in other industries it may be something else. Somewhere between the top (glorious presentation) and bottom (no data) we can draw a fuzzy line that partitions the "glam stack" and the "drudge stack". The more the reward structure is skewed toward the top, the greater the incentive is to just cook the data.

naijaboiler 4y ago

I think what was surprising for me dealing with junior data scientists was that I had to make this point. Why are you killing yourself trying to get insane accuracy on dirty data? Your energy is better spent getting better data.

joshspankit 4y ago
Some possible insight: For some of us there is a particular pull to tasks that are significant, complicated, and seem to have a finite solution. Cleaning/parsing existing data strikes me as one of those.
ramraj07 4y ago
It’s not a junior data scientist thing, it’s a bad DS thing. Just shows that they can’t see the forest for the trees and if somehow you make them realize this problem there will be the next problem with the same result.
robertlagrant 4y ago
Getting the data better is not the fun bit, I imagine.
- gunshai 4y ago
  oh but it certainly can be depending on the industry you are in. It can just be frustrating because you want to deliver results and this realization can get you stuck in the mud of "We need better measuring processes." However you also get to advocate for why these measuring practices matter.
- didgetmaster 4y ago
  Cleaning data is usually very low on the totem pole of priorities for many companies. It is often a very manual and time consuming process. Support from management to fix it can be very poor. Relational tables with millions or billions of rows can contain many different kinds of errors. Finding and fixing them can be like finding a few dozen needles in a field of haystacks. I was working on a tool to make it much easier to find and fix anomalies. I used the Chicago crime data (available for free download on the city's open data portal) to show how easy it was to find and fix problems. Here is a short video of some of the things it found: https://www.youtube.com/watch?v=kqkNeU1LYEQ https://www.youtube.com/watch?v=kqkNeU1LYEQ I contacted the people responsible for managing the data to show them some of the problems and how to fix them. They seemed like they could not be any less interested.

tejtm 4y ago

I have come to see it as a sort of "Maslow's Hierarchy" for Data-needs where the base is getting data and the top is producing "results". Where in academia "results" is publishing papers in other industries it may be something else. Somewhere between the top (glorious presentation) and bottom (no data) we can draw a fuzzy line that partitions the "glam stack" and the "drudge stack". The more the reward structure is skewed toward the top, the greater the incentive is to just cook the data.