5 ms·

This is fantastic. When people ask me "what do data scientists really do?" I am going to send them to this repo and the accompanying article. This looks like a

This is fantastic. When people ask me "what do data scientists really do?" I am going to send them to this repo and the accompanying article. This looks like a massive effort to send out each FOIA request and then manually process each individual report. At the end of the day, their conclusion has little to do with actually investigating trends about police misconduct, and is almost entirely about standardization of data collection practices.

uneekname 6y ago
I'm 21 years old, going to graduate from college in a year, and reading this comment makes me want to pursue this type of data science work. For every school or personal project I've done, the joy for me has been the collecting, organizing, and analyzing of data.
- deleted 6y ago
  [deleted]
- mhh__ 6y ago
  The world needs more data scientists who actually follow a scientific approach to uncertainty and bias - good luck!
alexilliamson 6y ago
Bingo. I worked for 4 years as a data scientist for city governments, and there is so much low hanging fruit to be captured simply with better data management. The analysis/modeling almost feels perfunctory once you have the data system in the state you need.
- Buttons840 6y ago
  I worked for a company that gathered medical license information from all 50 states into a central source. We gathered the "good lists" and the "bad lists" I liked to say; the license lists and the sanction lists. I heard stories about sanctioned doctors or nurses just moving to another state to continue practicing, though I don't personally know if those stories are true, it's easy to imagine the value of checking all these lists in a central location. The problem is all this data from every state is very very similar, but it's a huge job to gather all this data together. The company was doing a very poor job of it as well, with lots of tech debt and incompetent management. There was lots of legally questionable web scraping and manual work. Many states (shamefully) charge money to be able to see what doctors and nurses have been sanctioned, and actively try to prevent the data from being scraped. All this could be eliminated with a federally mandated data format. That entire company of 300 people could be automated away easily. While I was working there I was always aware that part of everyone's high medical bill ended up in my paycheck: the hospitals and clinics paid this company, and the company paid me.
  - DavidPeiffer 6y ago
    > I heard stories about sanctioned doctors or nurses just moving to another state to continue practicing, though I don't personally know if those stories are true, it's easy to imagine the value of checking all these lists in a central location. This was one of the reasons "Dr. Death" was able to keep practicing as long as he did. He resigned from positions and switched facilities/states before the investigations could be completed. https://en.wikipedia.org/wiki/Christopher_Duntsch https://en.wikipedia.org/wiki/Christopher_Duntsch https://www.nbc.com/american-greed/video/the-real-dr-death/4304259 https://www.nbc.com/american-greed/video/the-real-dr-death/4...
    - samstave 6y ago
      I think that public positions and critical positions, such as police/LEO, Doctors, Nurses, Teachers and other positions (maybe nuke plant operators) etc - should all have a UUID for each person - and one (with appropriate reasoning) should be able to track the UUID of any employee across the nation - such that you cant just shuffle offenders/let them shuffle themselves about assuming their is misconduct - and maybe even provide a further incentivization program for those who have stellar records - like "doctor of the month for the state" or Police officer of the year for the nation" etc...
      - Buttons840 6y ago
        Would be nice, but as a start, if we could just create a law that states can't hind their sanction lists behind paywalls, that would go a long way. You're less safe in state X because state Y charges a lot of money to get access to their sanction lists.
ed25519FUUU 6y ago
It’s strange to me you feel compelled to professionally justify your job through the lens of social justice. I feel no such obligation when people ask me what I do. As for my personal time, money, and issues that are important to me that’s a very different story. It feels a little bit to me that we’ve so eroded our personal lives compared to professional lives that people are starting to have a hard time distinguishing the two.
- uniclaude 6y ago
  This is a great thing that people intend to align what they’re paid to do the greater part of their lives with what they find just. This is hackernews, the place where people intend to make positive change with their brain, their time, and sometimes, their money.
- the_only_law 6y ago
  > It feels a little bit to me that we’ve so eroded our personal lives compared to professional lives that people are starting to have a hard time distinguishing the two. It comes up a lot on this site that a job is just a job, and thus you shouldn't focus so much on it to find meaning, fulfillment, etc., but I'll consider this when it doesn't take up majority of my daylight hours.
  - yunohn 6y ago
    +1 to this. Jobs take up the majority of our mental space and energy and hours in the day. Why shouldn’t we strive to achieve something more meaningful within it?
- scsilver 6y ago
  The money isnt filling the void man.
- mhh__ 6y ago
  If you want to do something that makes a difference in the world what's wrong with that?
- avz 6y ago

uneekname 6y ago

I'm 21 years old, going to graduate from college in a year, and reading this comment makes me want to pursue this type of data science work. For every school or personal project I've done, the joy for me has been the collecting, organizing, and analyzing of data.

deleted 6y ago
[deleted]
mhh__ 6y ago
The world needs more data scientists who actually follow a scientific approach to uncertainty and bias - good luck!

alexilliamson 6y ago

Bingo. I worked for 4 years as a data scientist for city governments, and there is so much low hanging fruit to be captured simply with better data management. The analysis/modeling almost feels perfunctory once you have the data system in the state you need.

Buttons840 6y ago
I worked for a company that gathered medical license information from all 50 states into a central source. We gathered the "good lists" and the "bad lists" I liked to say; the license lists and the sanction lists. I heard stories about sanctioned doctors or nurses just moving to another state to continue practicing, though I don't personally know if those stories are true, it's easy to imagine the value of checking all these lists in a central location. The problem is all this data from every state is very very similar, but it's a huge job to gather all this data together. The company was doing a very poor job of it as well, with lots of tech debt and incompetent management. There was lots of legally questionable web scraping and manual work. Many states (shamefully) charge money to be able to see what doctors and nurses have been sanctioned, and actively try to prevent the data from being scraped. All this could be eliminated with a federally mandated data format. That entire company of 300 people could be automated away easily. While I was working there I was always aware that part of everyone's high medical bill ended up in my paycheck: the hospitals and clinics paid this company, and the company paid me.
- DavidPeiffer 6y ago
  > I heard stories about sanctioned doctors or nurses just moving to another state to continue practicing, though I don't personally know if those stories are true, it's easy to imagine the value of checking all these lists in a central location. This was one of the reasons "Dr. Death" was able to keep practicing as long as he did. He resigned from positions and switched facilities/states before the investigations could be completed. https://en.wikipedia.org/wiki/Christopher_Duntsch https://en.wikipedia.org/wiki/Christopher_Duntsch https://www.nbc.com/american-greed/video/the-real-dr-death/4304259 https://www.nbc.com/american-greed/video/the-real-dr-death/4...
  - samstave 6y ago
    I think that public positions and critical positions, such as police/LEO, Doctors, Nurses, Teachers and other positions (maybe nuke plant operators) etc - should all have a UUID for each person - and one (with appropriate reasoning) should be able to track the UUID of any employee across the nation - such that you cant just shuffle offenders/let them shuffle themselves about assuming their is misconduct - and maybe even provide a further incentivization program for those who have stellar records - like "doctor of the month for the state" or Police officer of the year for the nation" etc...
    - Buttons840 6y ago
      Would be nice, but as a start, if we could just create a law that states can't hind their sanction lists behind paywalls, that would go a long way. You're less safe in state X because state Y charges a lot of money to get access to their sanction lists.

ed25519FUUU 6y ago

It’s strange to me you feel compelled to professionally justify your job through the lens of social justice. I feel no such obligation when people ask me what I do. As for my personal time, money, and issues that are important to me that’s a very different story. It feels a little bit to me that we’ve so eroded our personal lives compared to professional lives that people are starting to have a hard time distinguishing the two.

uniclaude 6y ago
This is a great thing that people intend to align what they’re paid to do the greater part of their lives with what they find just. This is hackernews, the place where people intend to make positive change with their brain, their time, and sometimes, their money.
the_only_law 6y ago
> It feels a little bit to me that we’ve so eroded our personal lives compared to professional lives that people are starting to have a hard time distinguishing the two. It comes up a lot on this site that a job is just a job, and thus you shouldn't focus so much on it to find meaning, fulfillment, etc., but I'll consider this when it doesn't take up majority of my daylight hours.
- yunohn 6y ago
  +1 to this. Jobs take up the majority of our mental space and energy and hours in the day. Why shouldn’t we strive to achieve something more meaningful within it?
scsilver 6y ago
The money isnt filling the void man.
mhh__ 6y ago
If you want to do something that makes a difference in the world what's wrong with that?
avz 6y ago