3 ms·

One thing many programmers fail to realize is that almost all problems in the world are ultimately political, not technical. Data analysis is almost always a to

One thing many programmers fail to realize is that almost all problems in the world are ultimately political, not technical. Data analysis is almost always a tool, not an answer. The hardest job in the world is getting a large group of people to all do something. The way you message something can be as important as what you say. If I was working on a major schedule change project like this, I wouldn't even tell people about the algorithm. That's just a tool to help you arrive at a schedule. People don't want to feel like their life is controlled by an unfeeling machine even if the result is better for them. I'm not saying this particular situation was winnable. But I've seen this kind of thing happen over and over in the tech world.

bitL 8y ago
That's why the smartest people should shut up, create a great technology that solves issues people can't agree on for centuries, then entice young smart generation to adopt it and then overrun the issue in the real world by being the only solution. Once politics and interests get involved, it's over.
- Bartweiss 8y ago
  I'm pretty sympathetic to this tactic - it's amazing how many 'intractable' political and social debates have been solved by rendering the whole problem moot for a group that's not already entrenched. Science rather famously "advances one funereal at a time", and most debates over resources don't get settled so much as growth and technology render them uninteresting. (Recall that brightly colored fabrics were once a source of enormous wealth and even legislation.) But I'll be damned if I can work out how to apply it to childhood education. It's almost definitionally a system run by the old guard, on 'behalf' of people too young to change anything. A lot of my friends and I got to our late teens and twenties, swore up and down we were going to fix some of the obvious idiocies and cruelties and wastes of education, and just utterly failed. There's no population to offer a competing system to, and very little road into the existing for any member of the 'young smart generation' which lacks either children (to be a stakeholder) or multiple degrees and time in the trenches (to be a participant).
mcrad 8y ago
This is to say that data mining analytics in business context is far more akin to journalism than data "science"
lodrein 8y ago
I agree that the role of politics is important, and maybe it is for the best if it can help create a conversation. One of the main objective of this particular project was to make a complex problem more accessible to a general audience, so that such a debate could take place. For example, before this "algorithm", it was harder to understand the impact of your school time on the other schools and the budget of the district. And this budget is also what pays for teachers, nurses and buildings. My hope is to convey that yes, algorithms can be scary when they decide for you, but sometimes they are just tools that bring information to the debate and help the minorities be heard. And ultimately, humans make the important decisions.
- ageitgey 8y ago
  Thanks for your insight. I think the work itself was really interesting and super useful. My point is just that often great work is a tiny component of successfully convincing a semi-rational and biased populace of agreeing to something. It's just plain hard. I hope you'll keep on doing great stuff like this even if sometimes the surrounding political situation outside of your control is messy and sometimes feels discouraging.
Bartweiss 8y ago
Certainly there are an awful lot of startups that fail over issues like this. They either invest massive effort into seeking technical fixes that don't exist for human problems, or they find genuinely superior technical answers that are impossible to achieve - because they upset key parties, or because the cost of change is too high, or because they just sound too damn silly to sell people on. But in this case, I'm not sure there was anything to be done. (As you acknowledge - I'm not disagreeing, just looking at the point.) Two grad students whose background was overwhelmingly technical were brought in to provide a technical solution. They had enormous amounts of logistical information, but it's not clear they had social/political information about their task, and certainly they had no control over messaging and notification. The outrage appears to have been less about the use of an algorithm than about massive disruptions to schedules. It's a predictable outcome - school start times are largely set around common parent work schedules - but it's also an irreducible one. The desired outcome was the source of the anger, however much the computer derivation added insult to injury. Doctors have been calling for these changes for years and been utterly ignored. The school district I grew up in tried something a lot like this a decade ago, with no fancy algorithm, and was slapped down with similar fury. At a certain point I think the error was not the superintendent seeking a technical solution to changing start times, but seeking a solution at all on a topic where parents didn't want change. (If I were to really get inflammatory, I might mention that school is a classic principal-agent problem, and the proposal failed because it tried to trade the agents' convenience for the principals' health. But that's another, uglier debate.)
- ageitgey 8y ago
  > At a certain point I think the error was not the superintendent seeking a technical solution to changing start times, but seeking a solution at all on a topic where parents didn't want change. Agreed. It's a super hard problem to solve. But this situation as a whole I think is still a good case study that developers should consider. You can do the best job possible and still get "blame" because of the political situation beyond your control. You can't fight politics and emotion with data alone.
- ghaff 8y ago
  >(If I were to really get inflammatory, I might mention that school is a classic principal-agent problem, and the proposal failed because it tried to trade the agents' convenience for the principals' health. But that's another, uglier debate.) Or, at least (to phrase it in a slightly less negative way) to trade relatively incremental, diffuse, and somewhat abstract benefits for the students with specific and sometimes disruptive change for parents. Furthermore, just about any change to an accepted status quo--even if it's arguably a mild net positive--is going to have a lot of vocal critics. Those who like the change will mostly not feel very strongly. Those who don't like it or have just learned to live with the status quo will often hate it.

bitL 8y ago

That's why the smartest people should shut up, create a great technology that solves issues people can't agree on for centuries, then entice young smart generation to adopt it and then overrun the issue in the real world by being the only solution. Once politics and interests get involved, it's over.

Bartweiss 8y ago
I'm pretty sympathetic to this tactic - it's amazing how many 'intractable' political and social debates have been solved by rendering the whole problem moot for a group that's not already entrenched. Science rather famously "advances one funereal at a time", and most debates over resources don't get settled so much as growth and technology render them uninteresting. (Recall that brightly colored fabrics were once a source of enormous wealth and even legislation.) But I'll be damned if I can work out how to apply it to childhood education. It's almost definitionally a system run by the old guard, on 'behalf' of people too young to change anything. A lot of my friends and I got to our late teens and twenties, swore up and down we were going to fix some of the obvious idiocies and cruelties and wastes of education, and just utterly failed. There's no population to offer a competing system to, and very little road into the existing for any member of the 'young smart generation' which lacks either children (to be a stakeholder) or multiple degrees and time in the trenches (to be a participant).

mcrad 8y ago

This is to say that data mining analytics in business context is far more akin to journalism than data "science"

lodrein 8y ago

I agree that the role of politics is important, and maybe it is for the best if it can help create a conversation. One of the main objective of this particular project was to make a complex problem more accessible to a general audience, so that such a debate could take place. For example, before this "algorithm", it was harder to understand the impact of your school time on the other schools and the budget of the district. And this budget is also what pays for teachers, nurses and buildings. My hope is to convey that yes, algorithms can be scary when they decide for you, but sometimes they are just tools that bring information to the debate and help the minorities be heard. And ultimately, humans make the important decisions.

ageitgey 8y ago
Thanks for your insight. I think the work itself was really interesting and super useful. My point is just that often great work is a tiny component of successfully convincing a semi-rational and biased populace of agreeing to something. It's just plain hard. I hope you'll keep on doing great stuff like this even if sometimes the surrounding political situation outside of your control is messy and sometimes feels discouraging.

Bartweiss 8y ago

Certainly there are an awful lot of startups that fail over issues like this. They either invest massive effort into seeking technical fixes that don't exist for human problems, or they find genuinely superior technical answers that are impossible to achieve - because they upset key parties, or because the cost of change is too high, or because they just sound too damn silly to sell people on. But in this case, I'm not sure there was anything to be done. (As you acknowledge - I'm not disagreeing, just looking at the point.) Two grad students whose background was overwhelmingly technical were brought in to provide a technical solution. They had enormous amounts of logistical information, but it's not clear they had social/political information about their task, and certainly they had no control over messaging and notification. The outrage appears to have been less about the use of an algorithm than about massive disruptions to schedules. It's a predictable outcome - school start times are largely set around common parent work schedules - but it's also an irreducible one. The desired outcome was the source of the anger, however much the computer derivation added insult to injury. Doctors have been calling for these changes for years and been utterly ignored. The school district I grew up in tried something a lot like this a decade ago, with no fancy algorithm, and was slapped down with similar fury. At a certain point I think the error was not the superintendent seeking a technical solution to changing start times, but seeking a solution at all on a topic where parents didn't want change. (If I were to really get inflammatory, I might mention that school is a classic principal-agent problem, and the proposal failed because it tried to trade the agents' convenience for the principals' health. But that's another, uglier debate.)

ageitgey 8y ago
> At a certain point I think the error was not the superintendent seeking a technical solution to changing start times, but seeking a solution at all on a topic where parents didn't want change. Agreed. It's a super hard problem to solve. But this situation as a whole I think is still a good case study that developers should consider. You can do the best job possible and still get "blame" because of the political situation beyond your control. You can't fight politics and emotion with data alone.
ghaff 8y ago
>(If I were to really get inflammatory, I might mention that school is a classic principal-agent problem, and the proposal failed because it tried to trade the agents' convenience for the principals' health. But that's another, uglier debate.) Or, at least (to phrase it in a slightly less negative way) to trade relatively incremental, diffuse, and somewhat abstract benefits for the students with specific and sometimes disruptive change for parents. Furthermore, just about any change to an accepted status quo--even if it's arguably a mild net positive--is going to have a lot of vocal critics. Those who like the change will mostly not feel very strongly. Those who don't like it or have just learned to live with the status quo will often hate it.