4 ms·

There is a large contingent of radical empiricists in machine learning who assume "big data + automation = truth", especially on HN, and this is a message they

by rspeer 9y ago

There is a large contingent of radical empiricists in machine learning who assume "big data + automation = truth", especially on HN, and this is a message they need to hear more of. People have been advocating radical empiricism in some increasingly uncomfortable contexts recently, and I hope it's just that it's the only thing they were taught and the only way they know how to think about their craft. The alternative is that an increasing number of people really do want machines to triumph over human judgment and morality.

digi_owl 9y ago
Gets me thinking of an Adam Curtis' documentary series: https://en.wikipedia.org/wiki/All_Watched_Over_by_Machines_of_Loving_Grace_%28TV_series%29 https://en.wikipedia.org/wiki/All_Watched_Over_by_Machines_o... Not exactly sure how relevant it is though.
bo1024 9y ago
This might be a case where classical econ ("social choice") can give some helpful perspective. Arrow's impossibility theorem is the most famous impossibility result in this area; it says that no algorithm can take in a set of rankings (e.g. match outcomes) and produce an aggregate ranking in a way that satisfies a small set of fairness criteria. This is classically interpreted as saying that any aggregation method must be "unfair" in one way or another.
- YokoZar 9y ago
  Arrow's theorem is mathematics, but even citing it is greatly misleading. All Arrow proved was that, sometimes, voting becomes rock-paper-scissors between the three most popular candidates. That isn't "unfair". It especially doesn't mean that all voting systems are equally bad, and yet the most common reason for people to cite Arrow's theorem is to try and dispute the idea that we can do better.
  - thaumasiotes 9y ago
    Unlike your parent comment, your comment is total nonsense. Arrow proved that, given a set of ordinal preference rankings held by several individuals, the concept of an aggregate preference ranking describing the overall "will of society" is not well defined; subject to four or five unobjectionable constraints, no function determining such an overall ranking exists. Voting systems which obey the assumptions of the theorem frequently break down in ways that cannot be described as "rock-paper-scissors between the three most popular candidates", as when strategic voting caused the papal conclave of 1334 to unanimously elect the least popular candidate to the papacy. Were you perhaps thinking of Condorcet's work when you referred to rock-paper-scissors? He, not Arrow, famously wrote about how voter preferences were not necessarily transitive.
    - YokoZar 9y ago
      That "rock-paper-scissors" cycle among the top preferences is the Smith set, and every sensible voting "violates" Arrows theorem by saying that sometimes it's ok to acknowledge that cycle. One of Arrows assumptions, "Irrelevance of Independent Alternatives", is far too strong. If you relax it to "Local Independence of Irrelevant Alternatives", then Arrows theorem no longer applies. It's just not a relevant theorem. Claiming that allowing a voting system to recognize a rock-paper-scissors cycle among the top candidates is the same as allowing dictators is obvious rubbish. But if you interpret Arrow's theorem naively, you might be tempted to make such a claim.
- thaumasiotes 9y ago
  > Arrow's impossibility theorem is the most famous impossibility result in this area; it says that no algorithm can take in a set of rankings (e.g. match outcomes) and produce an aggregate ranking in a way that satisfies a small set of fairness criteria. Ignoring the parenthetical "e.g. match outcomes", this is a correct description of Arrow's impossibility theorem. I don't see how match outcomes could possibly be an example of a set of rankings in the sense of the theorem, though.
  - bo1024 9y ago
    Yeah, that was unclear. I'm thinking of a match outcome between teams A and B as a partial ordering on all the teams. Classically Arrow's deals with only total orderings as input -- I'd been thinking that it extends to partial orderings, but hmm, I'm not sure what research says about if we restrict the inputs to be just pairwise orderings/outcomes.
da_chicken 9y ago
I always end up going back to Rump's example (https://link.springer.com/article/10.1023/A:1015569431383 https://link.springer.com/article/10.1023/A:1015569431383): > Rump’s example is to compute the expression > ƒ = 333.75 * b^6 + a^2 * (11 * a^2 * b^2 − b^6 − 121 * b^4 − 2) + 5.5 * b^8 + a/(2b) > with a = 77617 and b = 33096. On an IBM S/370 main frame he computed ƒ in (1.1) using single, double, and extended-precision arithmetic, to produce the results: > Single precision: ƒ = 1.172603... > Double precision: ƒ = 1.1726039400531... > Extended precision: ƒ = 1.172603940053178... > This suggests a reliable result of approximately 1.172603 or even 1.1726039400532. In fact, however, the correct result (within one unit of the last digit) is > ƒ = −0. 827396059946821368141165095479816... (1.2) > Even the sign was wrong. It's worth remembering that computers are very fast, but also very, very stupid. They do exactly what you tell them (or teach them, if we're talking about MCL), which isn't always what you want. Just because you've got a very scientific looking algorithm doesn't mean it's correct. Just because you're aggregating all your data doesn't mean the results are truly meaningful.

by rspeer 9y ago

digi_owl 9y ago

Gets me thinking of an Adam Curtis' documentary series: https://en.wikipedia.org/wiki/All_Watched_Over_by_Machines_of_Loving_Grace_%28TV_series%29 https://en.wikipedia.org/wiki/All_Watched_Over_by_Machines_o... Not exactly sure how relevant it is though.

bo1024 9y ago

This might be a case where classical econ ("social choice") can give some helpful perspective. Arrow's impossibility theorem is the most famous impossibility result in this area; it says that no algorithm can take in a set of rankings (e.g. match outcomes) and produce an aggregate ranking in a way that satisfies a small set of fairness criteria. This is classically interpreted as saying that any aggregation method must be "unfair" in one way or another.

YokoZar 9y ago
Arrow's theorem is mathematics, but even citing it is greatly misleading. All Arrow proved was that, sometimes, voting becomes rock-paper-scissors between the three most popular candidates. That isn't "unfair". It especially doesn't mean that all voting systems are equally bad, and yet the most common reason for people to cite Arrow's theorem is to try and dispute the idea that we can do better.
- thaumasiotes 9y ago
  Unlike your parent comment, your comment is total nonsense. Arrow proved that, given a set of ordinal preference rankings held by several individuals, the concept of an aggregate preference ranking describing the overall "will of society" is not well defined; subject to four or five unobjectionable constraints, no function determining such an overall ranking exists. Voting systems which obey the assumptions of the theorem frequently break down in ways that cannot be described as "rock-paper-scissors between the three most popular candidates", as when strategic voting caused the papal conclave of 1334 to unanimously elect the least popular candidate to the papacy. Were you perhaps thinking of Condorcet's work when you referred to rock-paper-scissors? He, not Arrow, famously wrote about how voter preferences were not necessarily transitive.
  - YokoZar 9y ago
    That "rock-paper-scissors" cycle among the top preferences is the Smith set, and every sensible voting "violates" Arrows theorem by saying that sometimes it's ok to acknowledge that cycle. One of Arrows assumptions, "Irrelevance of Independent Alternatives", is far too strong. If you relax it to "Local Independence of Irrelevant Alternatives", then Arrows theorem no longer applies. It's just not a relevant theorem. Claiming that allowing a voting system to recognize a rock-paper-scissors cycle among the top candidates is the same as allowing dictators is obvious rubbish. But if you interpret Arrow's theorem naively, you might be tempted to make such a claim.
thaumasiotes 9y ago
> Arrow's impossibility theorem is the most famous impossibility result in this area; it says that no algorithm can take in a set of rankings (e.g. match outcomes) and produce an aggregate ranking in a way that satisfies a small set of fairness criteria. Ignoring the parenthetical "e.g. match outcomes", this is a correct description of Arrow's impossibility theorem. I don't see how match outcomes could possibly be an example of a set of rankings in the sense of the theorem, though.
- bo1024 9y ago
  Yeah, that was unclear. I'm thinking of a match outcome between teams A and B as a partial ordering on all the teams. Classically Arrow's deals with only total orderings as input -- I'd been thinking that it extends to partial orderings, but hmm, I'm not sure what research says about if we restrict the inputs to be just pairwise orderings/outcomes.

da_chicken 9y ago

I always end up going back to Rump's example (https://link.springer.com/article/10.1023/A:1015569431383 https://link.springer.com/article/10.1023/A:1015569431383): > Rump’s example is to compute the expression > ƒ = 333.75 * b^6 + a^2 * (11 * a^2 * b^2 − b^6 − 121 * b^4 − 2) + 5.5 * b^8 + a/(2b) > with a = 77617 and b = 33096. On an IBM S/370 main frame he computed ƒ in (1.1) using single, double, and extended-precision arithmetic, to produce the results: > Single precision: ƒ = 1.172603... > Double precision: ƒ = 1.1726039400531... > Extended precision: ƒ = 1.172603940053178... > This suggests a reliable result of approximately 1.172603 or even 1.1726039400532. In fact, however, the correct result (within one unit of the last digit) is > ƒ = −0. 827396059946821368141165095479816... (1.2) > Even the sign was wrong. It's worth remembering that computers are very fast, but also very, very stupid. They do exactly what you tell them (or teach them, if we're talking about MCL), which isn't always what you want. Just because you've got a very scientific looking algorithm doesn't mean it's correct. Just because you're aggregating all your data doesn't mean the results are truly meaningful.