5 ms·

Rage Against the Algorithms

32 points by vellum 13y ago

tikhonj 13y ago
An especially interesting thing about machine learning algorithms is that even understanding the algorithm itself will often tell you nothing about its biases: the biases often come from the training data. The article talks about this a little bit, but I think it's a very important point for programmers in particular. For one, this means that even an open source algorithm, with the best intentions, might have significantly detrimental biases. This also explains why I personally do not find machine learning satisfying. It's obviously very useful, and I'm not making a judgement about the field, but I just find solving a problem with machine learning to often feel empty. You get a solution, sure, but you get no additional insight on the problem itself. And I'm often far more interested in this insight than in any given problem itself. I certainly find the underlying principles fascinating, but that fascination usually does not translate to whatever field machine learning is used for. Writing a system for identifying cat pictures will teach you quite a bit about "identifying", but not very much about cats. Ultimately, all this just means that you have to pay attention to what your machine learning algorithm is doing right now, even if you understand the algorithm itself really well.
- Qantourisc 13y ago
  You do feel empty, but there are plenty of other things that need fixing :)Or understanding. And often there is no perfect solution, and the solution changes over time. In these cases machine learning is very handy.
- tzs 13y ago
  > An especially interesting thing about machine learning algorithms is that even understanding the algorithm itself will often tell you nothing about its biases A related problem is that many (most?) machine learning algorithms cannot tell you WHY they make the decisions or classifications that they do. Suppose you are a bank, using a neural net to decide on credit applications, and the net turns down an applicant. You are required by law to tell them why they were denied credit. How do you do that for a neural net? I can think of one way: try tweaking their application until it passes, and then derive an explanation from that. For instance, if you find that they would pass if they made $20k/year more income, you tell them they were rejected because their income was too low. I have no idea if that would be sufficient to satisfy the legal requirements, though.
  - eru 13y ago
    Offering them a clear separating criteria in the passing and non-passing portions of the space of all properties considered is definitely a good idea. And, as an explanation / description, it's independent in form of the algorithm used.
  - waps 13y ago
    Well you don't train the algorithm to just give you Yes/No. You tell it to give you a) Yes, SUPER green b) Yes, Green c) mmmmmmmmmmmmmmmmyeah, ok, but watch this guy d) please review manually e) no, too many other loans f) no, but maybe yes, with more collateral g) no, convicted felon h) no, unemployed i) no, your wife has a billion loans j) no, insufficient money down k) ... (there's like 300 or so different reasons in practice) So the inputs to the neural net would be all the information you know about the applicant, appropriately encoded. Then you sprinkle hidden layers to personal taste, and the output layer would be a->i binary neurons.
    - neuralk 13y ago
      True, but I believe the argument still stands for other ML algorithms, such as support vector machines. There is no way to query directly why the SVM maps an instance to a label.
      - alanctgardner2 13y ago
        It seems like in SVM the best way to find out 'how to pass' is to find the shortest path from your point in the feature space to the separating hyperplane. The length of that vector in each axis explains how much you need to change to cross the threshold.
      - waps 13y ago
        How about you run 20 vector machines ? Each only identifies how likely one reason is (using distance from separating plane as a "how sure is this SVM" measure). Besides, SVM do not have a particularly good reputation due to the need for good kernels, which requires either massive computing power (just have 10 million kernels) or people with very good domain knowledge. And if you need people with very good domain knowledge, why bother with AI ? AI is hard and people who know both a lot about statistics, AI and programming AND a lot about, say insurance claims, well good look finding them. People with good knowledge about insurance claims and good knowledge of excel, they're, well, not a dime a dozen, but certainly much easier to find.
- w_t_payne 13y ago
  I think that this is more a criticism of the way that machine learning is commonly done, rather than a valid criticism of machine learning per se. I have certainly (sometimes) been able to build insights using (some) machine learning approaches, but it depends a lot on having the time, the resources, and the right organizational culture in place. Unfortunately, a lot of people approach machine learning algorithms expecting a magic bullet; attracted by the hyperbole, so the self-deception starts even before a line of code is written. Doubly-unfortunately, these people are usually the ones offering the highest salaries for "data scientists" and "big data engineers". Like most work of consequence, the mundane outweighs the exciting by a ratio of at least 10:1. Obtaining and cleaning the data, understanding what is missing, the sampling biases that are present, testing and training the algorithms, all of these things are deeply important and seemingly mundane; yet it is these very activities that help to build deep insight and experience. Perhaps we should pay our data scientists a bit (a lot) less, and we would get fewer flashy presentations and more substance.
- mrcactu5 13y ago
  it sounds like there's no problem with machine learning itself, it's how you are using it. if you're not learning about cats then you need to try a different strategy that deals more directly with them.
- taeric 13y ago
  How is this really any different than solving problems utilizing the raw speed of computers? That is, simply enumerating all possibilities of a problem might not really tell you much more than whether there is an answer, either. However, armed with that some folks may be able to find more. To take your cat example. Now hook up and improve this algorithm in cameras out in the wild and it will become much easier to track where cats are found. Want a rough count of the unique stray cats in an area? Is there a camera that covers it? :) That is to say, sometimes we simply need the data that these algorithms can provide in order to feed other decisions. This is really no different than any other situation where you have to infer a wider truth from limited data. We are just expanding our ability to do so, now. Not necessarily by finding the fundamental equation to something, but by applying simpler equations to more and more data. Right?
- istayhomeallday 13y ago
  This all depends on the algorithm and whether the algorithm is discriminative or generative. Regarding the cat identification task: If the goal is to identify cats from pictures, then what would you expect to learn about cats? The high-level features of a cat that we combine in a hierarchical fashion to arrive at 'cat'? After all, if we are just looking at pictures of cats, I think this is the best a human would be able to ascertain given numerous examples. If so, then algorithms like Convolutional Neural Nets do in fact provide such information - one just needs to look at the convolutional kernels to see these features emerge over the hidden layers. A simpler example is the task of identifying handwritten digits - and this is the example that is almost always referenced when discussing CNNs. A well-trained CNN on pictures of handwritten digits will extract things like diagonal and straight edges as high-level features and then combine those in appropriate spatial orderings to identify the digits 0-9...which is likely similar to how we discriminate between such digits. There are musical analogs as well of using NNs that extract out from a time-frequency representation, for example, individual instrument timbres (or at least decent approximations) in a mixture of instruments sounding simultaneously.
Theriac25 13y ago
> A recent survey found that 76 percent of consumers check online reviews before buying Bullshit.
- jonafato 13y ago
  I normally disagree with this type of comment, but based on the actual survey they refer to [1], you appear to be correct. The most probable way I see that they've come to this number is by using the other side of "24% of consumers never use online reviews". The rest of the data is "27% of consumers regularly use online reviews" with the remaining 49% checking occasionally. The way the statement is worded, I feel that many would read it as "A recent survey found that 76 percent of consumers regularly check online reviews before buying" even if that's not how it's written. A better statement would make the breakdown of regular vs. occasional checkers clear. That said, single word comments don't add much to the conversation. As stated, and if the survey is to be believed, the line is factually correct even if it is misleading. I'm not sure if "bullshit" refers to how you interpreted the statement or the results of the survey. Without clarifying your objection, though, it will likely be ignored by most. [1] http://searchengineland.com/study-72-of-consumers-trust-online-reviews-as-much-as-personal-recommendations-114152 http://searchengineland.com/study-72-of-consumers-trust-onli... Edit: added link to the survey
  - kylebrown 13y ago
    I think Theriac25's "bullshit" was reference to the fact that 86% of statistics are either made up on the spot, or fudged data from non-representative samples.
- solve 13y ago
  What country are you from? I'm very interested in consumer behavior in this area, and have seen massive differences between countries.
zzxxvvcc 13y ago
More transparecy into credit scoring algorithms would be nice.
willvarfar 13y ago
The thing about not auto-correcting abortion is, I speculate, actually because it'd be really unpopular if other misspellings were mis-corrected to read abortion. I think there's likely a short blacklist of words that are in the dictionary but aren't auto-corrected to.

32 points by vellum 13y ago

An especially interesting thing about machine learning algorithms is that even understanding the algorithm itself will often tell you nothing about its biases: the biases often come from the training data. The article talks about this a little bit, but I think it's a very important point for programmers in particular. For one, this means that even an open source algorithm, with the best intentions, might have significantly detrimental biases. This also explains why I personally do not find machine learning satisfying. It's obviously very useful, and I'm not making a judgement about the field, but I just find solving a problem with machine learning to often feel empty. You get a solution, sure, but you get no additional insight on the problem itself. And I'm often far more interested in this insight than in any given problem itself. I certainly find the underlying principles fascinating, but that fascination usually does not translate to whatever field machine learning is used for. Writing a system for identifying cat pictures will teach you quite a bit about "identifying", but not very much about cats. Ultimately, all this just means that you have to pay attention to what your machine learning algorithm is doing right now, even if you understand the algorithm itself really well.

Qantourisc 13y ago
You do feel empty, but there are plenty of other things that need fixing :)Or understanding. And often there is no perfect solution, and the solution changes over time. In these cases machine learning is very handy.
tzs 13y ago
> An especially interesting thing about machine learning algorithms is that even understanding the algorithm itself will often tell you nothing about its biases A related problem is that many (most?) machine learning algorithms cannot tell you WHY they make the decisions or classifications that they do. Suppose you are a bank, using a neural net to decide on credit applications, and the net turns down an applicant. You are required by law to tell them why they were denied credit. How do you do that for a neural net? I can think of one way: try tweaking their application until it passes, and then derive an explanation from that. For instance, if you find that they would pass if they made $20k/year more income, you tell them they were rejected because their income was too low. I have no idea if that would be sufficient to satisfy the legal requirements, though.
- eru 13y ago
  Offering them a clear separating criteria in the passing and non-passing portions of the space of all properties considered is definitely a good idea. And, as an explanation / description, it's independent in form of the algorithm used.
- waps 13y ago
  Well you don't train the algorithm to just give you Yes/No. You tell it to give you a) Yes, SUPER green b) Yes, Green c) mmmmmmmmmmmmmmmmyeah, ok, but watch this guy d) please review manually e) no, too many other loans f) no, but maybe yes, with more collateral g) no, convicted felon h) no, unemployed i) no, your wife has a billion loans j) no, insufficient money down k) ... (there's like 300 or so different reasons in practice) So the inputs to the neural net would be all the information you know about the applicant, appropriately encoded. Then you sprinkle hidden layers to personal taste, and the output layer would be a->i binary neurons.
  - neuralk 13y ago
    True, but I believe the argument still stands for other ML algorithms, such as support vector machines. There is no way to query directly why the SVM maps an instance to a label.
    - alanctgardner2 13y ago
      It seems like in SVM the best way to find out 'how to pass' is to find the shortest path from your point in the feature space to the separating hyperplane. The length of that vector in each axis explains how much you need to change to cross the threshold.
    - waps 13y ago
      How about you run 20 vector machines ? Each only identifies how likely one reason is (using distance from separating plane as a "how sure is this SVM" measure). Besides, SVM do not have a particularly good reputation due to the need for good kernels, which requires either massive computing power (just have 10 million kernels) or people with very good domain knowledge. And if you need people with very good domain knowledge, why bother with AI ? AI is hard and people who know both a lot about statistics, AI and programming AND a lot about, say insurance claims, well good look finding them. People with good knowledge about insurance claims and good knowledge of excel, they're, well, not a dime a dozen, but certainly much easier to find.
w_t_payne 13y ago
I think that this is more a criticism of the way that machine learning is commonly done, rather than a valid criticism of machine learning per se. I have certainly (sometimes) been able to build insights using (some) machine learning approaches, but it depends a lot on having the time, the resources, and the right organizational culture in place. Unfortunately, a lot of people approach machine learning algorithms expecting a magic bullet; attracted by the hyperbole, so the self-deception starts even before a line of code is written. Doubly-unfortunately, these people are usually the ones offering the highest salaries for "data scientists" and "big data engineers". Like most work of consequence, the mundane outweighs the exciting by a ratio of at least 10:1. Obtaining and cleaning the data, understanding what is missing, the sampling biases that are present, testing and training the algorithms, all of these things are deeply important and seemingly mundane; yet it is these very activities that help to build deep insight and experience. Perhaps we should pay our data scientists a bit (a lot) less, and we would get fewer flashy presentations and more substance.
mrcactu5 13y ago
it sounds like there's no problem with machine learning itself, it's how you are using it. if you're not learning about cats then you need to try a different strategy that deals more directly with them.
taeric 13y ago
How is this really any different than solving problems utilizing the raw speed of computers? That is, simply enumerating all possibilities of a problem might not really tell you much more than whether there is an answer, either. However, armed with that some folks may be able to find more. To take your cat example. Now hook up and improve this algorithm in cameras out in the wild and it will become much easier to track where cats are found. Want a rough count of the unique stray cats in an area? Is there a camera that covers it? :) That is to say, sometimes we simply need the data that these algorithms can provide in order to feed other decisions. This is really no different than any other situation where you have to infer a wider truth from limited data. We are just expanding our ability to do so, now. Not necessarily by finding the fundamental equation to something, but by applying simpler equations to more and more data. Right?
istayhomeallday 13y ago
This all depends on the algorithm and whether the algorithm is discriminative or generative. Regarding the cat identification task: If the goal is to identify cats from pictures, then what would you expect to learn about cats? The high-level features of a cat that we combine in a hierarchical fashion to arrive at 'cat'? After all, if we are just looking at pictures of cats, I think this is the best a human would be able to ascertain given numerous examples. If so, then algorithms like Convolutional Neural Nets do in fact provide such information - one just needs to look at the convolutional kernels to see these features emerge over the hidden layers. A simpler example is the task of identifying handwritten digits - and this is the example that is almost always referenced when discussing CNNs. A well-trained CNN on pictures of handwritten digits will extract things like diagonal and straight edges as high-level features and then combine those in appropriate spatial orderings to identify the digits 0-9...which is likely similar to how we discriminate between such digits. There are musical analogs as well of using NNs that extract out from a time-frequency representation, for example, individual instrument timbres (or at least decent approximations) in a mixture of instruments sounding simultaneously.

Theriac25 13y ago

> A recent survey found that 76 percent of consumers check online reviews before buying Bullshit.

jonafato 13y ago
I normally disagree with this type of comment, but based on the actual survey they refer to [1], you appear to be correct. The most probable way I see that they've come to this number is by using the other side of "24% of consumers never use online reviews". The rest of the data is "27% of consumers regularly use online reviews" with the remaining 49% checking occasionally. The way the statement is worded, I feel that many would read it as "A recent survey found that 76 percent of consumers regularly check online reviews before buying" even if that's not how it's written. A better statement would make the breakdown of regular vs. occasional checkers clear. That said, single word comments don't add much to the conversation. As stated, and if the survey is to be believed, the line is factually correct even if it is misleading. I'm not sure if "bullshit" refers to how you interpreted the statement or the results of the survey. Without clarifying your objection, though, it will likely be ignored by most. [1] http://searchengineland.com/study-72-of-consumers-trust-online-reviews-as-much-as-personal-recommendations-114152 http://searchengineland.com/study-72-of-consumers-trust-onli... Edit: added link to the survey
- kylebrown 13y ago
  I think Theriac25's "bullshit" was reference to the fact that 86% of statistics are either made up on the spot, or fudged data from non-representative samples.
solve 13y ago
What country are you from? I'm very interested in consumer behavior in this area, and have seen massive differences between countries.

zzxxvvcc 13y ago

More transparecy into credit scoring algorithms would be nice.

willvarfar 13y ago

The thing about not auto-correcting abortion is, I speculate, actually because it'd be really unpopular if other misspellings were mis-corrected to read abortion. I think there's likely a short blacklist of words that are in the dictionary but aren't auto-corrected to.