7 ms·
Top Algorithms in Data Mining (2008) [pdf]
- eterpstra 13y agoIs it just me, or was anyone else taken aback by an academic article using a linkbait-esque title? I'm sure the title is perfectly reasonable, given that it's a survey paper, but the past few years of being bombarded with articles like "10 weird tricks to optimize your everything" immediately triggers my skeptical meter. Funny how the internet warps the brain.
- minimaxir 13y agoUnlike most Top 10 lists that are deliberately designed with unexpected choices for shock value, the paper cites exactly how the 10 were chosen and attempts to justify why each choice was made. If the paper was titled "Wow, The Top 10 Algorithms in Data Mining Are Incredible, And I Can't Believe Where Naive Bayes Placed," then yeah, that might be link-baity.
- Houshalter 13y agoStatistics professors HATE him! Doctor's discovery revealed the secret to learning any problem with just 10 training samples. Find a separating hyperplane with this One Weird Trick. http://www.oneweirdkerneltrick.com/ http://www.oneweirdkerneltrick.com/
- spiffyman 13y agoMaybe I'm being selective about my memory, but I don't remember the awful linkbait titles being nearly so popular in 2007 when this was written. I don't think there's even the hint of imitation of Buzzfeed-style titling here. Also, I've taken a class from one of the authors, and I'd be surprised if he'd ever even seen those kinds of pages. :)
- Anon84 13y agoThis article was eventually expanded into a full fledged book with more in depth descriptions of each algorithm: http://www.crcpress.com/product/isbn/9781420089646 http://www.crcpress.com/product/isbn/9781420089646
- tdaltonc 13y agoI'm surprised that there aren't any PCA/Feature-Extraction/Dimension-Reduce algos on this list. And there's also a really strong emphasis on clustering and classification. Is this an artifact of how the list was put together or is clustering really that much more common than feature extraction? Is PCA thought to be too trivial to qualify?
- micro_cam 13y agoPageRank is a similar eigenvalue based algorithm and I think PCA can be seen as a relaxation of k-means so they sort of did. But yes, if i wrote the list it eigenvector based methods would be more prominent. Also If I wrote it the tree based methods would have been replaced with random forests which are similar to CART but use the crucial ideas of sample bagging and random feature subsets which have proven themselves over and over. I think it may just be out of date.
- jessaustin 13y agoAny suggestions for something more recent?
- micro_cam 13y agoThe machine learning reddit recently started a faq: http://www.reddit.com/r/MachineLearning/comments/20i0vi/meta_collection_of_links_for_beginners_faq/ http://www.reddit.com/r/MachineLearning/comments/20i0vi/meta... They also have a good selection of articles though they tend to be somewhat deep learning/neural network focused. These algorithms have seen a big surge in interest and success of late but may not be as applicable to "data mining" (as opposed to say image or speech recognition).
- aet 13y agoI think PCA is better understood in theorem-proof style of mathematics rather than algorithmically. For example, PCA can be understood as an optimization problem. Whereas CART and k-NN and the likes are best understood algorithmically. I know k-NN was introduced as an algorithm and only later figured out in terms of probability, etc..
- hydralist 13y agoi understand some of these words
- mindslight 13y agoQ: What's the difference between working on data mining and designing missiles? A: When you build missiles, you can at least take comfort knowing that they'll only be used to harm people who are far away.
- Houshalter 13y agoWhat are you talking about?
- philovivero 13y agoHe's on about the problem of data mining being used by NSA and corporate America to do Bad Things. And what are you on about? I'm guessing you're on about the "tool is only as good or evil as the wielder." Trying to take this back out of the meta, our society needs to figure out how to stop hurting each other. I think a good dose of critical thinking skills for our youngsters would go a good distance toward helping that.
- Houshalter 13y agoI guess that's sort of like saying computers are evil because they are used by the NSA. Is there even any evidence that they do use it for anything significant? Let alone that this guys work is contributing to it.
- mindslight 13y agoComputers have many applications, while data mining is generally used with working sets consisting of information about large groups of people - either to get a strategic leg up on them (say, price discrimination or advertising susceptibility), or to single out deviant individuals. I don't think the entire field is strictly a universal evil per se, but with the current climate of centralization I think any good that it can bring is massively dwarfed by the bad. Similarly, missiles themselves aren't a universal bad as they can be used for self defense, but in the current world basically their entire purpose is to preserve empires.
- joeblau 13y agoWow this paper is great. Back in November, I implemented the "Trend or No Trend: A Novel Nonparametric Method for Classifying Time Series"[1] paper in node.js and then I hooked it up to Twitters API. The algorithm was able to find trending topics pretty quickly and accurately. Does anyone know if the source for these algorithms is implemented anywhere? [1] - http://dspace.mit.edu/bitstream/handle/1721.1/85399/870304955.pdf http://dspace.mit.edu/bitstream/handle/1721.1/85399/87030495...
- mendicantB 13y agoThese algorithms are implemented in just about every major language, as they are standard tools. I'd say your best bet for leveraging them is python scikit learn (super good documentation)
- d0vs 13y agoIs is that famous algorithm that's able to predict future Twitter trending topics pretty accurately? Is your implementation opensource?
- KhalilK 13y agoIt was published in 2007 not 2008 ;)
- graycat 13y agoOkay, given a box of data, shove it into one of these algorithms, let the computer run, and now what? What do we have? E.g., for some 'classifier', what the heck do we have? That is, what guarantees do we have? That is, is there something specific and guaranteed that the algorithm has actually done? I know; I know; in some vague, unspecific sense, that seems to work in some practical cases, the algorithm will 'classify'. Still, we don't have precise description of (1) in what cases does the algorithm work and (2) how well. E.g., what is the rate (probability) of false classifications? Sure, we'd need some hypotheses to say. Okay, what the heck are the algorithms assuming for hypotheses that would let there even be a rate of false classifications. Then with those hypotheses, what are the probabilities of a false classification? Without such considerations, about all we have is some code that runs. Uh, where's the beef?
- mikkom 13y agoyou could start with Google and with terms: - K-fold - Confusion matrix
- graycat 13y agoOkay: Googled K-fold. It's a case of resampling. To justify it, use a finite group of measure preserving transformations, which I figured out and published about 16 years ago. But we still have not made clear what the heck the assumptions are. Confusion matrix: Wikipedia has nothing on it, but Matlab's documentation does. But there's a problem: Suppose we have some data on cats and dogs and want to have a 'classifier'. So, we do what Matlab said: Take half of the training data and build a classifier. Then we test the classifier with the other half of the data. But suppose the classifier finds three classifications instead of just the two, dogs and cats we already know exist. So, if we apply such methods where we don't know in advance the number of classifications we might get three classes and be quite wrong and not know it. Again, the methodology looks like just pushing some data into some software with too little in assumptions up front and too little about what we are getting out on the back side.
- mikkom 13y ago
- KhalilK 13y agoThese algorithms are all good, but they seems to be missing recurrent ANN, that is, the form of datamining tool our brains are using. I can add this paper as relevant for those being interested http://orre.neurologic.org/pubdoc/abstract/complex2003-abstract.html http://orre.neurologic.org/pubdoc/abstract/complex2003-abstr...
- tarlar 13y agoThe actual link is here http://orre.neurologic.org/pubdoc/abstract/complex2003-abstract.html http://orre.neurologic.org/pubdoc/abstract/complex2003-abstr...
- deleted 13y ago[deleted]
- maldeh 13y agoKevin Murphy's treatment of Kernel Methods in "Machine Learning - A Probabilistic Perspective" offers a take on SVMs that I don't often find in most discussions. He makes the case that SVMs are generally rather unusual from a Bayesian point of view. Apropos to other sparse vector machines, Murphy opines: "Note that SVMs are very unnatural from a probabilistic point of view. First, they encode sparsity in the loss function rather than the prior. Second, they encode kernels by using an algorithmic trick, rather then being an explicit part of the model. Finally, SVMs do not result in probabilistic outputs, which causes various difficulties, especially in the multi-class classification setting. "It is possible to obtain sparse, probabilistic, multi-class kernel-based classifiers, which work as well or better than SVMs, using techniques such as L1VM or RVM... However, we include a discussion of SVMs, despite their non-probabilistic nature, for two main reasons. First, they are very popular and widely used, so all students of machine learning should know about them. Second, they have some computational advantages over probabilistic methods in the structured output case... [specifically, SSVMs] ... "The conclusion is as follows: if speed matters, use an RVM, but if well-calibrated probabilistic output matters (e.g., for active learning or control problems), use a GP. The only circumstances under which using an SVM seems sensible is the structured output case, where likelihood-based methods can be slow. (We attribute the enormous popularity of SVMs not to their superiority, but to ignorance of the alternatives, and also to the lack of high quality software implementing the alternatives.)" ---- the tl;dr is that SVMs are an important tool for their historical impact and popularity, but the modern data scientist has a range of cookie-cutter algorithms to choose from that could possibly be much more useful.