8 ms·
A Tour of the Top Algorithms for Machine Learning Newbies
- echelon 8y agoI'm a machine learning newbie, and I'd really like to speak with someone in the field to figure out what I'm wanting to learn. The problem I have is that I have a large labeled data set of spoken words and phonemes from a single speaker. I'd like to train a model and generate new phonemes (of various pitches, speeds, and intonations) with which to build a concatenative speech engine. What algorithms and models would I be looking to use? What are the primary techniques? Is this something I could quickly begin to see results in, or would it take months or years of tweaking? To clarify what I'm doing, I own/built trumped.com, and I'm trying to improve the speech synthesis quality by generating better fitting units of speech.
- probably_wrong 8y agoThe simplest way would be downloading something like MaryTTS [1], read the documentation, and train your own voice model. It won't be perfect, but shouldn't be too hard. The best results would probably be achieved by implementing DeepMind's WaveNet paper [2], but it might be too much for what you need. I'm not really sure what to suggest in between those two. Some kind of convolutional NN, I guess? [1] mary.dfki.de [2] arxiv.org/abs/1609.03499
- PeterisP 8y agoIf you've got a model that can generate new phonemes (of various pitches, speeds, and intonations), then you have a parametric speech synthesis engine and can use it directly as-is instead of strapping a concatenative engine on top. For the techniques, Wavenet and Tacotron (e.g. https://google.github.io/tacotron/publications/tacotron2/ https://google.github.io/tacotron/publications/tacotron2/) seem to be the state of art, but they are reportedly hard to replicate.
- tw1010 8y agoIf you can't explain something in a different way from how you learned it, you probably don't understand it adequately.
- deleted 8y ago[deleted]
- Wiretrip 8y agoNice to see a roundup of ML that doesn't just go straight to Deep Learning for a change :-)
- polkapolka 8y agoThe image for logistic regression is hilariously wrong. It shows the sigmoid as a decision boundary. Also dont get hung up about no free lunch theorem. That is a great result in computer science theory with little practical impact: just pick neural networks for unstructured and GBDT for structured data. For the vast majority of real-life problems (not all possible problems) these are the single best algorithms.
- deleted 8y ago[deleted]
- 1024core 8y ago> just pick neural networks for unstructured data ... except that NNs require a large amount of training data.
- polkapolka 8y agoSure there are constraint and pros and cons, but still: pick a neural network for unstructured data. Can always unsupervised pretrain and fine tune on a tiny dataset.
- swsieber 8y agoWhat qualifies as unstructured data? Would you consider text content to be structured or unstructured? (e.g. for classification of documents)
- polkapolka 8y agoText data traditionally seen as unstructured. Try a simple MLP.
- yorwba 8y agoGeneral-purpose machine learning algorithms require a large amount of training data. There's no magic that can make accurate predictions without any data to base them on. If you don't have enough data, the information needs to come from somewhere else. If you have a large amount of data on a similar problem, you can try transfer learning to learn shared properties and only fine-tune the domain-specific stuff on a smaller data set. If you have no quantitative data, but know domain experts, you can build a custom model based on their advice, with fewer parameters that need to be fit to the data you do have. But if you have so little data that you can't train a neural network, you can't be getting new data very frequently. It might be cheaper to just pay a human to look at it. If you don't even have enough data for humans to work with, fancy machine learning isn't going to help you.
- ur-whale 8y agoNo mention of Deep Learning in a top ML algorithm list in 2018? Kinda odd if you ask me. Also, in the SVM section, no mention of kernel methods? (yet the picture shows a windy boundary). Also odd.
- deleted 8y ago[deleted]
- joe_the_user 8y agoThe first seven algorithms could be defended as "elementary" methods that would help someone work up to neural nets and deep learning. But once he starts talking about SVM, I think he's talking of a method as sophisticated as neural nets and deep learning. Neural nets and SVM were competitors - competitively applicable and competitively difficult - in the aughts. Deep learning has now pulled away but not by the discovery of fundamentally more complicated methods. Rather, the process has involved lot and lots and lots of little refinements, through throwing lots of people, advanced-math intuitions and computing power at it, etc. Learning everything needed to create state-of-the-art results is hard (as far I can tell/guess) but the basics are reasonably simple.
- didibus 8y agoI've been wondering, is nearest neighbour really ML? No part of the logic is learned from the data. It feels just like a glorified lookup table, where the look up is fuzzy to some predefined definition of nearest.
- 5minbreak 8y agok-nearest neighbors classification is one of the first non-linear supervised learning algorithms. Its predictions are derived from the data sample distances. It is basically a glorified fuzzy lookup table, but then again, so can one view deep learning (fuzzy hierarchical localized lookup). Pure memorization can even outperform logistic regression, especially with big data sets, so there is some recent debate as to what degree models memorize and to what degree they generalize.
- didibus 8y agoInteresting, though I don't totally see how deep learning would be similar. On deep learning, it is my understanding the weights are learned from the data. These are effectively constants, and represent logical rules. So in essence, the rules which relates input to output are learned from the data in deep learning. In nearest neighbour, the rule wasn't learned, we figured out the rule ourself: "use the nearest data point's result". But in deep learning, the rule might be something like when feature x and y are between z range of each other and etc. And this rule is not defined by us, but by the weights which are learned from the data. Effectively, deep learning thus learns the rules that define the relationship between input and class. But nearest neighbor is just a static rule that happens to be pretty general in essence, so it gives okay result for a lot of problems. Not an AI expert, so take all this as my simple current understanding.
- ipsa 8y agoYou could automatically encode a KNN model as a set of logical if-then rules: "if x1 > 10 and x2 < 3 then 4 nearest labels are [1, 1, 1, 0]" so the information is there. For KNN you could also train weights for every variable (how much should they count in the distance calculation?). For deep learning you have way more parameters and architecture choices than for nearest neighbors (mostly the distance metric and the number of neighbors to consider). After that, both learn a mapping from input data to a target.
- Balgair 8y ago> The technique (linear discriminate analysis) assumes that the data has a Gaussian distribution (bell curve), so it is a good idea to remove outliers from your data before hand. Um, this seems very fishy to do, right? What am I missing here?
- claytonjy 8y agoDepends on your goal. Outlier removal should be done much more carefully when the goal is inference; trying to test if your hypothesis is true. Here, outlier removal is a super easy way to accidentally p-hack. Current best-practice is to pre-register your analysis, including how you'll define and handle outliers. For predictive goals, where the idea is to predict the class/value of unseen data, outlier removal is often a good way to keep your bias in check and not bias your model towards the outliers. The trade-off is that future outliers will be predicted as though they weren't, e.g. much closer to the mean than they should be. This is what the article is trying to do. There's also a whole wide world of outlier & anomaly detection, where you want to say e.g. "this new data point is probably an outlier".
- deleted 8y ago[deleted]
- PLenz 8y agoThe algo is almost besides the point. The real work is in getting the data, cleaning it, and then explaining WTF is going on to lay people. Im most DS jobs how you get there is irrelevant.