6 ms·
I found this really frustrating to read; on 'training your own models': > Deep learning is a class of machine learning that has gotten a lot of well-deserved a
by shadowmint 9y ago
I found this really frustrating to read; on 'training your own models':
> Deep learning is a class of machine learning that has gotten a lot of well-deserved attention in recent years because it's working to solve a wide variety of AI problems in vision, natural language processing, and many others.
What others? Be specific.
The problem is not that people don't know how to build neural networks.
The tensorflow tutorial is extremely comprehensive and there are lots of resources for people who know what problem they're solving to try to build a solution.
The problem is that people don't understand what they can do with this technology, or even, in a meaningful sense, what it is.
Given a array of input numbers [i1, i2, i3...] it provides a black box that can map that input to a set of output numbers [o0, o2, ...] where the dimension of the input and output are different.
That's. All. It. Does.
It takes an input array and outputs an output array; a magic f(x) function.
The thing people don't understand well is not how do I implement f(x); the problem is how do I convert my domain problem (I have 2000000 shelf products in my database and a tonnes of sales information) how to convert that into an input array of numbers? What is the output array of numbers I want out of the function?
The reason this works well for images is because you can say, ok, this output number represents a tag (dog, cat, etc) for each discrete value, and the input numbers are just an array of the raw pixel data.
That's why its good for images; because a traditional function implementation of f(x) that takes a I of dimension say, 65535, and produces an output of dimension 1 is a hard problem.
...but if your problem is, given an input of say [price, date, origin] and an output of [value to business], you can solve that problem quite trivially without needing any neural network.
tldr; The problem people are having is figuring out input and output for domains other than image processing; not how to build a NN, or how to choose which NN to build.
Guide misses the point entirely.
(You might argue this is somewhat uncharitable to what is actually a well written guide to ML; but you can read the about section for yourself and see what you think; I feel like the guide needs to focus sharply on practical business outcomes if it wants to do what it says its doing)
- guscost 9y agoThanks for writing this. It sounds like what I've experienced in this kind of work isn't just accidental. I am working on leveling up math skills to be able to understand the process for TensorFlow etc, but I get the feeling that once it's finally tractable I'll still have the same problems of finding how to transform the inputs into whatever works best for the model, what to output, and how to use the output. This seems to be the part that can't be solved by off-the-shelf packages. And the part where many or most exciting discoveries will be made in the future.
- rs86 9y agoYou might find useful to study algebra. It helps finding mathematical representations. You are in luck if you can find a representation that can be mapped to a vector space.
- guscost 9y agoDefinitely, Linear Algebra is next up, and it is seriously needed.
- lilbobbytables 9y agoWould say Algebra (and Linear Algebra as mentioned below) are the most useful to learn for ML/AI?
- lechiffre10 9y agoI would suggest following Siraj on youtube: https://www.youtube.com/watch?v=N4gDikiec8E https://www.youtube.com/watch?v=N4gDikiec8E really insightful!
- zardo 9y agoI can't take it. He's blasting memes out at 1hz. That's 60 times the acceptable safe limit.
- axiom92 9y ago>...but if you problem is, given an input of say [price, date, origin] and an output of [value to business], you can solve that problem quite trivially without needing any neural network. That's one general statement over another. Consider https://arxiv.org/abs/1508.00021 https://arxiv.org/abs/1508.00021 as a counterexample.
- shadowmint 9y agoYes it is; and you are correct, it's not necessarily trivial; but the point I'm making isn't about reducing very high dimension inputs into lower dimension outputs or picking the best function to map between data sets. ...how you implement those functions, how you optimise for the best results and performance is an implementation detail. If you are picking your solution before you know what your problem is, you are falling into premature optimisation. Understanding how to reduce your problem domain into a series of functions that take numerical data and return numeric data is the key practical problem that most people face, and it's poorly described in the literature. (That's why most people just pull out that classic line '...its great for image recognition and ...many other things...')
- rs86 9y agoIt is an implementation detail if the current implementation is viable, but if it is not, then designing a viable one might be critical
- enraged_camel 9y ago>>The problem is that people don't understand what they can do with this technology, or even, in a meaningful sense, what it is. Yes, exactly. As a "Pull" programmer [1], I can't tell how I would use this technology to solve problems I have or my company has. This makes it difficult for me to get motivated to learn it, despite all the buzz and excitement. I went through the tensorflow tutorial. While it is interesting, I didn't find it useful, the way learning a new programming language or framework or library would be. [1]http://v25media.s3.amazonaws.com/edw519_mod.html#chapter_71 http://v25media.s3.amazonaws.com/edw519_mod.html#chapter_71
- MrMike 9y agoSo true. What's the best reference, guide, or write up you've come across that helps clarify?
- blazespin 9y agoThis is an important insight, however I think more of a research topic. The guide was careful to provide guidance on problems that have a very firm foundation. It didn't want to discuss something that would just drag its audience into the weeds.
- orthoganol 9y ago> tldr; The problem people are having is figuring out input and output for domains other than image processing. This and some of your other comments in this thread lead me to believe you are unaware of how DL is being used in NLP. I would argue real industry use of DL for NLP dwarfs that of DL for images. Personal assistants, search, customer service, sales, chat are the big general cases, let alone company-specific tasks (.e.g. financial data, reviews) or other NLP domains that are not particularly industry-adopted (.e.g. translation, summarization). There's a lot of demand for NLP DL experts, and it's no surprise this year that Socher's NLP DL class had 700 Stanford students enrolled.
- sillysaurus3 9y agoOkay, but you cherry-picked a quote and ignored the central argument, which is that this website doesn't address how to map your problem to the domain of deep learning. Nobody cares how many students the DL class at Stanford had enrolled. Your comment is kind of trying to defend deep learning (when it needs no defense) while putting down the parent without actually providing any meaningful assistance.
- white-flame 9y agoI think that's exactly what he's pointing at: People in general are unaware as to how to map such datasets into NNs. You could help by giving an example of some common application strategies from NLP, instead of just saying that it's done.
- ferdterguson 9y agoTotally agree. As someone who does scientific modeling, fitting models is easy. Finding the model that correctly describes my problem and can be trained with existing or easily acquired input data is hard.
- phreeza 9y agoIn many cases this is true but it is important to note that for deep learning, the training was the hard part that was missing for a long time. Better weight initialization and momentum methods were what really made deep networks work (not just GPUs, as some people tend to believe).
- ska 9y agoGPUs were necessary but not sufficient, true. But the real change (at least for supervised learning) was not in the polishing on training methods (an incremental improvement), it was the availability of big enough data sets. No amount of modeling can make up for insufficient data.
- Eliezer 9y agoTechnical but important quibble. You've described supervised learning: given pairs (X, Y), learn the function F such that Y = F(X). Unsupervised learning takes samples and generates more of the same kind of data. We can see unsupervised learning as being given samples (X) and training a function X = F(noise) that transforms a random vector into an output whose distribution resembles that of the original generating function. Reinforcement learning sees a reward R(X) of policies X and learns to find X such that R(X) is high. All of these have been known to operate on trees or other representations instead of arrays. This matters because there are business problems that are not supervised learning. Reinforcement learning problems especially. Sometimes I can say how well an output is doing and I want an output that does better.
- ska 9y agoIn practice a lot of problems arise because people do not understand the difference in the types of problem that are appropriate for supervised or unsupervised learning, or in other ways trying to wriggle out of having to generate real labels.
- davedx 9y agoSome examples from the AI Grant project (click Spring 2017 finalists): https://aigrant.org/ https://aigrant.org/
- jsemrau 9y agoI could not agree more. There should be more articles like this one (https://hackernoon.com/how-to-better-classify-coachella-with-machine-learning-part-2-27912be5b12 https://hackernoon.com/how-to-better-classify-coachella-with...) clearly outlining the steps what needs to be done to solve a real world problem. The algorithms by themselves are very powerful.
- deleted 9y ago[deleted]
- randcraw 9y ago> Given a array of input numbers [i1, i2, i3...] it provides a black box that can map that input to a set of output numbers [o0, o2, ...] where the dimension of the input and output are different. > That's. All. It. Does. No. That's. WHAT. It. Does. Your emotional overreaction to this site has induced bias leading to a false negative -- mistaking hype for signal. Fact is, using deep learning, newbie grad students have routinely and resoundingly outperformed nearly all the state-of-the-art techniques in signal processing in less than FIVE years of trying. Image analysis and speech recognition have been revolutionized; 40+ years of R&D expertise has been largely supplanted in the blink of an eye. Dismiss this tsunami at your own peril. Does DL scale to harder problems? Why yes, it does. No computer implementation of the game Go had outperformed the best humans until deep learning was used, then it beat the best humans almost immediately (unlike chess which took decades to reach that level using traditional AI methods). I have little doubt that significant achievements await DL's application to more complex tasks like language analysis too. To believe otherwise would be severely short-sighted. Does deep learning also solve trivial pattern rec problems like regression or curve fitting as efficiently as some linear ML models? No, but who cares? By that reasoning, we should dismiss Einstein because many 12 year old children can multiply numbers faster and for less money.
- sillysaurus3 9y agoThe parent didn't react emotionally and didn't even attack deep learning as a subject. He said deep learning gurus are failing to make it understandable. Here's someone else who cherry-picked a quote and ignored the central argument: https://news.ycombinator.com/item?id=14347068 https://news.ycombinator.com/item?id=14347068 Almost every one of your sentences are talking past the parent. If you want to be argumentative instead of helping the parent as he asked, then it might be best to find actual quotes to disagree with.
- sjg007 9y agoYeah I find that a lot of experts have a very difficult time trying to explain things in simple terms. But maybe they haven't been asked to.
- 9y ago
- thousandautumns 9y agoThe dimension of the input and output arrays needn't be different, right? There's no reason you can't input a image into a convolutional neural network and output an altered image of the same size.
- halflings 9y ago> The reason this works well for images is because you can say, ok, this output number represents a tag (dog, cat, etc) for each discrete value, and the input numbers are just an array of the raw pixel data. > because a traditional function implementation of f(x) that takes a I of dimension say, 65535, and produces an output of dimension 1 is a hard problem Nitpick maybe, but you do not represent each class by a number. This would create a bias to consider subsequent categories as similar (why would aardvark=0 be more similar to antilope=1 than cat=2?). Instead you have one dimension per possible-value (aka one-hot encoding) and the value in that dimension represents how likely the photo belongs to that class (with values summing to 1, softmax, if it's a one-class problem, or no such guarantee if it's a multiclass problem).
- smefi 9y agoCouldn't say it better. Totally agree.
- strin 9y agoThe ultimate goal of deep learning is to remove the need to transform input. The traditional computer vision pipeline works like this: you preprocess the image, featurize it with features like SIFT, and then run an SVM classifier. However, this pipeline is totally beaten by end-to-end neural networks. Same will hold true for other domains. As long as we've discovered the right architecture, aka. the family for f(x), learning still boils down to training your models.