3 ms·
All of those require a training step, and that is the same error minimization as curve fitting. Basically they are curve fitting in a high enough dimension. E
by pas 8y ago
All of those require a training step, and that is the same error minimization as curve fitting.
Basically they are curve fitting in a high enough dimension.
Even theory builders can be thought of as curve fitters. The error metric is greedy, the curve is not differentable, but that's it.
That said, of course there is much more to it, than training. (And as we saw with OpenAI's dota bots and DeepMind's AlphaGo Zero, with good feature engineering, adversarial learning setup, beating human performance is only a matter of processing power.)
- deleted 8y ago[deleted]
- YeGoblynQueenne 8y ago>> All of those require a training step, and that is the same error minimization as curve fitting. >> Basically they are curve fitting in a high enough dimension. >> Even theory builders can be thought of as curve fitters. The error metric is greedy, the curve is not differentable, but that's it. I don't understand most of what you are saying and the bits I understand, I don't see how they're true. Could you please explain in a little more detail? For example, how is, say, recursive partitioning, as in decision tree learners, or storing data points for later comparison, as in KNN, "curve fitting in a high enough dimension"? What is the function being fit and where are its parameters? Full disclosure: I study ILP algorithms for my PhD and I have implemented a couple myself. There is nothing like optimisation going on in there, just (inverse) resolution and some ordering of the search space. Most have nothing like an "error metric", greedy or not. And I have never heard anyone describe logic-based learners, including ILP algorithms, as curve fitting before.
- pas 8y agoIf something requires training, it constructs a model, the model can be represented as a probability distribution. I think of that as the curve. ILP (let's say simplex algorithm or other hill climbers) don't require training, but they nevertheless try to optimize the utility (or goal) function (which implicitly defines a metric on the space), which is a hypersurface, which is a curve in an even higher dimension. Since the cardinality of the set of solutions is not one, and usually infinite, picking one solution over the other is optimization. (And - if I'm not mistaken - unless we know that the search space is convex, we don't know if there are better solutions or not.) I don't think anything that doesn't build a model is machine learning. (Decision trees build a model, Bayes classifiers build a model, etc. And I'd say all are curve fitting.) I'm not doing a PhD at all, so I'm not claiming that I'm an authority on each correct terminus technicus for AI/ML/optimization math/CompSci, but to me they fall under the model/curve fitting umbrella.
- deleted 8y ago[deleted]
- YeGoblynQueenne 8y agoThe Simplex algorithm is a linear programming algorithm. ILP is Inductive Logic Progamming, not linear programming. ILP algorithms do not perform linear programming, do not have a utility function and do not use optimisation. What they do is search a space of hypotheses for a hypothesis that is consistent with some background theory and that explains some set of examples. Wikipedia has an introduction: https://en.wikipedia.org/wiki/Inductive_logic_programming https://en.wikipedia.org/wiki/Inductive_logic_programming You will not be able to understand the operation of ILP algorithms if you try to look at them from the point of view of function optimisation, or, indeed "curve fitting". You need a different set of tools, that of logic programming and automated theorem proving- because the process is essentially that of logical deduction. Please let me know if you delve into the subject and have any questions (see my profile for my email address). I'm always happy to support attempts to understand my research subject. The same goes for other machine learning algorithms: if you use the wrong abstractions to understand them, you'll get the wrong idea about them. For instance- KNN doesn't "require training" as per your condition for machine learning. It just memorises all its examples. Is KNN not machine learning? It is- because it gets better the more examples it has. If you stick to thinking of everything as "curve fitting", either you'll misunderstand what KNN does, or you'll start thinking that KNN is not machine learning. Decision tree nodes are all unique - their probability is always 1/n, where n is the number of all nodes in the tree. In other words, their distribution is uniform. If you try to understand decision trees as "a model represented as a probability distribution" you'll find out that every decision tree learner builds the same model. Obviously that's not true, because each decision tree recognises different classes. And so on. When we talk of "curve fitting" what we mean is that we have the function of a line (not necessarily straight) like αx + βy + γz + ... where the variables are the attributes of a set of examples and the constants are unknown and we wish to learn them. These are the "parameters" of the function that are optimised by curve fitting algorithms. We say that a curve is being "fit" because we assume there is some function that we don't know that describes a line whose points are the entire class of entities from where our examples are drawn. And since we don't know what this function is, we instead try to approximate it with a function that "steps" on the few points that we have- or in other words, "fits" those points. In decision trees, KNN, Bayesian inference, etc, etc, there is no line function and therefore no curve that we are trying to fit. You won't find a line in higher dimensions either. Whereas there is probably some curve that can fit the data points given as training examples to the algorithm, the algorithm, lacking a line function, cannot represent that curve. And while you can sometimes visualise the end result as a curve, this is only a convenience and, more importantly, it is not always possible. For example- decision trees again (because they're easy): look around for visualisations of decision trees. You will find tree diagrams aplenty, but not a single line curve. Because you can't really fit a line on a bunch of decision tree nodes. It's a graph. Where's the line? Where are the parameters? You've obviously made an effort to organise what you know about machine learning, the better to understand it. But you're missing lots of details and the result is actually hindering your undersanding. In a sense, by looking at everything from the point of view of "curve fitting" you're missing the forest for the um, trees. My advice is: if you really want to understand machine learning (or at least to sound a bit knowledgeable about it) keep an open mind and don't rush to conclusions. There is much more to machine learning than what you can get from reading a few blogs, a couple of wikipedia pages and a few popular articles in the tech press. Even experts don't yet have a high-level theory of what machine learning is and how to treat different machine learning algorithms in a high-level, unifrom manner, as some well defined class of objects with common characteristics. That should give you a hint about how close your all-encompasing, "it's all curve fitting" definition is to explaining anything.