3 ms·
> too little data > very broken paradigm Well, that first mention makes it evident that the context is "Big Data driven Machine Learning", and this paradigm l
by mdp2021 4y ago
> too little data
> very broken paradigm
Well, that first mention makes it evident that the context is "Big Data driven Machine Learning", and this paradigm linked to BDDML is not the "intended ML" - it is just a passage in our toying. Because it is also very clear to us (well¹) that some proper learning has "one shot" properties, so at least until single information instances are determinant in model building (as opposed to slightly retouching weights) like in symbolic communication and more, we will know that something is amiss, that we are still crossing the river's fords. But the preparation of the environment to enable e.g. "one shot learning" is the current problem in such direction.
(¹even though some seem as if so inebriated with the current toys to claim "that's it, done, it rolls, TheBall 2.0, it's animated, it lives, we have created the Golem ... Look, a gramophone, it speaks, ask him for advice and serve him food")
This given, it is not that the "toying" has not produced noteworthy aids in real life application.
> I find it extremely concerning
That concern is that we are in need of enhanced human learning - we have more and more complexity to manage -, yet the effect and promise from these areas swing from boost to complacency. Some ask, "where is the problem in the "transparency problem", in blackbox responses": in that we do require more understanding, the human part is that which should primarily seek refinement.
- YeGoblynQueenne 4y agoWell, since we're going a little mad with speculation in this thread, I have to point out that true one-shot learning (as opposed to "one-billion-plus-one-shot") works just fine, but only in the symbolic machine learning paradigm. For example, see: https://github.com/stassa/louise#capabilities https://github.com/stassa/louise#capabilities In particular the second example listed there. A trivial example, but one that cannot be reproduced by current approaches without big-data pre-training. I bring up the 80/20% training/test split that is standard in machine learning because I remember an interaction with my supervisor at the start of my PhD. In one of our meetings my supervisor asked me about the details of some experiments I was running, with a system called metagol (linked from the Louise repository above). En passant, I mentioned that I was training with a 20/80% training/test split and my supervisor stopped me to ask me if I thought that was a standard setup for machine learning. Thinking he meant the splitting of my training data to training and testing partitions I answered, a bit bemused, that yes, of course, that's the standard thing. To which my supervisor laughed and replied "I don't think so". Later of course I realised that he meant that the done thing in machine learning is to use most of the data for training and leave as little as possible for testing. In Inductive Logic Programming, it's typically the other way around, and the training sets are often a few examples, like a dozen or so. Of course our systems don't do the spectacular, impressive things that deep learning systems do, but then again deep learning systems aren't anywhere near ILP approaches for sample complexity and robust generalisation from very few examples (as in one). And of course we don't have a dozen thousand graduates racing to out-do each other with new feats of engineering. Which is a bit of a shame because I think that if we had a couple more hundred people working on ILP, we 'd make huge progress in applications, as well as in understanding of machine learning in general. And the resulting systems would address many of the current concerns (mine anyway) about the use of machine learning in science (e.g. no more black boxes; "ultra-strong machine learning" in Donald Michie's sense of a system that improves the performance of its user; etc). Oh well. It's probably all for the best. Who wants to build genuinely useful and intelligent systems anyway?
- mdp2021 4y ago> Who wants to build genuinely useful and intelligent systems anyway? I do :) We need all the intelligence we can squeeze in this historical phase (where, among the rest, some conditions enabled an influent propagation of the opposite). And: I will never be able to read the whole of even just my personal library. I dream of a "son" with the ability to do it... :)