3 ms·
I see I made you feel bad with my advice to read up. I'm sorry, because that was not my intention. However, you really do need to take my advise seriously. You'
by YeGoblynQueenne 6y ago
I see I made you feel bad with my advice to read up. I'm sorry, because that
was not my intention. However, you really do need to take my advise seriously.
You've insisted throughout our conversation that you don't need to read older
machine learning or AI papers because they're not relevant anymore. And yet,
they are. And you do need to read them because without them you will not be
able to understand the recent developments you seem to be intersted in.
Take for instance your example of ViT. This is a transformer, so it's clearly
not an unbiased generaliser that learns only from examples. You say so
yourself: "it's all one inductive bias". Yes, that's how machine learning
works and deep neural nets don't do anything different, neither do they learn
only from examples, as you seemed to suggest in your previous comment (you
replied "That's literally what DL is" to my comment that "you can't learn only
from examples").
But I think you misunderstood my comment about how the biggest advances in
deep neural nets have come from purpose-built architectures. That is not to
say that the same architectures cannot be applied to different domains- but
the state of the art systems are always fine-tuned for specific tasks or
datasets. This hasn't changed recently and it hasn't changed in the last 30
years.
>> For sure, one might call this ‘strong inductive biases’, in that the
program is not random bytes (as a truly bias-free algorithm must be), but
please at least admit that this is a complete different conceptual plane to
the sort of biases you give Louise. Louise's biases aren't merely task
specific, they're problem-specific. It would be one thing if Louise's biases
were a handwritten web of a million BK rules: fine, whatever, as long as it
solves the task that is obviously possible to solve. But they're not, they're
tuned per example.
A truly bias-free algorithm is not "random bytes". It's a learner that
memorises its training examples and can only recognise its training examples.
Hence why it can't generalise. This is in Mitchell's paper which I suggested
you read.
Louise's biases are not problem-specific in the short example I showed
you. I defined BK predicates with wide applicability in programs processing
lists and numbers. There is no such limitation, theoretical or practical, in
the general sense, either. You can give Louise a million irrelevant BK
predicates, if you like, and it will still find the ones it needs to complete
the learning task assuming they're in there somewhere. In fact, it will find
all of the relevant ones - and return the superset of all programs that
solve the task (so you can use it for example to identify interesting
relations in your dataset). Like I say in a previous comment, Louise's
learning algorithm was originally designed to select relevant BK.
Additionally, like I said in an earlier comment, Louise can learn its own
bias, both the BK and the metarules, so it is not only not limited to
task-specific bias, it is not even limited to user-provided bias. Under some
circumstances it can even invent new examples. And then use them to learn a
hypothesis that generalises better to unseen examples.
*
>> Louise can perhaps generate all Prolog programs. Louise cannot search the
space of Prolog programs.
I don't follow. What do you mean?
- Veedrac 6y ago> Louise's biases are not problem-specific in the short example I showed you. This is clearly untrue. You were customizing the BK to each specific task. You were also customizing the stepping stones for each specific task. Justifications can come later. At least admit that you customized the BK for each problem instance and prior to doing so the solver did not solve the problem asked. Not responding to the rest since you've missed my entire point and I don't feel like rephrasing it.
- YeGoblynQueenne 6y agoI did not "customize the BK to each specific task". You can go back and see what I did. I provided some generic BK predicates that manipulate lists and numbers, I defined some metarules and I gave a few examples of each program's inputs and outputs. I don't understand your criticism and I don't understand what you want me to "at least admit". >> This is clearly untrue. Can you show me which biases in the example I showed are problem-specific? >> Not responding to the rest since you've missed my entire point and I don't feel like rephrasing it. I don't think I missed your point. I think you, yourself, are horribly confused about what point you are trying to make. And the reason of course is that you want to be able to express strong opinions about AI and machine learning, but you don't want to have to do the hard work to understand the subject. So you keep saying "five impossible things before breakfast", like asking for a learner that learns only from examples, or saying that's what deep learning is, etc. I'm sorry but despite what the article above suggests, there is't an easy way to being an expert- not even in machine learning. If you want to know what you're talking about, then you'll have to do your homework.
- Veedrac 6y agoAs I said, rather don't reply than reply unpleasantly. I'm cutting this here.
- YeGoblynQueenne 6y agoI don't know why you need to reply unpleasantly. It should be possible to give and receive criticism, even strong criticism, without having it turn into a flamewar just because we're on the internet. Indeed, you yourself have criticised my work and my field mercilessly in this thread and I did not once reply with unpleasantness. In fact, what you keep dismissing as irrelevant and basically cheating (Louise) is my PhD research. I would be well within reason to be defensive about it. Instead, I believe I have remained polite and respectful towards you throughout and strove to answer all your questions about it. Although you did take my criticism as snark, so this is perhaps something that is not entirely objective - you might perceive my criticism as a personal attack, say. Again, this should not be the case. In my field of work, criticism is what makes your work better and without criticism one never improves. So I do mean it when I say that my contribution to this thread was for your sake and to help you improve your knowledge of a subject you seem to be interested in. In any case, I'm sorry this conversation turned sour. I didn't want to make you upset and I apologise for having done so.