3 ms·
Formalisations do break down in the limit, Wittgenstein showed everyone that, but the denial is a deep and powerful choice "he must transcend these propositions
by sgt101 8y ago
Formalisations do break down in the limit, Wittgenstein showed everyone that, but the denial is a deep and powerful choice "he must transcend these propositions and then he will see the world alright."
There were engine room troubles for symbolic AI, committed choice meant that reasoners were unpredictable and impractical. We saw 100k improvement of compute change sub symbolic AI, and it has done the same for symbolic, and there have been some really smashing advances in the algorithms. Answer sets explode all solutions and then eliminate contradictions, we couldn't have dealt with them in the 90's (much less in 1985) as the idea of having 10G data structures in memory would have made people lol... but now so what?
Probabilistic formalisms and MCMC have changed what we can write down and reason about effectively as well.
I was very interested to read the story a few days ago about the relationship between compute resource and results in deep learning. https://blog.openai.com/ai-and-compute/ https://blog.openai.com/ai-and-compute/? I was more interested to read the HN reaction! My team has had lots of success creating models using our cluster (in 2014) and then a cluster of GPU's which we bought because we could see that this was the way to go. We haven't gone and spent 100x more since then, and yet we now can see plotted out that the leading teams are doing this. The hardware has not improved at a rate of doubling every 3.4mths, it's hardly improved at all for 12 mths. What's happening is that people are spending money to make waves.
This is denial big time. Brute force is a wall, you can climb a few bricks, but the top is unobtainable. There have been some improvements in tree search and there have been some insights in terms of where we are wasting time with optimisations that aren't working but I've always thought that DL is brute force at heart and I think we are where we are with it now, and Alphazero is where we are going to be until someone really sees a new way.
- YeGoblynQueenne 8y ago>> I was very interested to read the story a few days ago about the relationship between compute resource and results in deep learning. https://blog.openai.com/ai-and-compute/ https://blog.openai.com/ai-and-compute/? Aye. My reaction to that graph was that this rate of progress can't be sustainable. If industry was throwing good money after bad to find out for how long you can keep ussing bubblesort given more and more compute, before having to develop a better sorting algorithm- that's what it would look like. >> Answer sets explode all solutions and then eliminate contradictions, we couldn't have dealt with them in the 90's (much less in 1985) as the idea of having 10G data structures in memory would have made people lol... but now so what? ASP is interesting. There are symbolic learning techniques that learn answer set programs from data, did you know that?
- sgt101 8y agoI didn't know that anyone had done that for answersets, could you share the reference? But on the other hand Stephen Muggleton has been developing systems to learn parts of logic programs for a long time, also I wrote a genetic programming system to learn prolog clauses but that was vs a hand coded objective function rather than vs. data.
- YeGoblynQueenne 8y agoTo be honest, I'm not terribly familiar with the bibliography on learning AS programs, but you could look for work on ASPAL, ILASP and ILASP2 by Alessandra Russo, Mark Law and Krysia Broda. The following are relevant, but not exhaustive and probably not the first papers discussing this but you can follow the refs :) Learning Through Hypothesis Refinement Using Answer Set Programming https://link.springer.com/content/pdf/10.1007%2F978-3-662-44923-3_3.pdf https://link.springer.com/content/pdf/10.1007%2F978-3-662-44... Iterative Learning of Answer Set Programs from Context Dependent Examples https://arxiv.org/pdf/1608.01946v1.pdf https://arxiv.org/pdf/1608.01946v1.pdf Learning Weak Constraints in Answer Set Programming https://spiral.imperial.ac.uk/bitstream/10044/1/33615/8/AcceptedVersion.pdf https://spiral.imperial.ac.uk/bitstream/10044/1/33615/8/Acce... Finally, this is from Mark Law's portion in the Logic-based Machine Learning course at Imperial College London: https://www.doc.ic.ac.uk/~ml1909/teaching/Non-monotonic%20Logic-based%20Learning,%20Summary.pdf https://www.doc.ic.ac.uk/~ml1909/teaching/Non-monotonic%20Lo... Full disclosure: I'm a Phd student of Stephen Muggleton's. If you're interested in where ILP is nowadays, our group is currently working on Meta-Interpretive Learning, a recent advance that learns complete logic programs, with recursion and predicate invention from relational data: https://github.com/metagol/metagol https://github.com/metagol/metagol ... and I'm very excited about it :)
- sgt101 8y agoOh super - would you have any time to show it to me if I toddled over to Imperial at some point ?
- 8y ago