4 ms·

Would it be easier to swallow if it were called clustering instead of analogy? Simply put, logic doesn't scale. Unless you've already solved language, you can't

Would it be easier to swallow if it were called clustering instead of analogy? Simply put, logic doesn't scale. Unless you've already solved language, you can't have sufficiently detailed and accurate logic to be useful. But clustering does scale, at least this limited version of clustering.

YeGoblynQueenne 11y ago
>> logic doesn't scale You have to try and remember that our current computer circuits are (boolean) logic, implemented in hardware. In fact, there's a very good reason why that is so. Turing, and the logicians of Russle's and Goedel's generation before him, conceived of computers as logic machines that reasoned like humans. In Turing's time, computers with logic circuits were even proposed as artificial intelligence. And you know what? For their time, they certainly were. Yet, somehow, a couple of generations later we're ready to do without logic. Except, that's what our computers run on. But who knows, maybe quantum computers will run not on logic circuits but on hardware neural net implementations. It will take petabytes to train a "helo world" program and you'll have to wait for a couple of hours for it to learn, but it will be worth the pain, just to get rid of that blasted old logic that "doesn't scale".
- ThomPete 11y ago
  Our computers run on transistors being turned on or off. Thats the premise that creates the rest and allow us to use it for boolean logical circuits which again allow os to simulate almost anything. The logical part is our interpretation of phenomena not necessarily "out there". We are pattern recognizing feedback loops the rest is interpretation IMO.
- laxatives 11y ago
  In this context, I'm referring to logic as a knowledge base or some similar. Either this information has to be manually curated (which doesn't scale), have huge gaps, or have loads of false positives (ie incorrect information). No one is arguing whether we should or shouldn't be using programming languages or hardware based in logic.
  - YeGoblynQueenne 11y ago
    Alright, thank you for clarifying this. I guess I got carried away a bit last night and allowed this to escalate to a sort of unproductive us-vs-them kind of thing. Sorry about that. It wasn't my intention and some of my replies from yesterday are a bit shit. Now- >> this information has to be manually curated (which doesn't scale) Aye, that's exactly right. I know this first-hand. I study machine learning, so I'm not really-really against it (it works, so). My hunch though is that although we've made a lot of progress with what Hinton calls "analogy" (and what I think he means to call "approximation") we haven't gone _that_ far and every time we made some progress we've eventually hit a wall. Take language for instance: first there was the Bag of Words model, language as independent words. That worked for a while, but eventually it hit a wall. Now we got word embeddings. It gives great results and yes, you can do arithmetic with words, which is major, but then we still haven't made real progress. And note that the top-rated NLP system currently, Watson (Schoher's grad student obviously didn't make something as performant as that) combines statistical learning with knowledge engineering. They use a statistical parser to fill in frame [1] slots. Each approach handles different aspects of the problem. Both together cover more of the solution than each on its own can ever hope to. If you want to do really well, you need to pick the best of both worlds, approximation and knowledge, together. [1] https://en.wikipedia.org/wiki/Frame_%28artificial_intelligence%29 https://en.wikipedia.org/wiki/Frame_%28artificial_intelligen...
YeGoblynQueenne 11y ago
>> clustering does scale I was reading Lin & Wu [1] today; have a look in there for this bit: to create 3000 clusters among 20 million phrases using 3-word windows, each K-Means iteration takes about 20 minutes on 1000 CPUs. Without using the indexing technique in Section 2.4, each iteration takes about 4 times as long. So, sure, clustering scales... if you have a um, cluster, of 1000 CPUs lying around to spread it over. You're saying something fundamentally wrong here. There are classically efficient algorithms (linear and sub-linear in time) for first order logic with search, as in unification, RETE etc, whereas all the machine learning algorithms are typically starting at quadratic times and then increase in crazy, mad, insane rates of complexity that would never be accepted in any other field. It's machine learning that "doesn't scale", not logic. [1] http://www.aclweb.org/anthology/P09-1116 http://www.aclweb.org/anthology/P09-1116

YeGoblynQueenne 11y ago

>> logic doesn't scale You have to try and remember that our current computer circuits are (boolean) logic, implemented in hardware. In fact, there's a very good reason why that is so. Turing, and the logicians of Russle's and Goedel's generation before him, conceived of computers as logic machines that reasoned like humans. In Turing's time, computers with logic circuits were even proposed as artificial intelligence. And you know what? For their time, they certainly were. Yet, somehow, a couple of generations later we're ready to do without logic. Except, that's what our computers run on. But who knows, maybe quantum computers will run not on logic circuits but on hardware neural net implementations. It will take petabytes to train a "helo world" program and you'll have to wait for a couple of hours for it to learn, but it will be worth the pain, just to get rid of that blasted old logic that "doesn't scale".

ThomPete 11y ago
Our computers run on transistors being turned on or off. Thats the premise that creates the rest and allow us to use it for boolean logical circuits which again allow os to simulate almost anything. The logical part is our interpretation of phenomena not necessarily "out there". We are pattern recognizing feedback loops the rest is interpretation IMO.
laxatives 11y ago
In this context, I'm referring to logic as a knowledge base or some similar. Either this information has to be manually curated (which doesn't scale), have huge gaps, or have loads of false positives (ie incorrect information). No one is arguing whether we should or shouldn't be using programming languages or hardware based in logic.
- YeGoblynQueenne 11y ago
  Alright, thank you for clarifying this. I guess I got carried away a bit last night and allowed this to escalate to a sort of unproductive us-vs-them kind of thing. Sorry about that. It wasn't my intention and some of my replies from yesterday are a bit shit. Now- >> this information has to be manually curated (which doesn't scale) Aye, that's exactly right. I know this first-hand. I study machine learning, so I'm not really-really against it (it works, so). My hunch though is that although we've made a lot of progress with what Hinton calls "analogy" (and what I think he means to call "approximation") we haven't gone _that_ far and every time we made some progress we've eventually hit a wall. Take language for instance: first there was the Bag of Words model, language as independent words. That worked for a while, but eventually it hit a wall. Now we got word embeddings. It gives great results and yes, you can do arithmetic with words, which is major, but then we still haven't made real progress. And note that the top-rated NLP system currently, Watson (Schoher's grad student obviously didn't make something as performant as that) combines statistical learning with knowledge engineering. They use a statistical parser to fill in frame [1] slots. Each approach handles different aspects of the problem. Both together cover more of the solution than each on its own can ever hope to. If you want to do really well, you need to pick the best of both worlds, approximation and knowledge, together. [1] https://en.wikipedia.org/wiki/Frame_%28artificial_intelligence%29 https://en.wikipedia.org/wiki/Frame_%28artificial_intelligen...

YeGoblynQueenne 11y ago

>> clustering does scale I was reading Lin & Wu [1] today; have a look in there for this bit: to create 3000 clusters among 20 million phrases using 3-word windows, each K-Means iteration takes about 20 minutes on 1000 CPUs. Without using the indexing technique in Section 2.4, each iteration takes about 4 times as long. So, sure, clustering scales... if you have a um, cluster, of 1000 CPUs lying around to spread it over. You're saying something fundamentally wrong here. There are classically efficient algorithms (linear and sub-linear in time) for first order logic with search, as in unification, RETE etc, whereas all the machine learning algorithms are typically starting at quadratic times and then increase in crazy, mad, insane rates of complexity that would never be accepted in any other field. It's machine learning that "doesn't scale", not logic. [1] http://www.aclweb.org/anthology/P09-1116 http://www.aclweb.org/anthology/P09-1116