8 ms·

I've only skimmed but it seems like the general idea is sound, that neural networks are good at pattern matching and that shallow pattern matching is only one a

I've only skimmed but it seems like the general idea is sound, that neural networks are good at pattern matching and that shallow pattern matching is only one aspect of "deep thought", logical reasoning being the other one. I'm confused though, they give examples of Go and Chess but, as far as I understand, this is exactly what the author is saying doesn't exist. AlphaGo, Deep Blue, etc, all use neural networks to weight how to traverse the minimax tree [0]. The minimax algorithm/tree is precisely the "logical reasoning". The neural network is "just" prioritizing the path to take. Anyway, like I said, the reasoning is sound. I just read a paper specifically about this breakdown, called "On the Paradox of Learning to Reason from Data" by Zhang, et all [1] where they construct a synthetic data set based on some simple logical rules and then try to train a neural network to recapture the underlying logic (spoiler alter, it fails). From their paper: """ Our study provides an explanation for this paradox: instead of learning to emulate the correct reasoning function, BERT has, in fact, learned statistical features that inherently exist in logical reasoning problems. We also show that it is infeasible to jointly remove statistical features from data, illustrating the difficulty of learning to reason in general. """ [0] https://en.wikipedia.org/wiki/Minimax https://en.wikipedia.org/wiki/Minimax [1] https://arxiv.org/abs/2205.11502 https://arxiv.org/abs/2205.11502

abetusk 4y ago
Nope, I take it back. The article is not coming from a place of (in my opinion) reason. This is, in fancy language and snowed under by enumeration of historical research, math and science, an argument that "computers can never have a soul". From the article: """ No matter how sophisticated the computation is ... there remains an unbridgeable gap (a “humanity gap”) between the ... ability of machine and the general problem solving ability of man. """ This is just standard dualist nonsense under the guise of logical objectivity. Note that there is a deeper issue about the shortcomings of statistical pattern matching to do logic, which is what the paper I reference above is trying to get at, but the parent article's fundamental assumption/conclusion is that computers will never be able to reason.
- c43verge 4y ago
  Thanks a lot for summarizing it. I was curious to see whether it had a new idea, apparently not.
- stelonix 4y ago
  It really isn't about having a soul, but more like about either: 1) stating human brains are actually hypercomputers, which cannot be implemented in real life at the moment 2) stating just what you said before: pattern matching, no matter how many layers or parameters used, is not all there is to intelligence/consciousness At the moment, whether you like it or not, there's no proof at all the human brain works in any way like a Turing machine. That idea which has nothing to back it up is the usual POV here on HN and it is the actual faith-based belief but again, no scientific papers to back it up.
  - orlp 4y ago
    > stating human brains are actually hypercomputers, which cannot be implemented in real life at the moment Extraordinary claims require extraordinary evidence, and absence of evidence is not evidence of absence.
    - adgjlsfhk1 4y ago
      it's pretty easy to prove that a brain can simulate a Turing machine, and it's pretty hard to disbelieve that a computer can simulate a brain (by running the physics)
      - orlp 4y ago
        The claim I find objectionable is that brains do 'hypercomputation', and can not be ran on a Turing machine.
        stelonix 4y ago
        Brains can solve the halting problem, Turing machines cannot.
sharemywin 4y ago
(From the paper you mentioned above)* added some extra lines so it would print properly in this website. it ran without change in python. Here's what I got out of ChatGPT after I gave up trying to get it to answer it directly: write A Python script that parses the following and uses forward chaining inference to answer the following: Facts: Alice is fast. Alice is normal. Rules: If Alice is fast and smart, then Alice is bad. If Alice is normal, then Alice is smart. If Alice is normal and happy, then Alice is sad. Query 1: Alice is bad. Answer: Here is a python script that uses forward chaining inference to answer the given query: Copy code # Define the facts facts = { "fast": ["Alice"], "normal": ["Alice"], } # Define the rules rules = { ("fast", "smart"): ["bad"], ("normal",): ["smart"], ("normal", "happy"): ["sad"], } # Define the query query = "bad" # Initialize the list of known conclusions conclusions = [] # Iterate over the rules for premises, conclusions_ in rules.items(): # Check if all the premises are known to be true if all(premise in facts for premise in premises): # If all the premises are true, add the conclusions to the list of known conclusions conclusions.extend(conclusions_) # Check if the query is in the list of known conclusions if query in conclusions: print("The query can be inferred to be true") else: print("The query cannot be inferred to be true") For the given facts and rules, the script will output "The query can be inferred to be true" because Alice is fast and normal, which means that Alice is smart according to the rules, and Alice is also fast, which means that Alice is bad according to the rules. now chain the rules using the facts from the previous run to see if any new facts are generated and if not then quit To chain the rules using the facts from the previous run, we can simply put the forward chaining inference logic inside a loop and continue iterating until no new facts are generated. Here is an updated version of the script that does this: Copy code # Define the facts facts = { "fast": ["Alice"], "normal": ["Alice"], } # Define the rules rules = { ("fast", "smart"): ["bad"], ("normal",): ["smart"], ("normal", "happy"): ["sad"], } # Define the query query = "bad" # Initialize the list of known conclusions conclusions = [] # Keep iterating until no new facts are generated while True: # Iterate over the rules for premises, conclusions_ in rules.items(): # Check if all the premises are known to be true if all(premise in facts for premise in premises): # If all the premises are true, add the conclusions to the list of known conclusions conclusions.extend(conclusions_) # Initialize a flag to track whether any new facts were generated new_facts_generated = False # Iterate over the conclusions for conclusion in conclusions: # Check if the conclusion is already a known fact if conclusion not in facts: # If the conclusion is not already a known fact, add it to the list of facts facts[conclusion] = [] # Set the flag to indicate that a new fact was generated new_facts_generated = True # If no new facts were generated, break out of the loop if not new_facts_generated: break # Check if the query is in the list of known conclusions if query in conclusions: print("The query can be inferred to be true") else: print("The query cannot be inferred to be true") This script will iterate over the rules and apply them to the known facts until no new facts are generated. It will then check if the query is in the list of known conclusions and output the result.
janalsncm 4y ago
Slight correction, Deep Blue beat Kasparov in the 90s with Minimax but did not use neural networks afaik. Alpha Go does use neural networks to traverse a game tree but it’s algorithm is Monte Carlo Tree Search.
CrimpCity 4y ago
The paper you shared is WAY more interesting than the original post. Maybe this is a naive criticism but if the goal of the model is to "emulate the correct reasoning function" by which is implied that it learns prepositional logic rules then shouldn't the model have some architecture that tries to model generic logic rules and then using that score the input data and predict the result? Currently it seems like the model is simply scoring each predicate since each of it's reasoning layers performs one step of forward chaining, adding some predicates to the Proved Facts. My naive (and way undercooked) way of conducting this experiment would have been to try to use a GAN where the generator model comes up with new predicates and then the discriminator model that tries to classify them as real/fake. I would then try to train an MLP on top of the generator to classify the result and swap out the MLP depending on the sampling method basically the generator model becomes the pre-trained thing we care about to see if it can generalize. Another nitpick is that the authors claim that BERT has enough "capacity" to solve SimpleLogic BUT this isn't actually what they want to achieve since solving != learning reasoning. So it feels like a bait and switch since IMO if a model has the capacity for something then it has some architectural aspect to it that can be used to show that it can achieve a smaller version of what you want and they didn't prove BERT can learn ANY prepositional logic rule.

abetusk 4y ago

Nope, I take it back. The article is not coming from a place of (in my opinion) reason. This is, in fancy language and snowed under by enumeration of historical research, math and science, an argument that "computers can never have a soul". From the article: """ No matter how sophisticated the computation is ... there remains an unbridgeable gap (a “humanity gap”) between the ... ability of machine and the general problem solving ability of man. """ This is just standard dualist nonsense under the guise of logical objectivity. Note that there is a deeper issue about the shortcomings of statistical pattern matching to do logic, which is what the paper I reference above is trying to get at, but the parent article's fundamental assumption/conclusion is that computers will never be able to reason.

c43verge 4y ago
Thanks a lot for summarizing it. I was curious to see whether it had a new idea, apparently not.
stelonix 4y ago
It really isn't about having a soul, but more like about either: 1) stating human brains are actually hypercomputers, which cannot be implemented in real life at the moment 2) stating just what you said before: pattern matching, no matter how many layers or parameters used, is not all there is to intelligence/consciousness At the moment, whether you like it or not, there's no proof at all the human brain works in any way like a Turing machine. That idea which has nothing to back it up is the usual POV here on HN and it is the actual faith-based belief but again, no scientific papers to back it up.
- orlp 4y ago
  > stating human brains are actually hypercomputers, which cannot be implemented in real life at the moment Extraordinary claims require extraordinary evidence, and absence of evidence is not evidence of absence.
  - adgjlsfhk1 4y ago
    it's pretty easy to prove that a brain can simulate a Turing machine, and it's pretty hard to disbelieve that a computer can simulate a brain (by running the physics)
    - orlp 4y ago
      The claim I find objectionable is that brains do 'hypercomputation', and can not be ran on a Turing machine.
      - stelonix 4y ago
        Brains can solve the halting problem, Turing machines cannot.

sharemywin 4y ago

(From the paper you mentioned above)* added some extra lines so it would print properly in this website. it ran without change in python. Here's what I got out of ChatGPT after I gave up trying to get it to answer it directly: write A Python script that parses the following and uses forward chaining inference to answer the following: Facts: Alice is fast. Alice is normal. Rules: If Alice is fast and smart, then Alice is bad. If Alice is normal, then Alice is smart. If Alice is normal and happy, then Alice is sad. Query 1: Alice is bad. Answer: Here is a python script that uses forward chaining inference to answer the given query: Copy code # Define the facts facts = { "fast": ["Alice"], "normal": ["Alice"], } # Define the rules rules = { ("fast", "smart"): ["bad"], ("normal",): ["smart"], ("normal", "happy"): ["sad"], } # Define the query query = "bad" # Initialize the list of known conclusions conclusions = [] # Iterate over the rules for premises, conclusions_ in rules.items(): # Check if all the premises are known to be true if all(premise in facts for premise in premises): # If all the premises are true, add the conclusions to the list of known conclusions conclusions.extend(conclusions_) # Check if the query is in the list of known conclusions if query in conclusions: print("The query can be inferred to be true") else: print("The query cannot be inferred to be true") For the given facts and rules, the script will output "The query can be inferred to be true" because Alice is fast and normal, which means that Alice is smart according to the rules, and Alice is also fast, which means that Alice is bad according to the rules. now chain the rules using the facts from the previous run to see if any new facts are generated and if not then quit To chain the rules using the facts from the previous run, we can simply put the forward chaining inference logic inside a loop and continue iterating until no new facts are generated. Here is an updated version of the script that does this: Copy code # Define the facts facts = { "fast": ["Alice"], "normal": ["Alice"], } # Define the rules rules = { ("fast", "smart"): ["bad"], ("normal",): ["smart"], ("normal", "happy"): ["sad"], } # Define the query query = "bad" # Initialize the list of known conclusions conclusions = [] # Keep iterating until no new facts are generated while True: # Iterate over the rules for premises, conclusions_ in rules.items(): # Check if all the premises are known to be true if all(premise in facts for premise in premises): # If all the premises are true, add the conclusions to the list of known conclusions conclusions.extend(conclusions_) # Initialize a flag to track whether any new facts were generated new_facts_generated = False # Iterate over the conclusions for conclusion in conclusions: # Check if the conclusion is already a known fact if conclusion not in facts: # If the conclusion is not already a known fact, add it to the list of facts facts[conclusion] = [] # Set the flag to indicate that a new fact was generated new_facts_generated = True # If no new facts were generated, break out of the loop if not new_facts_generated: break # Check if the query is in the list of known conclusions if query in conclusions: print("The query can be inferred to be true") else: print("The query cannot be inferred to be true") This script will iterate over the rules and apply them to the known facts until no new facts are generated. It will then check if the query is in the list of known conclusions and output the result.

janalsncm 4y ago

Slight correction, Deep Blue beat Kasparov in the 90s with Minimax but did not use neural networks afaik. Alpha Go does use neural networks to traverse a game tree but it’s algorithm is Monte Carlo Tree Search.

CrimpCity 4y ago

The paper you shared is WAY more interesting than the original post. Maybe this is a naive criticism but if the goal of the model is to "emulate the correct reasoning function" by which is implied that it learns prepositional logic rules then shouldn't the model have some architecture that tries to model generic logic rules and then using that score the input data and predict the result? Currently it seems like the model is simply scoring each predicate since each of it's reasoning layers performs one step of forward chaining, adding some predicates to the Proved Facts. My naive (and way undercooked) way of conducting this experiment would have been to try to use a GAN where the generator model comes up with new predicates and then the discriminator model that tries to classify them as real/fake. I would then try to train an MLP on top of the generator to classify the result and swap out the MLP depending on the sampling method basically the generator model becomes the pre-trained thing we care about to see if it can generalize. Another nitpick is that the authors claim that BERT has enough "capacity" to solve SimpleLogic BUT this isn't actually what they want to achieve since solving != learning reasoning. So it feels like a bait and switch since IMO if a model has the capacity for something then it has some architectural aspect to it that can be used to show that it can achieve a smaller version of what you want and they didn't prove BERT can learn ANY prepositional logic rule.