4 ms·

What if the machine does not know that it is dealing with numbers? It's all symbolic: '1'+'1'='2', etc.

aswanson 19y ago
The neural network doesn't "know" that it is dealing with anything, just as you don't "know" the function your body uses to expand and contract your heart. You could be feeding it stock quotes, rgb pixel values, your daily weight, anything. If the information can reduced to numeric values (it can) the network will determine the relationship in the form of a function.
- pixcavator 19y ago
  If the input is numbers then what happens is simply extrapolation (or regression etc). If the input is symbols, then what? The fact that you can associate numbers to the symbols does not help. The extrapolation will be meaningless. Unless you happen to have the correspondence '1'->1, '2'->2, etc,
  - aswanson 19y ago
    How are you defining symbols? How are you defining meaningless, or meaning at all for that matter?
    - pixcavator 19y ago
      Suppose you have a function f of two variables and all you know about it is that f(x,y)=x+y for a few values of x,y. With this data, the computer extrapolates linearly and finds f(x,y) for _all_ values of x,y. The computer just "learnt" how to add! If all you have is symbols, however, you have to have numbers associated with those symbols. But you can't expect 1 correspond to '1' and 2 to '2'. As a result, your function f - and its extrapolation - will have nothing to do with addition (f won't even be linear).
      - aswanson 19y ago
        So am I correct in assuming that you are looking for a machine that can do the mapping of a symbol into a form that the function can learn from? Your visual and auditory systems do this, and since they are physical systems this mapping is computable. By the way, for the addition example , the neural network needs only one example, e.g. 1+1=2, applied repeatedly to discover the addition of any two inputs. There is no extrapolation between examples needed in this case.
        pixcavator 19y ago
        "Extrapolation is the process of constructing new data points outside a discrete set of known data points."
        aswanson 19y ago
        Right. Confusing extrapolation with interpolation.
        aswanson 19y ago
        I have rethought what you have said, and I think I am now closer to understanding what your question is. You are posing the question of what to do with an input, not the operation itself. For instance, if I feed the network an image of 5 and 3, with the output being the image 8, the network would not perform addition but some other functional mapping. Is that the feature you are suggesting that it should 'understand'?
        pixcavator 19y ago
        Yes.
machine 19y ago
What you are talking about is more along the lines of inductive logic programming, where the goal is to induce general symbolic rules from specific examples. Even with ILP I'm not sure how exactly you could phrase things so that the machine could learn "addition" though. An ILP system could probably learn something like a + b = c implies b + a = c (i.e. commutativity) and other properties of addition from examples.
- pixcavator 19y ago
  If it can learn these properties, it may be even better! Then it can do all computations. So, can it learn commutativity, and other properties? Would it be smart enough to look for them in the first place?
  - machine 19y ago
    I don't know a lot about ILP algorithms, but my understanding of them is that they basically search the space of logical rules for rules that "explain" the examples they are given. More formally, they look for rules from which you could then prove the given examples. So in an ideal world you could give an ILP system a bunch of formulas and it would give you back the Peano axioms or something like that I'm not sure if state of the art ILP systems are good enough to do that in practice or not.

The neural network doesn't "know" that it is dealing with anything, just as you don't "know" the function your body uses to expand and contract your heart. You could be feeding it stock quotes, rgb pixel values, your daily weight, anything. If the information can reduced to numeric values (it can) the network will determine the relationship in the form of a function.

pixcavator 19y ago
If the input is numbers then what happens is simply extrapolation (or regression etc). If the input is symbols, then what? The fact that you can associate numbers to the symbols does not help. The extrapolation will be meaningless. Unless you happen to have the correspondence '1'->1, '2'->2, etc,
- aswanson 19y ago
  How are you defining symbols? How are you defining meaningless, or meaning at all for that matter?
  - pixcavator 19y ago
    Suppose you have a function f of two variables and all you know about it is that f(x,y)=x+y for a few values of x,y. With this data, the computer extrapolates linearly and finds f(x,y) for _all_ values of x,y. The computer just "learnt" how to add! If all you have is symbols, however, you have to have numbers associated with those symbols. But you can't expect 1 correspond to '1' and 2 to '2'. As a result, your function f - and its extrapolation - will have nothing to do with addition (f won't even be linear).
    - aswanson 19y ago
      So am I correct in assuming that you are looking for a machine that can do the mapping of a symbol into a form that the function can learn from? Your visual and auditory systems do this, and since they are physical systems this mapping is computable. By the way, for the addition example , the neural network needs only one example, e.g. 1+1=2, applied repeatedly to discover the addition of any two inputs. There is no extrapolation between examples needed in this case.
      - pixcavator 19y ago
        "Extrapolation is the process of constructing new data points outside a discrete set of known data points."
        aswanson 19y ago
        Right. Confusing extrapolation with interpolation.
        aswanson 19y ago
        I have rethought what you have said, and I think I am now closer to understanding what your question is. You are posing the question of what to do with an input, not the operation itself. For instance, if I feed the network an image of 5 and 3, with the output being the image 8, the network would not perform addition but some other functional mapping. Is that the feature you are suggesting that it should 'understand'?
        pixcavator 19y ago
        Yes.

machine 19y ago

What you are talking about is more along the lines of inductive logic programming, where the goal is to induce general symbolic rules from specific examples. Even with ILP I'm not sure how exactly you could phrase things so that the machine could learn "addition" though. An ILP system could probably learn something like a + b = c implies b + a = c (i.e. commutativity) and other properties of addition from examples.

pixcavator 19y ago
If it can learn these properties, it may be even better! Then it can do all computations. So, can it learn commutativity, and other properties? Would it be smart enough to look for them in the first place?
- machine 19y ago
  I don't know a lot about ILP algorithms, but my understanding of them is that they basically search the space of logical rules for rules that "explain" the examples they are given. More formally, they look for rules from which you could then prove the given examples. So in an ideal world you could give an ILP system a bunch of formulas and it would give you back the Peano axioms or something like that I'm not sure if state of the art ILP systems are good enough to do that in practice or not.