164 ms·

> Focusing on mathematical abstracta completely misses the problem... I don’t think it does. We’ve been studying these networks for a long time now, and “comm

> Focusing on mathematical abstracta completely misses the problem... I don’t think it does. We’ve been studying these networks for a long time now, and “common sense” interpretations are largely “completely bs” when examined closely. These investigations have a tangible impact on how to prune and design networks. High level hand waving about measuring the measuring device may seem profound, but I fail to see any meaningful application. You can reconstruct 3D images from a few samples; see neural radiance fields. The observer is irrelevant.

mjburgess 6y ago
My comment here wasnt aimed at the problem of understanding and improving DNNs; rather, of generalisation and 'comprehension' proper. 3d image reconstruction is trivially possible, with the right data and with the right assumptions encoded into the right algorithm. My target of attack here is the equivocation that 'compression is comprehension'. In my view comprehension isnt about 'generalization via compressed archetypes' as in ordinary NNs. And it isnt about 3d reconstruction given the relevant modification to data and approach. Rather generalization is abduction. It is the ability to, via a single instance, form law-like provisional universalising models which explain your environment. This process will lead to a 'compressed', 'representation', but nothing like the sense in which compression is here used. It is this that intuitively is naively assumed of these systems. They do not, and cannot, abduct. Abduction isnt a statistical process; involving at the very least, counterfactual reasoning and hypothetical action. This is the problem that such equivocations miss (ie. that c = c). And in my view this is an engineering challenge; not something to be specified for a universal computer. The relevant capacities missing arent better means of compression.
- wokwokwok 6y ago
  > In my view comprehension isnt about 'generalization via compressed archetypes' as in ordinary NNs... How is this helpful? Be specific. The point being made is that inductions can be made by a compressed representation of the data. No one is claiming this is a key insight to AGI; you’re just taking the word “comprehension” and arbitrarily assigning it some personal meaning. This is why mathematicial descriptions are useful; you’re personal view of what “comprehension” means isn’t relevant here. The article doesn’t discuss comprehension. That is something you’ve, as far as I can tell, meaninglessly brought into this discussion. How are your insights helpful for improving DNNs? Be specific.
  - trap_chateau 6y ago
    Sorry but I have to chime in here. His thoughts seem insightful and relevant to discussion. You are calling his contributions 'meaningless' and condescendingly telling him twice to 'be specific'. That's not how things are meant to be discussed around here and not how you get an answer out of someone.
    - mjburgess 6y ago
      Yip. I couldn't move myself to type the obvious: a person replying to my comment doesn't determine whether I am "on topic".
    - wokwokwok 6y ago
      I’m comfortable, reading his comment history, that a blunt request to explain how the the comments are relevant to the OP (and the failure to articulate them when bluntly requested to do so) I indicates that what seems “insightful” is in fact, not falsifiable and therefore nothing more than armchair philosophy. I’m not into that, so if it terminates the discussion then so be it. There was nothing meaningful to discuss.
mscharrer 6y ago
Isn't the observer model explicitly built into NeRF architectures?
- mjburgess 6y ago
  It is; ive just skim read a paper. Though, One trivial way to do it, with NNs in any case, is just to project forward from a range of observer models and guess the observer parameters from them. This is still the wrong sense of generalisation. What cant be guessed is why a person took consecutive pictures at given angles/etc. Such information is necessary to resolve deep ambiguities in cases where your observer model will fail. Eg., yesterday i looked out my window and thought i saw two people; it was actually one with a shadow+bag. I moved my eyes/head/body in such away so as to fit a variety of models and i was able to 'read the scene' in the end. That is comprehension.
  - jmmcd 6y ago
    And there's no reason we couldn't have a deep learning system where the input data (images) included time-stamps and movement vectors, and it could be good both at easy image classification, and at choosing particular "head movements" like those you performed, to help resolve ambiguous cases. Further food for thought: these ambiguous cases seem (do you agree?) to be very rare.
    - mjburgess 6y ago
      Ambiguity is the norm, it isn't rare. Almost all visual input, ie., light, is ambiguous. We (animals) use the history of our prior geometrical-light experiences (ie., walking around) to use environmental cues to resolve ambiguity. That billions (, trillions) of images are needed to aproximate what we can do for a single instance, i think is a good guide to the magnitude of the problem. Google the "amnes room" -- that "illusion" is how we are always seeing.
      - jmmcd 6y ago
        > Ambiguity is the norm Right. But as you say, usually our priors are good enough. The cases where we stop, double-take, and deliberately look from another angle are rare.
Erlich_Bachman 6y ago
> long time 4 decades is not a long time. It is a minuscule amount of time in the grand scheme of things.
- p1esk 6y ago
  It might be the case, like with quantum mechanics, that NN theory is just fundamentally weird (e.g. because the space is so multidimensional), and it's hard for us to understand it no matter how long we study it.

mjburgess 6y ago

My comment here wasnt aimed at the problem of understanding and improving DNNs; rather, of generalisation and 'comprehension' proper. 3d image reconstruction is trivially possible, with the right data and with the right assumptions encoded into the right algorithm. My target of attack here is the equivocation that 'compression is comprehension'. In my view comprehension isnt about 'generalization via compressed archetypes' as in ordinary NNs. And it isnt about 3d reconstruction given the relevant modification to data and approach. Rather generalization is abduction. It is the ability to, via a single instance, form law-like provisional universalising models which explain your environment. This process will lead to a 'compressed', 'representation', but nothing like the sense in which compression is here used. It is this that intuitively is naively assumed of these systems. They do not, and cannot, abduct. Abduction isnt a statistical process; involving at the very least, counterfactual reasoning and hypothetical action. This is the problem that such equivocations miss (ie. that c = c). And in my view this is an engineering challenge; not something to be specified for a universal computer. The relevant capacities missing arent better means of compression.

wokwokwok 6y ago
> In my view comprehension isnt about 'generalization via compressed archetypes' as in ordinary NNs... How is this helpful? Be specific. The point being made is that inductions can be made by a compressed representation of the data. No one is claiming this is a key insight to AGI; you’re just taking the word “comprehension” and arbitrarily assigning it some personal meaning. This is why mathematicial descriptions are useful; you’re personal view of what “comprehension” means isn’t relevant here. The article doesn’t discuss comprehension. That is something you’ve, as far as I can tell, meaninglessly brought into this discussion. How are your insights helpful for improving DNNs? Be specific.
- trap_chateau 6y ago
  Sorry but I have to chime in here. His thoughts seem insightful and relevant to discussion. You are calling his contributions 'meaningless' and condescendingly telling him twice to 'be specific'. That's not how things are meant to be discussed around here and not how you get an answer out of someone.
  - mjburgess 6y ago
    Yip. I couldn't move myself to type the obvious: a person replying to my comment doesn't determine whether I am "on topic".
  - wokwokwok 6y ago
    I’m comfortable, reading his comment history, that a blunt request to explain how the the comments are relevant to the OP (and the failure to articulate them when bluntly requested to do so) I indicates that what seems “insightful” is in fact, not falsifiable and therefore nothing more than armchair philosophy. I’m not into that, so if it terminates the discussion then so be it. There was nothing meaningful to discuss.

mscharrer 6y ago

Isn't the observer model explicitly built into NeRF architectures?

mjburgess 6y ago
It is; ive just skim read a paper. Though, One trivial way to do it, with NNs in any case, is just to project forward from a range of observer models and guess the observer parameters from them. This is still the wrong sense of generalisation. What cant be guessed is why a person took consecutive pictures at given angles/etc. Such information is necessary to resolve deep ambiguities in cases where your observer model will fail. Eg., yesterday i looked out my window and thought i saw two people; it was actually one with a shadow+bag. I moved my eyes/head/body in such away so as to fit a variety of models and i was able to 'read the scene' in the end. That is comprehension.
- jmmcd 6y ago
  And there's no reason we couldn't have a deep learning system where the input data (images) included time-stamps and movement vectors, and it could be good both at easy image classification, and at choosing particular "head movements" like those you performed, to help resolve ambiguous cases. Further food for thought: these ambiguous cases seem (do you agree?) to be very rare.
  - mjburgess 6y ago
    Ambiguity is the norm, it isn't rare. Almost all visual input, ie., light, is ambiguous. We (animals) use the history of our prior geometrical-light experiences (ie., walking around) to use environmental cues to resolve ambiguity. That billions (, trillions) of images are needed to aproximate what we can do for a single instance, i think is a good guide to the magnitude of the problem. Google the "amnes room" -- that "illusion" is how we are always seeing.
    - jmmcd 6y ago
      > Ambiguity is the norm Right. But as you say, usually our priors are good enough. The cases where we stop, double-take, and deliberately look from another angle are rare.

Erlich_Bachman 6y ago

> long time 4 decades is not a long time. It is a minuscule amount of time in the grand scheme of things.

p1esk 6y ago
It might be the case, like with quantum mechanics, that NN theory is just fundamentally weird (e.g. because the space is so multidimensional), and it's hard for us to understand it no matter how long we study it.