3 ms·

The manifold hypothesis is true when the measurement bases aren't orthogonal, and there's lots of them eg., when we measure 3D objects by their relative positio

The manifold hypothesis is true when the measurement bases aren't orthogonal, and there's lots of them eg., when we measure 3D objects by their relative position to 100 objects in a room. That isnt a bridge principle between spaces which measure different properties. The low-D structure of any high-D dataset is only a "washing out" of spare measurements of the same properties. These arent different "property-spaces", ie., the low-D structure isnt a world one has bridged to via statistics (or grad desc, etc.). The properties the formal structure of text captures aren't the properties of objects.. they're something like the social-vocab-coincidental usage of language users in how they "orient their words" around topics. If we change our usage, models which find this structure, become invalidated. We language-users routinely change this structure, and robustly against misunderstanding, because we bridge to the relevant property spaces by being in the world.

VirusNewbie 5y ago
>because we bridge to the relevant property spaces by being in the world. Isn't there an implicit conclusion here that the extra dimensions of 'world being' would actually make the manifold hypothesis true, it's just missing data in the training model? Not to say it's trivial to fix, that missing data might be derived laws of physics, smell, shared world models of a society etc. but none the less I fail to see how conceptually any of that couldn't be reduced to data.
- mjburgess 5y ago
  If you have all possible ways of measuring the world, and all possible measurements you can produce all possible models. A subset of those models will be appear relevantly intelligent. But that isnt intelligence. Intelligence is the solution to the problem of those "all possibles" being imposssible. Namely, intelligence is how animal's conduct themselves in the face of not knowing what the data is.. they do this by conducting themselves and their bodies in their environment -- they dont focus on the problem in the AI/ML/etc. sense... In this way they dont make decisions based on "data" -- they move according to self-regulatory goals. The data which they encounter, in that movement, becomes useful for reasoning about the world. But its radically more accidental, partial, and speculative than ML requires.

VirusNewbie 5y ago

>because we bridge to the relevant property spaces by being in the world. Isn't there an implicit conclusion here that the extra dimensions of 'world being' would actually make the manifold hypothesis true, it's just missing data in the training model? Not to say it's trivial to fix, that missing data might be derived laws of physics, smell, shared world models of a society etc. but none the less I fail to see how conceptually any of that couldn't be reduced to data.

mjburgess 5y ago
If you have all possible ways of measuring the world, and all possible measurements you can produce all possible models. A subset of those models will be appear relevantly intelligent. But that isnt intelligence. Intelligence is the solution to the problem of those "all possibles" being imposssible. Namely, intelligence is how animal's conduct themselves in the face of not knowing what the data is.. they do this by conducting themselves and their bodies in their environment -- they dont focus on the problem in the AI/ML/etc. sense... In this way they dont make decisions based on "data" -- they move according to self-regulatory goals. The data which they encounter, in that movement, becomes useful for reasoning about the world. But its radically more accidental, partial, and speculative than ML requires.