4 ms·

My hot take is that this correct in spirit. However, it just moves the conceptual line between data/feature engineering and modeling so as to make the data/fea

My hot take is that this correct in spirit. However, it just moves the conceptual line between data/feature engineering and modeling so as to make the data/feature engineering encompass nearly all of the process. By this definition of ML, one might just say that clever ML is a really clever way to engineer useful features, after which we "merely" average them. It feels like semantics.

mjburgess 5y ago
From a certain POV the structure of an animal's body, as for example, it learns to ride a bike can be given this kind of formulation. Namely, that the muscle tissue and the motor cortext express a latent representation of riding which is formed under the animal's self-regulative goals of "stability" and "getting to some location" etc. From this POV, one can model any aspect of reality with this sort of language. I used to be annoyed by this, but less so now. Annoyed because it's pretty hopelessly useless describe any complex physical system this way. It only makes sense if your relevant "feature spaces" are discrete and computationally tractable... i dont think our muscles are good candidates. (Nor likewise, is almost anything). Nevertheless, you are right that "ML" on a domain whose dimensions are infinite, and in which slices through those dimensions are infinite, and whose models are chaotic -- then ML with an infinity of time to run would produce an animal. And indeed, an intelligent system. One has to remember here that no one is doing this. What theyre doing is taking a highly regulated non-chatoic, tractable, measured and largely unambigious slice through these spaces --- sampling them --- and taking an average. Who gave you the knife to take this slice? Was it your cognition, was it the world? No, neither, it was your body -- largely; and its ability to regulate your imagination; and to guide action. Formulating reality as-if its patterns can be determined statistically leads to absurd levels of non-computability. Every possible partitioning of the world, in every possible arrangment, moving in every possible way -- what discrete model of this is there? There isn't one. And this isnt semantic. The challenge of building an intelligent system is a (bio-)engineering one: we require a material which grows intelligently, whose internal chaos is useful. Not a dumb little calculator which comes in to average our efforts after the fact.
- neatze 5y ago
  There are methods that build networks from scratch, including position of individual neurons, connections, weights, and mutation methods this models perform poorly to say the least compared to RL models in single or multi agent settings just learning only through interaction within environment. In my limited perspective we do not understand how/where/why brain changes on second by second bases on neural level, furthermore we have no slightest idea how instincts in brain are encoded through DNA. In terms of plains and birds analogy, it is like trying to build airplain without understanding basics of aerodynamics.

mjburgess 5y ago

From a certain POV the structure of an animal's body, as for example, it learns to ride a bike can be given this kind of formulation. Namely, that the muscle tissue and the motor cortext express a latent representation of riding which is formed under the animal's self-regulative goals of "stability" and "getting to some location" etc. From this POV, one can model any aspect of reality with this sort of language. I used to be annoyed by this, but less so now. Annoyed because it's pretty hopelessly useless describe any complex physical system this way. It only makes sense if your relevant "feature spaces" are discrete and computationally tractable... i dont think our muscles are good candidates. (Nor likewise, is almost anything). Nevertheless, you are right that "ML" on a domain whose dimensions are infinite, and in which slices through those dimensions are infinite, and whose models are chaotic -- then ML with an infinity of time to run would produce an animal. And indeed, an intelligent system. One has to remember here that no one is doing this. What theyre doing is taking a highly regulated non-chatoic, tractable, measured and largely unambigious slice through these spaces --- sampling them --- and taking an average. Who gave you the knife to take this slice? Was it your cognition, was it the world? No, neither, it was your body -- largely; and its ability to regulate your imagination; and to guide action. Formulating reality as-if its patterns can be determined statistically leads to absurd levels of non-computability. Every possible partitioning of the world, in every possible arrangment, moving in every possible way -- what discrete model of this is there? There isn't one. And this isnt semantic. The challenge of building an intelligent system is a (bio-)engineering one: we require a material which grows intelligently, whose internal chaos is useful. Not a dumb little calculator which comes in to average our efforts after the fact.

neatze 5y ago
There are methods that build networks from scratch, including position of individual neurons, connections, weights, and mutation methods this models perform poorly to say the least compared to RL models in single or multi agent settings just learning only through interaction within environment. In my limited perspective we do not understand how/where/why brain changes on second by second bases on neural level, furthermore we have no slightest idea how instincts in brain are encoded through DNA. In terms of plains and birds analogy, it is like trying to build airplain without understanding basics of aerodynamics.