2 ms·

I wonder frequently about what would happen if we stopped searching for a network architecture that would learn intelligence from training data amd treated it m

I wonder frequently about what would happen if we stopped searching for a network architecture that would learn intelligence from training data amd treated it more as an engineering problem, taking these very successful components (GPT-3, an object recognition model, one of the strategy game playing reinforcement learning networks) and then putting enormous human effort into the plumbing between them and an interface between the combined result and the world. At the least, you would learn which domains really do need new architectures and which are basically good enough when used as part of a larger system that can help compensate for the shortcomings.

jkhdigital 6y ago
One of my big takeaways from reading GEB was that while higher-level semantics can emerge from any low-level symbolic substrate, the details of how that semantics emerges are not at all simple or obvious or “likely to happen by random chance”. Dawkins’ book The Selfish Gene, published just a few years earlier, is the clearest exposition that I have read of how this process probably played out in terrestrial life: the “semantics” encoded by amino acid sequences correspond to a molecule/cell/organism’s likelihood of surviving and replicating. For all but the simplest and most ephemeral replicators, this generally means accurately predicting environmental conditions. General intelligence, then, would conceivably arise simply due to selection pressures pushing organisms to live in the broadest range of environments. In some sense, this process does sound more like an engineering problem, as the embodiment which “contains” the intelligence is probably not an optional component.