There's an axiom of software engineering that any sufficiently large system (or, I suppose, algorithm) is incomprehensible by a single person. That's to say, a

There's an axiom of software engineering that any sufficiently large system (or, I suppose, algorithm) is incomprehensible by a single person. That's to say, a single person may be a high-level architect and understand how the high-level modules work together, but he won't understand how the inner workings of those modules work, especially if there are multiple layers to those modules, both bureaucratically and technically. And somebody who busies themselves understanding and working on the lowest levels, won't have the time to work on the higher architecture, nor would it interest him, as long as boundaries are well established and respected. It's not that far of a stretch at all to claim that strong AI would be similar. At some point, everything worked on by today's AI researchers becomes some refined lower-level module, and tomorrow's researchers will use them as building blocks for the architecture for strong AI. Those lower-level modules will have coefficients derived from massive training sets, and tomorrow's researchers will simply understand that those coefficients work, without any real understanding of how those coefficients were derived. And that's OK.