tl;dr - some little thing that the bottom feeder made for his dog nose scratcher recommender tool may prove useful in attacking cancer. -----------------------

tl;dr - some little thing that the bottom feeder made for his dog nose scratcher recommender tool may prove useful in attacking cancer. ---------------------------- Kuhn's paradigm shifts are an idealization. An extreme. Technology doesn't advance in sudden priorly unimaginable leaps of innovations by giants. See, I do not agree with your delineation nor your presentation but I do agree with your core premise. Because we cannot know what will be useful (your negative) we cannot say whether the bottom feeders or those pushing against the hard problems will have the biggest impacts. Likely some from both. The thing is "paradigm changing shifts" are of the same kind as "normal innovation". They are not grander or more special. Allow me to belabour. Most paradigm shifts are the synthesis of lots of little disparate ideas that someone was crafty enough to put in a creative combination. Anyone with sufficient imagination (super intelligence not necessary), the requisite need and correct knowledge could have taken the necessary steps. Quantum Mechanics, Newton's principia, Calculus, special relativity, steam engine and most inventions are like this. There are exceptions (i can think of Cantor's set theory and Grassman's algebra from maths) but they number very few. This is why it is not so surprising that we find often, all of a sudden the same inventions/discoveries appear around the same time by different people - some further along than others (but not necessarily the ones which gain recognition) but all containing pieces of what would come to be studied as a whole by latter generations. In order that these paradigm shifts be made all the requisite concepts needed to have been unlocked. Once so, those skilled at taking in and creating broad vistas of theory shine brightest. As well often times, things are invented that the inventor did not think much of and were just tools on his or her path to something they felt was larger. Of particular relevance to this crowd, Church was trying to provide a foundation for maths and the lambda calculus was just a tool. Or consider that scholars thinking about grain yield for various patches of farm land would eventually lead the way to a theory of determinants and eventually matrices. Most paradigm shifts like zero seem so obvious in hindsight, we struggle to imagine that people took centuries to nail it. In the same way these "big data" and facebook are pushing the state of the art in mining data, maintaining and querying large data stores and pushing computational intelligence, applying non-trivial techniques to solve very menial tasks - like reminding you to go to some club or providing stupid recommendations.. Various companies contributing bits and pieces some combination of which may prove to be vital tools in unravelling things like genetic disorders or cancer later on. Who would have thought that gambling in games of chance would later lead to the mathematics of probability and later to statistical inference and then machine learning. The way I see it, what we need is a change in education. And I don't mean all this university is useless or public schools are failing chatter. Those are another story. What I mean is, as the problems become harder and the people working on them number larger, the silos of knowledge will inevitably become taller and deeper. Unless some framework is developed where it is easy to describe and transfer the core and relations of different theories; the ability to take a step back, look at the pieces and come up with a paradigm shifting generalization will become ever more difficult. With progress slowing as we continually reinvent the same broken pieces over and over again in our little corners of expertise.