A lot of the comments are listing their personal suspicion of causes, without attempting to illustrate or provide evidence. Some of those suspicions are probabl

A lot of the comments are listing their personal suspicion of causes, without attempting to illustrate or provide evidence. Some of those suspicions are probably true, and some false. Obviously it is also an outlet for all our personal gripes with the system. Irrespective of the veracity of a proposed cause, I would like to see more discussion of actual examples of current progress, and trying to identify what caused or enabled the progress to be made by comparing with the average paper in the back of our mind: what was different, why did they achieve the progress today and did nobody achieve the insight say five years ago? Ideally, since we are discussing on HN, it should be an example that would be understood by most participants here. It is in this spirit that I will give exactly such an example of recent progress (with which I am entirely unaffiliated). First some minimal background which I assume you are not familiar with: quantum chemistry and solid state physics software. Just take a quick look at this list on wikipedia, you may recognize the names of some pieces of software like ABINIT... make sure you pay attention to the DFT column, and that virtually all packages support DFT calculations, which has been pretty much state of the art for the last decades. https://en.wikipedia.org/wiki/List_of_quantum_chemistry_and_solid_state_physics_software https://en.wikipedia.org/wiki/List_of_quantum_chemistry_and_... In physics gradients often arise naturally for example forces as gradients of potential energy... But in general even outside physics gradients are useful in nearly all field for optimization... Now the part you probably already understand: AD, Automatic Differentiation or Algorithmic Differentiation, and in the often occuring case of a single scalar function in N variables, reverse-mode AD... And that it takes on the order of 5 times a single function evaluation to calculate a gradient. Now specific subfields of physics have been using AD and adjoint sensitivities for a long time (nuclear engineering, oceanography) but it is not a standard part of physics curricula. Outside of these specific subfields, Automatic Differentiation has been gaining momenta over the last decades (books, comprehensive reviews, ...) Physics students of course learn differentiation symbolically on paper for short formulas, or in software packages like MACSYMA, Maple, ... but even then you keep the number of variables low for tractability. These students will also understand you can emulate differentiation numerically by using a finite delta: d f(x1, x2, ..., xN) / dx2 = ~ [f(x1, x2 +delta, ..., xN) - f(x1, x2, ..., xN)]/delta fully understanding that for a complete gradient you need (N+1) evaluations of f, that delta too large will be inaccurate due to functional nonlinearity, and delta too small will be inaccurate due to numerical rounding of floats... Nobody thinks of showing reverse mode automatic differentiation to physics students as part of their curricula! If you crash into a physics course and ask the students how to calculate the gradient of a big function of 1000 variables, they won't be able to help you, but then you can explain that such a thing is in fact possible! You see where this is going... What if some of the numerical computations, say molecular modeling, could benefit from this insight? That's exactly what happened recently: https://arxiv.org/abs/1010.5560?context=cond-mat https://arxiv.org/abs/1010.5560?context=cond-mat Now look at the authors' institutions of this paper: SISSA, International School for Advanced Studies, 2 DEMOCRITOS National Simulation Center, Quantitative Strategies, Investment Banking Division, Credit Suisse Group That's people with an interdisciplinary background. Could it be that we are over-specialized? Or perhaps mis-specialized? Imagine an alternate world where there is so much math that mathematics has decided to specialize thus: after a common course of Fundamental Math, you can specialize in Definitions, or perhaps in Theorems, or perhaps in Proofs, or perhaps in Conjectures... clearly their progress in math is going to suffer!! Could it be that experts and professionals have become "too polite" to the point of circle j* ? A kind of "politeness omerta"? Imagine a mathematician and a physicist talking: as long as we're discussing math, the physicist nods with interest, and doesn't make suggestions how he would do it differently and vice versa. The "don't criticize a professional in his domain" attitude? Wouldn't it have been better if someone who understood AD say 20 years ago, who ends up in a conversation of molecular chemistry software physicist to say something like "I don't have the domain knowledge, but first thing I would do is find out if the calculation could be viewed as the computation of a gradient, or else as an optimization of an explicit goal" "how dare you make a suggestion outside of math?! and thats thousands of variables, its intractable" ? We might have had this 20 years ago! Mansplain it if necessary. it's this upside down world where we are polite and stoic in our papers, but elbow working back stabbers undermining the workgroup next door, all the while feigning this "politeness professional omerta"