4 ms·

So we're firmly in the era of few people caring about few things now aren't we.

lotsofpulp 7mo ago
I have always assumed the further away from math and physics a field is, the higher the probability of any given “research” to be false. Even biology, I might give 50% odds at best, but that is due to the difficulty of observing and measuring in that field. Anything past biology might as well be as fiction due to how easy it is to come up with whatever results you want.
- erikerikson 7mo ago
  I appreciate that physics and math are simple, reductive, and first principles enough to be tractable. Solving easier problems always has better optics so long as all problems look equivalent. I'm guilty myself, only rising to neuroscience and relatively superficially at that...
- austinjp 7mo ago
  Oh I'm sure the grifters will find ways in. The other disciplines may have provided a "moat" for the past few decades, but it won't last forever.
- p-e-w 7mo ago
  I fully expect that future programs for formalizing mathematics will reveal that most sufficiently complex proofs are riddled with gaps and errors, and that some of them actually led to false results. Annals of Mathematics once published a supposed proof (related to intersection bodies IIRC) for a statement that turned out to be false, and it was discovered only by someone else proving the opposite, not by someone finding an error.
- Georgelemental 7mo ago
  Quantum physics, due to its own "difficulty of observing and measuring", has its fair share of nonsense too
- chromacity 7mo ago
  I think that theoretical math and physics are special, but probably not in the way you assume. It's just that there isn't a whole lot of grant money, prestige, or influence associated with them (unless you accomplish something truly exceptional). Computer science is very close to math and should be even easier to verify, but there's plenty of dubious results published every year, simply because it's more profitable to game the system. For example, I'd wager that 50%+ of academic claims related to information security are bogus or useless. Similarly, in the physics-adjacent world of materials science, a lot of announcements related to metamaterials and nanotech are suspect.
  - kmaitreys 7mo ago
    Great take. I have seen the discussion on this often gets turned into a hard vs soft science debate where in actuality it's just simply about money.
    - stogot 7mo ago
      I track these across all fields. It’s money and prestige and arrogance and ignorance and “keep my job” and more
  - pjdesno 7mo ago
    I would point out that most products are useless, and either fail or replace other products which weren't any worse. None of which prevented me from cashing my paychecks for the first half of my career when I worked in private industry. Most scientific research represents about the same amount of improvement over the state of the art as the shitty web app or whatever that you're working on right now. It's not zero, but very few are going to be groundbreaking. And since the rules are that we all have to publish papers[*], the scientific literature (at least in my field, CS) looks less like a carefully curated library of works by geniuses, and more like an Amazon or Etsy marketplace of ideas, where most are crappy. [* just like software engineers have to write code, even if the product ends up being shitty or ultimately gets canceled] Neither of us are going to be changing how the system works, so my advice is to deal with it.
- hnburnsy 7mo ago
  If it doesn't have "science" in the name, it's a science If it has the suffix "logy", it's a semi-science If it has the word "science", it's not
  - fdupress 7mo ago
    Astrology thanks you for your service.
  - wavemode 7mo ago
    Philosophers rejoice.
- sigbottle 7mo ago
  Observing, measuring, but also repeatability and ground truth. Math (and theoretical adjacents like TCS) claim not to make any fundamental claims about the actual world (compared to 17th century philosopher-mathematicians like Leibniz), and physics studies the basest of, well, physical phenomenon. I don't even know how you would begin actually rigorously studying sociology unless you could start simulating real humans in a vat, or you inject everybody with neuralink. (but that already selects for a type of society, and probably not a good one...) To be clear, I don't think all sociological observations are bad. However, I tend to heavily disregard "mathematical sociological studies" in favor of just... hearing perspectives. New ones and unconventional ones especially, as in a domain where a lot of theories "seem legit", I want to just hear very specific new ways of thinking that I didn't think about before. I find that to be a pretty good heuristic for finding value, if the verification process itself is broken.
- suzzer99 7mo ago
  > Anything past biology might as well be as fiction due to how easy it is to come up with whatever results you want. I used to work for the leading statistical expert witness in the country. Whenever I read something like this: > The empirical strategy in Eccles, Ioannou, and Serafeim (2014) rests on a demanding requirement: the “treated” and “control” firms must be so closely matched that which firm is treated is essentially random. The authors appear to recognize this, reporting that they used very strict matching criteria “to ensure that none of the matched pairs is materially different.” I just assume they kept trying different "very strict matching criteria" until they got the matches they wanted. Which is basically what we did all day to support our client (usually big auto or big tobacco). We never presented any of the detrimental analyses to our boss, so he couldn't testify about them on the stand even if asked. Although in this case it sounds like the authors couldn't even do that, and just fudged the data instead.
- wavemode 7mo ago
  There's plenty of results in math and physics that are true, in the sense that the math checks out, but are useless, in the sense that the authors claim they've made a breakthrough, but actually they've just tweaked a few parameters of an existing unverified theory and constructed a new unverified theory. (If you've ever read a news headline like "physicists now believe reality may actually have 400 dimensions!", they were probably citing one of these papers.) There are also plenty of physics papers where, the math actually just doesn't check out at all. But those, at least, rarely make it into headlines or reputable journals.

I have always assumed the further away from math and physics a field is, the higher the probability of any given “research” to be false. Even biology, I might give 50% odds at best, but that is due to the difficulty of observing and measuring in that field. Anything past biology might as well be as fiction due to how easy it is to come up with whatever results you want.

erikerikson 7mo ago
I appreciate that physics and math are simple, reductive, and first principles enough to be tractable. Solving easier problems always has better optics so long as all problems look equivalent. I'm guilty myself, only rising to neuroscience and relatively superficially at that...
austinjp 7mo ago
Oh I'm sure the grifters will find ways in. The other disciplines may have provided a "moat" for the past few decades, but it won't last forever.
p-e-w 7mo ago
I fully expect that future programs for formalizing mathematics will reveal that most sufficiently complex proofs are riddled with gaps and errors, and that some of them actually led to false results. Annals of Mathematics once published a supposed proof (related to intersection bodies IIRC) for a statement that turned out to be false, and it was discovered only by someone else proving the opposite, not by someone finding an error.
Georgelemental 7mo ago
Quantum physics, due to its own "difficulty of observing and measuring", has its fair share of nonsense too
chromacity 7mo ago
I think that theoretical math and physics are special, but probably not in the way you assume. It's just that there isn't a whole lot of grant money, prestige, or influence associated with them (unless you accomplish something truly exceptional). Computer science is very close to math and should be even easier to verify, but there's plenty of dubious results published every year, simply because it's more profitable to game the system. For example, I'd wager that 50%+ of academic claims related to information security are bogus or useless. Similarly, in the physics-adjacent world of materials science, a lot of announcements related to metamaterials and nanotech are suspect.
- kmaitreys 7mo ago
  Great take. I have seen the discussion on this often gets turned into a hard vs soft science debate where in actuality it's just simply about money.
  - stogot 7mo ago
    I track these across all fields. It’s money and prestige and arrogance and ignorance and “keep my job” and more
- pjdesno 7mo ago
  I would point out that most products are useless, and either fail or replace other products which weren't any worse. None of which prevented me from cashing my paychecks for the first half of my career when I worked in private industry. Most scientific research represents about the same amount of improvement over the state of the art as the shitty web app or whatever that you're working on right now. It's not zero, but very few are going to be groundbreaking. And since the rules are that we all have to publish papers[*], the scientific literature (at least in my field, CS) looks less like a carefully curated library of works by geniuses, and more like an Amazon or Etsy marketplace of ideas, where most are crappy. [* just like software engineers have to write code, even if the product ends up being shitty or ultimately gets canceled] Neither of us are going to be changing how the system works, so my advice is to deal with it.
hnburnsy 7mo ago
If it doesn't have "science" in the name, it's a science If it has the suffix "logy", it's a semi-science If it has the word "science", it's not
- fdupress 7mo ago
  Astrology thanks you for your service.
- wavemode 7mo ago
  Philosophers rejoice.
sigbottle 7mo ago
Observing, measuring, but also repeatability and ground truth. Math (and theoretical adjacents like TCS) claim not to make any fundamental claims about the actual world (compared to 17th century philosopher-mathematicians like Leibniz), and physics studies the basest of, well, physical phenomenon. I don't even know how you would begin actually rigorously studying sociology unless you could start simulating real humans in a vat, or you inject everybody with neuralink. (but that already selects for a type of society, and probably not a good one...) To be clear, I don't think all sociological observations are bad. However, I tend to heavily disregard "mathematical sociological studies" in favor of just... hearing perspectives. New ones and unconventional ones especially, as in a domain where a lot of theories "seem legit", I want to just hear very specific new ways of thinking that I didn't think about before. I find that to be a pretty good heuristic for finding value, if the verification process itself is broken.
suzzer99 7mo ago
> Anything past biology might as well be as fiction due to how easy it is to come up with whatever results you want. I used to work for the leading statistical expert witness in the country. Whenever I read something like this: > The empirical strategy in Eccles, Ioannou, and Serafeim (2014) rests on a demanding requirement: the “treated” and “control” firms must be so closely matched that which firm is treated is essentially random. The authors appear to recognize this, reporting that they used very strict matching criteria “to ensure that none of the matched pairs is materially different.” I just assume they kept trying different "very strict matching criteria" until they got the matches they wanted. Which is basically what we did all day to support our client (usually big auto or big tobacco). We never presented any of the detrimental analyses to our boss, so he couldn't testify about them on the stand even if asked. Although in this case it sounds like the authors couldn't even do that, and just fudged the data instead.
wavemode 7mo ago
There's plenty of results in math and physics that are true, in the sense that the math checks out, but are useless, in the sense that the authors claim they've made a breakthrough, but actually they've just tweaked a few parameters of an existing unverified theory and constructed a new unverified theory. (If you've ever read a news headline like "physicists now believe reality may actually have 400 dimensions!", they were probably citing one of these papers.) There are also plenty of physics papers where, the math actually just doesn't check out at all. But those, at least, rarely make it into headlines or reputable journals.