4 ms·

The article here is responding to an original blog post [1] that is not really saying the Higgs was not discovered (despite its trolling title), but raising que

The article here is responding to an original blog post [1] that is not really saying the Higgs was not discovered (despite its trolling title), but raising questions about the meaning of "discovery" in systems that are so complicated as those in modern particle physics. I think the author is using the original motivation of musing on null hypotheses to derive the title "The Higgs Discovery Did Not Take Place", and he has successfully triggered the controversy the subtitle ironically denies and the inevitable surface reading condemnations that we see in some of the comments here. [1] https://www.argmin.net/p/the-higgs-discovery-did-not-take https://www.argmin.net/p/the-higgs-discovery-did-not-take

noslenwerdna 2y ago
He is implying that the scientists involved haven't thought of those questions, when in reality this field is one of the strictest in terms of statistical procedures like pre registeration, blinding, multiple hypothesis testing etc Also he makes many factual claims that are just incorrect. Just seems like an extremely arrogant guy who hasn't done his homework
- ttpphd 2y ago
  A computer scientist/electrical engineer who is arrogant? I dunno, I need to see the statistical test to believe that's possible.
  - eightysixfour 2y ago
    Computers are a “complete” system where everything they do is inspectable and, eventually, explainable, and I have observed that people who work with computers (myself included) over estimate their ability to interrogate and explain complex, emergent systems - economics, physics, etc. - which are not literally built on formal logic.
    - dekhn 2y ago
      a single computer might be complete (even then, not everything is inspectable unless you have some very expensive equipment) but distributed systems are not. There was an entire class of engineers at google- SREs- many of whom were previously physicists (or experts in some other quantitative field). A fraction of them (myself included) were "cluster whisperers"- able to take a collection of vague observations and build a testable hypothesis of why things were Fucked At Scale In Prod. Then come up with a way to fix it that didn't mess up the rest of the complete system. Nothing- not even computers are truly built on formal logic. They are fundamentally physics-driven machines with statistical failure rates, etc. There's nothing quite like coming across a very expensive computer which occasionally calculates the equivalent of 1*1 = inf, simply because some physical gates have slightly more electrical charge on them due to RF from a power supply that's 2 feet away.
      - eightysixfour 2y ago
        I think you're mixing up two different things: the challenges of building these systems at scale, and their fundamental properties. Take your example of the expensive computer returning 1*1 = inf because of a nearby power supply - that actually proves my point about computers being knowable systems. You were able to track down that specific environmental interference precisely because computers are built on logic with explicit rules and dependencies. When these types of errors are caught, we know because they do not conform to the rules of the system, which are explicitly defined, by us. We can measure and understand their failures exactly because we designed them. Even massive distributed systems, while complex, still follow explicit rules for how they change state. Every bit of information exists in a measurable form somewhere. Sure, at Google scale we might not have tools to capture everything at once, and no single person could follow every step from electrical signal to final output. But it's theoretically possible - which is fundamentally different from natural systems. You could argue the universe itself is deterministic (and philosophically, I agree), but in practice, the emergent systems we deal with - like biology or economics - follow rules we can't fully describe, using information we can't fully measure, where complete state capture isn't just impractical, it's impossible.
- BeetleB 2y ago
  > when in reality this field is one of the strictest in terms of statistical procedures like pre registeration, blinding, multiple hypothesis testing etc I'm not in HEP, but my graduate work had overlap with condensed matter physics. I worked with physics professors/students in a top 10 physics school (which had Nobel laureates, although I didn't work with them). Things may have changed since then, but the majority of them had no idea what pre-registration meant, and none had taken a course on statistics. In most US universities, statistics is not required for a physics degree (although it is for an engineering one). When I probed them, the response was "Why should we take a whole course on it? We study what we need in quantum mechanics courses." No, my friend. You studied probability. Not statistics. Whatever you can say about reproducibility in the social sciences, a typical professor in those fields knew and understood an order of magnitude more statistics than physicists.
  - noslenwerdna 2y ago
    As an ex-HEP, I can confirm that yes, we had blinding and did correct for multiple hypothesis testing explicitly. As Kyle Cranmer points out, we called it the "look elsewhere effect." Blinding is enforced by the physics group. You are not allowed to look at a signal region until you have basically finished your analysis. For pre-registration, this might be debatable, but what I meant was that we have teams of people looking for specific signals (SUSY, etc). Each of those teams would have generated monte carlo simulations of their signals and compared those with backgrounds. Generally speaking, analysis teams were looking for something specific in the data. However, there are sometimes more general "bump hunts", which you could argue didn't have preregistration. But on the other hand, they are generally looking for bumps with a specific signature (say, two leptons). So yes, people in HEP generally are knowledgeable about stats... and yes, this field is extremely strict compared to psychology for example.
exmadscientist 2y ago
> so complicated as those in modern particle physics But... modern particle physics is one of the simplest things around. (Ex-physicist here, see username.) It only looks complicated because it is so simple that we can actually write down every single detail of the entire thing and analyze it! How many other systems can you say that about?
- spookie 2y ago
  Other systems might not be part of a field as mature as yours, I would argue.
  - exmadscientist 2y ago
    It has nothing to do with "maturity" and everything to do with just hierarchy in general. There is something to the old XKCD joke: https://xkcd.com/435/ https://xkcd.com/435/ because the disciplines really are divided like that. You have to know physics to do chemistry well. You have to know chemistry to do biology well. You have to know biology to ... etc. Whereas to do physics well you need only mathematics. Well, at least, to do the theories well. To actually execute the experiments is, ah, more challenging. So I would argue the Standard Model is pretty much the only thing in all of human knowledge that depends on no other physical theories. It's the bottom. Shame it's pretty useless (intractable) as soon as you have three or more particles to calculate with, though....
- deleted 2y ago
  [deleted]
jaculabilis 2y ago
> I think the author is using the original motivation of musing on null hypotheses to derive the title "The Higgs Discovery Did Not Take Place", It's probably a reference to "The Gulf War Did Not Take Place" by Jean Baudrillard, which took a similar critical view of the Gulf War as TFA takes of the Higgs discovery.
- mellosouls 2y ago
  Possibly! I remember that but completely missed it as an inspiration here.

noslenwerdna 2y ago

He is implying that the scientists involved haven't thought of those questions, when in reality this field is one of the strictest in terms of statistical procedures like pre registeration, blinding, multiple hypothesis testing etc Also he makes many factual claims that are just incorrect. Just seems like an extremely arrogant guy who hasn't done his homework

ttpphd 2y ago
A computer scientist/electrical engineer who is arrogant? I dunno, I need to see the statistical test to believe that's possible.
- eightysixfour 2y ago
  Computers are a “complete” system where everything they do is inspectable and, eventually, explainable, and I have observed that people who work with computers (myself included) over estimate their ability to interrogate and explain complex, emergent systems - economics, physics, etc. - which are not literally built on formal logic.
  - dekhn 2y ago
    a single computer might be complete (even then, not everything is inspectable unless you have some very expensive equipment) but distributed systems are not. There was an entire class of engineers at google- SREs- many of whom were previously physicists (or experts in some other quantitative field). A fraction of them (myself included) were "cluster whisperers"- able to take a collection of vague observations and build a testable hypothesis of why things were Fucked At Scale In Prod. Then come up with a way to fix it that didn't mess up the rest of the complete system. Nothing- not even computers are truly built on formal logic. They are fundamentally physics-driven machines with statistical failure rates, etc. There's nothing quite like coming across a very expensive computer which occasionally calculates the equivalent of 1*1 = inf, simply because some physical gates have slightly more electrical charge on them due to RF from a power supply that's 2 feet away.
    - eightysixfour 2y ago
      I think you're mixing up two different things: the challenges of building these systems at scale, and their fundamental properties. Take your example of the expensive computer returning 1*1 = inf because of a nearby power supply - that actually proves my point about computers being knowable systems. You were able to track down that specific environmental interference precisely because computers are built on logic with explicit rules and dependencies. When these types of errors are caught, we know because they do not conform to the rules of the system, which are explicitly defined, by us. We can measure and understand their failures exactly because we designed them. Even massive distributed systems, while complex, still follow explicit rules for how they change state. Every bit of information exists in a measurable form somewhere. Sure, at Google scale we might not have tools to capture everything at once, and no single person could follow every step from electrical signal to final output. But it's theoretically possible - which is fundamentally different from natural systems. You could argue the universe itself is deterministic (and philosophically, I agree), but in practice, the emergent systems we deal with - like biology or economics - follow rules we can't fully describe, using information we can't fully measure, where complete state capture isn't just impractical, it's impossible.
BeetleB 2y ago
> when in reality this field is one of the strictest in terms of statistical procedures like pre registeration, blinding, multiple hypothesis testing etc I'm not in HEP, but my graduate work had overlap with condensed matter physics. I worked with physics professors/students in a top 10 physics school (which had Nobel laureates, although I didn't work with them). Things may have changed since then, but the majority of them had no idea what pre-registration meant, and none had taken a course on statistics. In most US universities, statistics is not required for a physics degree (although it is for an engineering one). When I probed them, the response was "Why should we take a whole course on it? We study what we need in quantum mechanics courses." No, my friend. You studied probability. Not statistics. Whatever you can say about reproducibility in the social sciences, a typical professor in those fields knew and understood an order of magnitude more statistics than physicists.
- noslenwerdna 2y ago
  As an ex-HEP, I can confirm that yes, we had blinding and did correct for multiple hypothesis testing explicitly. As Kyle Cranmer points out, we called it the "look elsewhere effect." Blinding is enforced by the physics group. You are not allowed to look at a signal region until you have basically finished your analysis. For pre-registration, this might be debatable, but what I meant was that we have teams of people looking for specific signals (SUSY, etc). Each of those teams would have generated monte carlo simulations of their signals and compared those with backgrounds. Generally speaking, analysis teams were looking for something specific in the data. However, there are sometimes more general "bump hunts", which you could argue didn't have preregistration. But on the other hand, they are generally looking for bumps with a specific signature (say, two leptons). So yes, people in HEP generally are knowledgeable about stats... and yes, this field is extremely strict compared to psychology for example.

exmadscientist 2y ago

> so complicated as those in modern particle physics But... modern particle physics is one of the simplest things around. (Ex-physicist here, see username.) It only looks complicated because it is so simple that we can actually write down every single detail of the entire thing and analyze it! How many other systems can you say that about?

spookie 2y ago
Other systems might not be part of a field as mature as yours, I would argue.
- exmadscientist 2y ago
  It has nothing to do with "maturity" and everything to do with just hierarchy in general. There is something to the old XKCD joke: https://xkcd.com/435/ https://xkcd.com/435/ because the disciplines really are divided like that. You have to know physics to do chemistry well. You have to know chemistry to do biology well. You have to know biology to ... etc. Whereas to do physics well you need only mathematics. Well, at least, to do the theories well. To actually execute the experiments is, ah, more challenging. So I would argue the Standard Model is pretty much the only thing in all of human knowledge that depends on no other physical theories. It's the bottom. Shame it's pretty useless (intractable) as soon as you have three or more particles to calculate with, though....
deleted 2y ago
[deleted]

jaculabilis 2y ago

> I think the author is using the original motivation of musing on null hypotheses to derive the title "The Higgs Discovery Did Not Take Place", It's probably a reference to "The Gulf War Did Not Take Place" by Jean Baudrillard, which took a similar critical view of the Gulf War as TFA takes of the Higgs discovery.

mellosouls 2y ago
Possibly! I remember that but completely missed it as an inspiration here.