6 ms·

Losing the Nobel Prize

110 points by chmaynard 9y ago

forapurpose 9y ago
The article seems one-sided and, from the outside, I don't know what I don't know. Can someone comment on other substantive points of view? And was BICEP2 really as reckless/dishonest as is portrayed? EDIT: Some other perspectives in the comments: > ... part of what gets overlooked in some of this is that BICEP2 had done lovely work — they had great data at far higher sensitivity than any previous polarisation experiment, if I am recalling things correctly; the analysis was flawed but the data was good. > PRL referees saved the BICEP2 people from themselves by getting them to remove the worst of their mistakes. If the referees had quickly let the paper through, likely the paper would have had to have been formally retracted. > I did some searching around for examples of any apology from the BICEP2 people or negative effects on any of their careers, could find nothing. A quite remarkable fact is that Kovac’s Wikipedia entry still has the following as description of his career: / “He is the principal investigator of the BICEP2 telescope, which is part of the BICEP and Keck Array series of experiments.[3][4][5] Measurements announced on 17 March 2014 from the BICEP2 telescope give support to the idea of cosmic inflation, by reporting the first evidence for a primordial B-Mode pattern in the polarization of the CMB.[6][7][8] If confirmed, this measurement provides a direct image of primordial gravitational waves and the quantization of gravity.” > If Keating is complaining about how a single prize determines research (ehm… does it really?) then he’d better complain about tycoons determining research by handing out money to people from failed experiments. ... The latter is a troubling sign to me. It was predictable (I predicted it, at least) that as governments cut taxes and cut funding to universities and science, more power would concentrate in the hands of a wealthy few (private and corporate) and research would become far more dependent on pleasing them. What value is research without independence? Arguably it does more harm than good, by creating disinformation and undermining legitimate research. EDIT2: Added a key comma in the first sentence
- tombone12 9y ago
  I don't know how to estimate their reckless from the outside, but it is simple fact that however sensitive, they only had one waveband and so no real way of estimating the foreground contamination. I think it is also well established that the foreground estimation was made by looking closely at pictures from a presentation showing preliminary Planck skymaps of the dust emission, and then building a foreground model from that. They also put out this adorable video as part of the announcement, maybe not the greatest show of caution ever: https://www.youtube.com/watch?v=ZlfIVEy_YOA https://www.youtube.com/watch?v=ZlfIVEy_YOA
- sytelus 9y ago
  There seems to be plenty of evidence that almost everyone in the community thought that claims made by bicep2 was reckless and hyped for the purpose of generating the news headlines. They didn’t even disclosed clearly about possiblity that claims could be wrong even when they full well knew they could. A reviewer forced them to add that possibility in that paper which they only did grudgingly. If it wasn’t for the reviewer, the paper would have been now looked as outright lie. This was one of the most sought after experimental result and it was expected that team would be very careful in making any claims. So when big claims were indeed made, it didn’t took too lonng to do independent review and refute their claims. https://phys.org/news/2014-06-experts-big-bolstering-discovery.html https://phys.org/news/2014-06-experts-big-bolstering-discove... https://dreamheron.wordpress.com/category/bicep2-cmb/page/6/ https://dreamheron.wordpress.com/category/bicep2-cmb/page/6/ http://blogs.discovermagazine.com/crux/2015/01/30/bicep2-wrong-sharing-results/#.Wtp6qsqxWf0 http://blogs.discovermagazine.com/crux/2015/01/30/bicep2-wro... https://www.washingtonpost.com/national/health-science/bicep2-experiments-big-bang-controversy-highlights-challenges-for-modern-science/2014/07/23/707bc9e6-02c6-11e4-b8ff-89afd3fad6bd_story.html?noredirect=on&utm_term=.efdd48fe9e84 https://www.washingtonpost.com/national/health-science/bicep...
lkrubner 9y ago
About this: "As far as I can tell, the BICEP2 scientists haven’t suffered much professionally from the fiasco. When David Spergel talked here at Columbia about the subject, he noted that this hadn’t stopped the PI, John Kovac, from getting tenure at Harvard. In the book, Keating mentions some “embarrassment and guilt”, but no negative professional consequences" In the long run, it is good for society if researchers can make mistakes and keep their careers. In the long run, it is good for society if journalists are sometimes embarrassed by reporting on a story when they don't have access to the data behind the story. So in the long run, it is probably for the best that these researchers suffered no adverse effects. If anyone changes their behavior, I'd rather it be the journalists. They should not give too much publicity to teams making big announcements while hiding their data.
- jackfrodo 9y ago
  Is this a reasonable thing to ask of journalists? I can't imagine a reporter telling their editor, "Yes, this team tells us with confidence that their findings have confirmed a huge story, but I disagree with their interpretation of the data, so I don't think we should run it." A reporter shouldn't be embarrassed that a team of scientists lied to them. Obviously the publications should retract/update these stories, but journalists, even ones with science backgrounds just aren't equipped and trained to be able to independently verify every claim they report on.
  - GCA10 9y ago
    The best science reporters earn their keep by balancing "wow!" and "really?" at moments like this. Sometimes it's a matter of spotting an outright hoax (i.e. Piltdown Man fossils in the 1920s.) Other times, what's at stake is an overly decisive attempt to interpret exciting data that might not be so exciting after all. The cold-fusion "discoveries" of 1989 fall into that camp, along with this gravitational-wave claim. It's hard for journalists to referee everything in the first few hours. But by talking to other experts in the field, and by being willing to comb through the methodology underlying the claims, it's possible to get a sense within a few days (weeks?) of what's holding together very, very well, and what's starting to seem a little shaky. There's nothing wrong -- and a lot right -- with writing post-announcement articles that acknowledge such uncertainties.
bagrow 9y ago
I’ve often wondered: Should the number of Nobel prizes awarded scale with the population? There are far more scientists working today than 50 or 100 years ago, and cappping the number of winners makes them more prestigious over time. Perhaps they will become TOO prestigious...
- SiempreViernes 9y ago
  The fact that they can’t be awarded to a collective rather detracts from the prestige, because for big experiments it is really the collaboratin as a whole that made the discovery and not some guy at the top.
  - ItsMe000001 9y ago
    That is true for any discovery after the stone hand ax. A "true" discovery that can be ascribed to one person 100% instead of to a network of people working jointly in space and time (the dead are involved too!) needs that person to work without using knowledge as well as any tools created by others (in space as well as in time). Humanity is more like a brain - the secret of our success is in the connections between us (again, in space as well as in time). My own conclusion, but I found this in support: https://youtu.be/jaoQh6BoH3c https://youtu.be/jaoQh6BoH3c ("Henrich is professor of human evolutionary biology at Harvard University.") I also recommend this - and please ignore which concrete product was more or less randomly chosen, it does no matter: "What Coke Contains" - https://medium.com/@kevin_ashton/what-coke-contains-221d449929ef https://medium.com/@kevin_ashton/what-coke-contains-221d4499... > The number of individuals who know how to make a can of Coke is zero. The number of individual nations that could produce a can of Coke is zero. ... Invention and creation is something we are all in together. Modern tool chains are so long and complex that they bind us into one people and one planet. They are not only chains of tools, they are also chains of minds: local and foreign, ancient and modern, living and dead — the result of disparate invention and intelligence distributed over time and space. ---- Same in business. Each time you hear about an extremely well-paid executive or a "self-made(!!?) billionaire" remind yourself that those people's main "accomplishment" is to live today. How much of the network effects (of the human network in space and in time(!)) does one person "deserve"? I would tell those "self-made" people complaining about taxation that if they go into the deep forest and stop using the human network - which is everybody else (let's even give them access to the dead people's achievements in the form of vast knowledge) - they can keep 100% of the grass and mud huts they manage to create. Humanities wealth and more important its ability to create more of it is almost all network effects. If all the people alive would work a) in isolation and b) without anything they got from previous generations, what would we accomplish? Einstein was a genius, so was Newton. What would they have accomplished a) without the exchange with their peers (network in space), b) without building on prior knowledge (network in time)? So, who deserves that Nobel Prize? That billion dollar empire? I don't mind any of those existing - I'm not an "evil communist". I do mind when people get a bit too excited about "deserving" and "stealing from me" (taxes). Relax and remember that you are one lucky bastard, able to benefit from network effects that you did next to nothing to help create (because most of it was and is created by everybody else). My favorite 5s of Clint Eastwood where he summarizes it for me: https://youtu.be/OuvEJ-U1UDc https://youtu.be/OuvEJ-U1UDc
tombone12 9y ago
Go read Hossenfelder's review instead, its a better teardown. The fourth paragraph is: > “Losing the Nobel Prize” is well written and engaging and has a lot of figures and, ah, here I run out of nice things to say. But we all know you didn’t come for the nice things anyway, so let’s get to the beef.
- kn0where 9y ago
  Link for the lazy: http://backreaction.blogspot.com/2018/04/book-review-losing-nobel-prize-by-brian.html?spref=tw http://backreaction.blogspot.com/2018/04/book-review-losing-...
- Luc 9y ago
  Brian Keating's reaction to the review: http://backreaction.blogspot.be/2018/04/guest-post-brian-keating-about-his-book.html http://backreaction.blogspot.be/2018/04/guest-post-brian-kea... If you ask me, he's just digging deeper, especially when taking into consideration Sabine Hossenfelder's remarks within his reply.
- mannykannot 9y ago
  The linked article makes some very pertinent additional points (as well as offering a counter to Keating's claim that Hossenfelder was unfairly reviewing an uncorrected proof, though Hossenfelder herself makes a good case for that being a non-sequitur.) For example, there is the part about how they needed data from the Planck team, and tried to get it without saying what they were doing, which is how they ended up scraping data, that they apparently did not fully understand, off a presentation slide.
KKKKkkkk1 9y ago
But there is one feature I notice that is generally missing in Cargo Cult Science. That is the idea that we all hope you have learned in studying science in school—we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty—a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they have been eliminated. Richard Feynman
- truculation 9y ago
  Feynman again, on the topic of prizes: https://www.youtube.com/watch?v=AEUcmKDaklY https://www.youtube.com/watch?v=AEUcmKDaklY So the real prize is seeing one's work used by others in their work.
- YeGoblynQueenne 9y ago
  That's great in principle. In practice, to keep doing the work of science you need to keep getting funding; and to keep getting funding you need to keep publishing (read: to respectable conferences and journals). Unfortunately, the most respectable journals have a nasty tendency to refuse to publish negative, or even equivocal results. And that puts a lot of pressure on researchers to find a signal, somehow, anyhow- and to viciously attack any suspicion that the signal was just a spider on the lens or whatever. Which is not to say Feynamn is wrong. On the contrary: reform is sorely needed. But reform can only come from the inside, and the only people "inside" are those who accept they must play by the rules, rotten as they are. Bottom line: we all want to Do the Right Thing, For Science. But doing the right thing and still doing science is getting harder and harder as time goes by.
ggg9990 9y ago
Ugh, I always hate hearing the griping from the people who didn't win the Nobel Prize. Get over it, it's not an objective listing of the best scientists of all time. It's a human-driven process driven as much by circumstance as by talent. It's like politicians griping about losing elections or middle managers talking about being skipped over for a promotion.

110 points by chmaynard 9y ago

The article seems one-sided and, from the outside, I don't know what I don't know. Can someone comment on other substantive points of view? And was BICEP2 really as reckless/dishonest as is portrayed? EDIT: Some other perspectives in the comments: > ... part of what gets overlooked in some of this is that BICEP2 had done lovely work — they had great data at far higher sensitivity than any previous polarisation experiment, if I am recalling things correctly; the analysis was flawed but the data was good. > PRL referees saved the BICEP2 people from themselves by getting them to remove the worst of their mistakes. If the referees had quickly let the paper through, likely the paper would have had to have been formally retracted. > I did some searching around for examples of any apology from the BICEP2 people or negative effects on any of their careers, could find nothing. A quite remarkable fact is that Kovac’s Wikipedia entry still has the following as description of his career: / “He is the principal investigator of the BICEP2 telescope, which is part of the BICEP and Keck Array series of experiments.[3][4][5] Measurements announced on 17 March 2014 from the BICEP2 telescope give support to the idea of cosmic inflation, by reporting the first evidence for a primordial B-Mode pattern in the polarization of the CMB.[6][7][8] If confirmed, this measurement provides a direct image of primordial gravitational waves and the quantization of gravity.” > If Keating is complaining about how a single prize determines research (ehm… does it really?) then he’d better complain about tycoons determining research by handing out money to people from failed experiments. ... The latter is a troubling sign to me. It was predictable (I predicted it, at least) that as governments cut taxes and cut funding to universities and science, more power would concentrate in the hands of a wealthy few (private and corporate) and research would become far more dependent on pleasing them. What value is research without independence? Arguably it does more harm than good, by creating disinformation and undermining legitimate research. EDIT2: Added a key comma in the first sentence

tombone12 9y ago
I don't know how to estimate their reckless from the outside, but it is simple fact that however sensitive, they only had one waveband and so no real way of estimating the foreground contamination. I think it is also well established that the foreground estimation was made by looking closely at pictures from a presentation showing preliminary Planck skymaps of the dust emission, and then building a foreground model from that. They also put out this adorable video as part of the announcement, maybe not the greatest show of caution ever: https://www.youtube.com/watch?v=ZlfIVEy_YOA https://www.youtube.com/watch?v=ZlfIVEy_YOA
sytelus 9y ago
There seems to be plenty of evidence that almost everyone in the community thought that claims made by bicep2 was reckless and hyped for the purpose of generating the news headlines. They didn’t even disclosed clearly about possiblity that claims could be wrong even when they full well knew they could. A reviewer forced them to add that possibility in that paper which they only did grudgingly. If it wasn’t for the reviewer, the paper would have been now looked as outright lie. This was one of the most sought after experimental result and it was expected that team would be very careful in making any claims. So when big claims were indeed made, it didn’t took too lonng to do independent review and refute their claims. https://phys.org/news/2014-06-experts-big-bolstering-discovery.html https://phys.org/news/2014-06-experts-big-bolstering-discove... https://dreamheron.wordpress.com/category/bicep2-cmb/page/6/ https://dreamheron.wordpress.com/category/bicep2-cmb/page/6/ http://blogs.discovermagazine.com/crux/2015/01/30/bicep2-wrong-sharing-results/#.Wtp6qsqxWf0 http://blogs.discovermagazine.com/crux/2015/01/30/bicep2-wro... https://www.washingtonpost.com/national/health-science/bicep2-experiments-big-bang-controversy-highlights-challenges-for-modern-science/2014/07/23/707bc9e6-02c6-11e4-b8ff-89afd3fad6bd_story.html?noredirect=on&utm_term=.efdd48fe9e84 https://www.washingtonpost.com/national/health-science/bicep...

lkrubner 9y ago

About this: "As far as I can tell, the BICEP2 scientists haven’t suffered much professionally from the fiasco. When David Spergel talked here at Columbia about the subject, he noted that this hadn’t stopped the PI, John Kovac, from getting tenure at Harvard. In the book, Keating mentions some “embarrassment and guilt”, but no negative professional consequences" In the long run, it is good for society if researchers can make mistakes and keep their careers. In the long run, it is good for society if journalists are sometimes embarrassed by reporting on a story when they don't have access to the data behind the story. So in the long run, it is probably for the best that these researchers suffered no adverse effects. If anyone changes their behavior, I'd rather it be the journalists. They should not give too much publicity to teams making big announcements while hiding their data.

jackfrodo 9y ago
Is this a reasonable thing to ask of journalists? I can't imagine a reporter telling their editor, "Yes, this team tells us with confidence that their findings have confirmed a huge story, but I disagree with their interpretation of the data, so I don't think we should run it." A reporter shouldn't be embarrassed that a team of scientists lied to them. Obviously the publications should retract/update these stories, but journalists, even ones with science backgrounds just aren't equipped and trained to be able to independently verify every claim they report on.
- GCA10 9y ago
  The best science reporters earn their keep by balancing "wow!" and "really?" at moments like this. Sometimes it's a matter of spotting an outright hoax (i.e. Piltdown Man fossils in the 1920s.) Other times, what's at stake is an overly decisive attempt to interpret exciting data that might not be so exciting after all. The cold-fusion "discoveries" of 1989 fall into that camp, along with this gravitational-wave claim. It's hard for journalists to referee everything in the first few hours. But by talking to other experts in the field, and by being willing to comb through the methodology underlying the claims, it's possible to get a sense within a few days (weeks?) of what's holding together very, very well, and what's starting to seem a little shaky. There's nothing wrong -- and a lot right -- with writing post-announcement articles that acknowledge such uncertainties.

bagrow 9y ago

I’ve often wondered: Should the number of Nobel prizes awarded scale with the population? There are far more scientists working today than 50 or 100 years ago, and cappping the number of winners makes them more prestigious over time. Perhaps they will become TOO prestigious...

SiempreViernes 9y ago
The fact that they can’t be awarded to a collective rather detracts from the prestige, because for big experiments it is really the collaboratin as a whole that made the discovery and not some guy at the top.
- ItsMe000001 9y ago
  That is true for any discovery after the stone hand ax. A "true" discovery that can be ascribed to one person 100% instead of to a network of people working jointly in space and time (the dead are involved too!) needs that person to work without using knowledge as well as any tools created by others (in space as well as in time). Humanity is more like a brain - the secret of our success is in the connections between us (again, in space as well as in time). My own conclusion, but I found this in support: https://youtu.be/jaoQh6BoH3c https://youtu.be/jaoQh6BoH3c ("Henrich is professor of human evolutionary biology at Harvard University.") I also recommend this - and please ignore which concrete product was more or less randomly chosen, it does no matter: "What Coke Contains" - https://medium.com/@kevin_ashton/what-coke-contains-221d449929ef https://medium.com/@kevin_ashton/what-coke-contains-221d4499... > The number of individuals who know how to make a can of Coke is zero. The number of individual nations that could produce a can of Coke is zero. ... Invention and creation is something we are all in together. Modern tool chains are so long and complex that they bind us into one people and one planet. They are not only chains of tools, they are also chains of minds: local and foreign, ancient and modern, living and dead — the result of disparate invention and intelligence distributed over time and space. ---- Same in business. Each time you hear about an extremely well-paid executive or a "self-made(!!?) billionaire" remind yourself that those people's main "accomplishment" is to live today. How much of the network effects (of the human network in space and in time(!)) does one person "deserve"? I would tell those "self-made" people complaining about taxation that if they go into the deep forest and stop using the human network - which is everybody else (let's even give them access to the dead people's achievements in the form of vast knowledge) - they can keep 100% of the grass and mud huts they manage to create. Humanities wealth and more important its ability to create more of it is almost all network effects. If all the people alive would work a) in isolation and b) without anything they got from previous generations, what would we accomplish? Einstein was a genius, so was Newton. What would they have accomplished a) without the exchange with their peers (network in space), b) without building on prior knowledge (network in time)? So, who deserves that Nobel Prize? That billion dollar empire? I don't mind any of those existing - I'm not an "evil communist". I do mind when people get a bit too excited about "deserving" and "stealing from me" (taxes). Relax and remember that you are one lucky bastard, able to benefit from network effects that you did next to nothing to help create (because most of it was and is created by everybody else). My favorite 5s of Clint Eastwood where he summarizes it for me: https://youtu.be/OuvEJ-U1UDc https://youtu.be/OuvEJ-U1UDc

tombone12 9y ago

Go read Hossenfelder's review instead, its a better teardown. The fourth paragraph is: > “Losing the Nobel Prize” is well written and engaging and has a lot of figures and, ah, here I run out of nice things to say. But we all know you didn’t come for the nice things anyway, so let’s get to the beef.

kn0where 9y ago
Link for the lazy: http://backreaction.blogspot.com/2018/04/book-review-losing-nobel-prize-by-brian.html?spref=tw http://backreaction.blogspot.com/2018/04/book-review-losing-...
Luc 9y ago
Brian Keating's reaction to the review: http://backreaction.blogspot.be/2018/04/guest-post-brian-keating-about-his-book.html http://backreaction.blogspot.be/2018/04/guest-post-brian-kea... If you ask me, he's just digging deeper, especially when taking into consideration Sabine Hossenfelder's remarks within his reply.
mannykannot 9y ago
The linked article makes some very pertinent additional points (as well as offering a counter to Keating's claim that Hossenfelder was unfairly reviewing an uncorrected proof, though Hossenfelder herself makes a good case for that being a non-sequitur.) For example, there is the part about how they needed data from the Planck team, and tried to get it without saying what they were doing, which is how they ended up scraping data, that they apparently did not fully understand, off a presentation slide.

KKKKkkkk1 9y ago

But there is one feature I notice that is generally missing in Cargo Cult Science. That is the idea that we all hope you have learned in studying science in school—we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty—a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they have been eliminated. Richard Feynman

truculation 9y ago
Feynman again, on the topic of prizes: https://www.youtube.com/watch?v=AEUcmKDaklY https://www.youtube.com/watch?v=AEUcmKDaklY So the real prize is seeing one's work used by others in their work.
YeGoblynQueenne 9y ago
That's great in principle. In practice, to keep doing the work of science you need to keep getting funding; and to keep getting funding you need to keep publishing (read: to respectable conferences and journals). Unfortunately, the most respectable journals have a nasty tendency to refuse to publish negative, or even equivocal results. And that puts a lot of pressure on researchers to find a signal, somehow, anyhow- and to viciously attack any suspicion that the signal was just a spider on the lens or whatever. Which is not to say Feynamn is wrong. On the contrary: reform is sorely needed. But reform can only come from the inside, and the only people "inside" are those who accept they must play by the rules, rotten as they are. Bottom line: we all want to Do the Right Thing, For Science. But doing the right thing and still doing science is getting harder and harder as time goes by.

ggg9990 9y ago

Ugh, I always hate hearing the griping from the people who didn't win the Nobel Prize. Get over it, it's not an objective listing of the best scientists of all time. It's a human-driven process driven as much by circumstance as by talent. It's like politicians griping about losing elections or middle managers talking about being skipped over for a promotion.