7 ms·

The first pre-print came out on 22 July (if I am not mistaken). Not enough time has passed pure and simple. If I was an academic working on this, I would be fo

The first pre-print came out on 22 July (if I am not mistaken). Not enough time has passed pure and simple. If I was an academic working on this, I would be following the Sagan Standard and triple checking my results before claiming anything. I think we are all suffering from 24 hour news cycles. The work in China and now Varda is very exciting but they are warp speed developments.

anonylizard 3y ago
This is simply what post-peer-review looks like. Instead of a few anonymous peer reviews, you get the entire planet to replicate. Moment you get something meaningful, publish it on arxiv. Quantity, not quality is king. Anomalies are drowned out by large samples. Questionable methods get mass debated in public. Scientists are dependent on public funding, why squander the biggest moment in pop science for literal decades? Now this won't work for 99.99% of research. But top tier research will probably get published on youtube and arxiv in the future...
- CharlieDigital 3y ago
  Not all meaningful science is this level of groundbreaking for this model to work. Formal peer review is still very much necessary.
- ironborn123 3y ago
  This kind of public interest and demonstrations is unique only to RTS and few other phenomena - ordinary ppl more or less understand the importance of RTS - a visible and understandable effect (Meissner) can be shown on successful replication - the materials required are easier to understand, but the synthesis procedure is tricky, and purity is important - The skills of chemists comes into play, leading to kind of a Breaking Bad feel
- svara 3y ago
  > Questionable methods get mass debated in public. By people who don't know what they're talking about. You're seriously underestimating the level of expertise that it takes to professionally assess most research. It wouldn't be unusual for there to be a small handful of people with that ability in the world.
  - generalizations 3y ago
    What matters is the discovery of new things that are true, that we can do engineering with. Publication by youtube & arxiv doesn't have to be oriented at public consumption - it works just as well when it's by-researchers, for-researchers. Except without gatekeeping, people like McCalip can participate too. Who cares who's sitting on the sidelines? In post-peer-review, industry-funded academics rush a paper to a preprint server, a space engineer replicates it, and someone else figures out how to mass-produce it, and we get room-temp ambient-pressure superconductors, and we know we did because the engineering works. This works just as well with less exciting tech, because the point is that we collaboratively learned how to build something.
    - MetaWhirledPeas 3y ago
      Agreed. What I don't understand is the gatekeeping. Why would the actions of outsiders have any bearing on "real" research? Are they worried that grant money will get redirected toward shallower, more glamorous groups?
      - dotnet00 3y ago
        I think the gatekeeping is relevant because almost every time in recent years that the general completely unvetted, often actively ignorant public has gotten an in on the scientific/R&D process, it has been a disaster for the field. NASA can't take risks and seriously innovate anymore because even intentional explosions will be passed off as failures, we all experienced the sheer madness of the covid years, similarly anything nuclear brings complete randos out of the woodworks claiming absurd things about its dangers. Not having some amount of gatekeeping just lowers the bar too much and introduces so much more noise.
        MetaWhirledPeas 3y ago
        > Not having some amount of gatekeeping just lowers the bar too much and introduces so much more noise. I can see how this would be a very controversial subject. Discouraging scientific participation? That's a tough pill to swallow without a lot of justification.
- mcpackieh 3y ago
  > This is simply what post-peer-review looks like. Peer review was never a substitute for replication in the first place. In this sense we've been "post-peer-review" the entire time.
- Waterluvian 3y ago
  Only if you’re doing sexy science. The peer review system is broken but I think we still need some sort of structured system in addition to this “hey world, check this out!” method.
- jcranmer 3y ago
  There is a new claim or two of room-temperature superconductors published on Arxiv every year: https://nitter.net/peternemes/status/1684483863999827973#m https://nitter.net/peternemes/status/1684483863999827973#m... and yet this is the only one that sets the Twitters afire. The reaction I've seen from most of the scientists has been pretty negative on the papers. The originals aren't strong papers. The chemistry has issues: https://nitter.net/Robert_Palgrave/status/1685450156806918144#m https://nitter.net/Robert_Palgrave/status/168545015680691814... (unbalanced equations, starting materials have pretty different element ratios to final products). As for the main claim, note that LK-99 seems to be a poorer conductor than copper wire at room temperature: https://nitter.net/bedoya_pinto/status/1686848392616423425#m https://nitter.net/bedoya_pinto/status/1686848392616423425#m... which is the kind of thing that should really cause a record scratch and make everyone go "um, I think you did something wrong there"--it's something that absolutely would be caught in peer review.
  - notfish 3y ago
    I’m not sure I understand your point. Are you trying to say that people shouldn’t be paying attention to this? I would personally be horrified if I found out that the paper on the synthesis of a room temperature superconductor went through the normal peer review process instead of letting the world know they discovered something huge.
    - jcranmer 3y ago
      My point is why should this unreviewed claim of ambient condition superconductor be exalted as the holy grail and all of the other unreviewed claims of ambient condition superconductor languish in obscurity when they come out? It's definitely not for the strength of evidence!
      - marcosdumay 3y ago
        It's pretty much due to the strength of evidence. There is no conclusive proof that this is a superconductor on the replications, but the evidence has been on a different level from nearly everything else you'll find on Arxiv since the beginning.
- lamontcg 3y ago
  > top tier research will probably get published on youtube and arxiv in the future... "Top tier science" is already published on arXiv. The future happened back in 1995 or something like that. > Quantity, not quality is king. No, quality is still king. We don't need videos of flakes standing on their end we need someone who has been working with superconductors all their lives who is competent enough to measure zero resistance across the sample. If one of the wild theories is right we need someone who actually knows how to prove it and not just sound like they're intelligent by talking about 1-d superconductors.
- marcosdumay 3y ago
  > Quantity, not quality is king. Anomalies are drowned out by large samples. Yeah, that's not how it works. At practice, anomalies are hyped into everybody's attention, and large samples are drowned out by lack of novelty. You don't need questionable methods to get bad results, you just need quantity. All the things that are happening are great, and this may be an astonishing discovery. But don't put more certainty on it than it's warranted.
- codethief 3y ago
  > Instead of a few anonymous peer reviews, you get the entire planet to replicate. Notably, peer reviews typically don't involve reviewers trying to replicate the results. So we've essentially skipped peer review (at least temporarily) and went full speed ahead with the post-peer-review process.
- Accujack 3y ago
  >This is simply what post-peer-review looks like. No, this is a new face for peer review. Existing peer review processes have stagnated in the information age, possibly with good reason. There are a lot of advantages to more immediate review of papers and data, and also to bypassing the systems of qualifications presently used to choose who is a valid reviewer.
lumost 3y ago
Speed is correlated with execution quality. All else being equal, A lab which is better equipped - with better access to materials, and motivated and competent researchers will deliver before the equivalent lab which is deficient in any of the above. A twitter post isn't peer review, but it is a nice way of sharing part of the process. A video etc. will help others in their attempts to replicate.
moolcool 3y ago
> If I was an academic working on this, I would be following the Sagan Standard and triple checking my results before claiming anything. Counterpoint: It's better that a lot of people know "Hey, we might be onto something interesting here" early on.

anonylizard 3y ago

This is simply what post-peer-review looks like. Instead of a few anonymous peer reviews, you get the entire planet to replicate. Moment you get something meaningful, publish it on arxiv. Quantity, not quality is king. Anomalies are drowned out by large samples. Questionable methods get mass debated in public. Scientists are dependent on public funding, why squander the biggest moment in pop science for literal decades? Now this won't work for 99.99% of research. But top tier research will probably get published on youtube and arxiv in the future...

CharlieDigital 3y ago
Not all meaningful science is this level of groundbreaking for this model to work. Formal peer review is still very much necessary.
ironborn123 3y ago
This kind of public interest and demonstrations is unique only to RTS and few other phenomena - ordinary ppl more or less understand the importance of RTS - a visible and understandable effect (Meissner) can be shown on successful replication - the materials required are easier to understand, but the synthesis procedure is tricky, and purity is important - The skills of chemists comes into play, leading to kind of a Breaking Bad feel
svara 3y ago
> Questionable methods get mass debated in public. By people who don't know what they're talking about. You're seriously underestimating the level of expertise that it takes to professionally assess most research. It wouldn't be unusual for there to be a small handful of people with that ability in the world.
- generalizations 3y ago
  What matters is the discovery of new things that are true, that we can do engineering with. Publication by youtube & arxiv doesn't have to be oriented at public consumption - it works just as well when it's by-researchers, for-researchers. Except without gatekeeping, people like McCalip can participate too. Who cares who's sitting on the sidelines? In post-peer-review, industry-funded academics rush a paper to a preprint server, a space engineer replicates it, and someone else figures out how to mass-produce it, and we get room-temp ambient-pressure superconductors, and we know we did because the engineering works. This works just as well with less exciting tech, because the point is that we collaboratively learned how to build something.
  - MetaWhirledPeas 3y ago
    Agreed. What I don't understand is the gatekeeping. Why would the actions of outsiders have any bearing on "real" research? Are they worried that grant money will get redirected toward shallower, more glamorous groups?
    - dotnet00 3y ago
      I think the gatekeeping is relevant because almost every time in recent years that the general completely unvetted, often actively ignorant public has gotten an in on the scientific/R&D process, it has been a disaster for the field. NASA can't take risks and seriously innovate anymore because even intentional explosions will be passed off as failures, we all experienced the sheer madness of the covid years, similarly anything nuclear brings complete randos out of the woodworks claiming absurd things about its dangers. Not having some amount of gatekeeping just lowers the bar too much and introduces so much more noise.
      - MetaWhirledPeas 3y ago
        > Not having some amount of gatekeeping just lowers the bar too much and introduces so much more noise. I can see how this would be a very controversial subject. Discouraging scientific participation? That's a tough pill to swallow without a lot of justification.
mcpackieh 3y ago
> This is simply what post-peer-review looks like. Peer review was never a substitute for replication in the first place. In this sense we've been "post-peer-review" the entire time.
Waterluvian 3y ago
Only if you’re doing sexy science. The peer review system is broken but I think we still need some sort of structured system in addition to this “hey world, check this out!” method.
jcranmer 3y ago
There is a new claim or two of room-temperature superconductors published on Arxiv every year: https://nitter.net/peternemes/status/1684483863999827973#m https://nitter.net/peternemes/status/1684483863999827973#m... and yet this is the only one that sets the Twitters afire. The reaction I've seen from most of the scientists has been pretty negative on the papers. The originals aren't strong papers. The chemistry has issues: https://nitter.net/Robert_Palgrave/status/1685450156806918144#m https://nitter.net/Robert_Palgrave/status/168545015680691814... (unbalanced equations, starting materials have pretty different element ratios to final products). As for the main claim, note that LK-99 seems to be a poorer conductor than copper wire at room temperature: https://nitter.net/bedoya_pinto/status/1686848392616423425#m https://nitter.net/bedoya_pinto/status/1686848392616423425#m... which is the kind of thing that should really cause a record scratch and make everyone go "um, I think you did something wrong there"--it's something that absolutely would be caught in peer review.
- notfish 3y ago
  I’m not sure I understand your point. Are you trying to say that people shouldn’t be paying attention to this? I would personally be horrified if I found out that the paper on the synthesis of a room temperature superconductor went through the normal peer review process instead of letting the world know they discovered something huge.
  - jcranmer 3y ago
    My point is why should this unreviewed claim of ambient condition superconductor be exalted as the holy grail and all of the other unreviewed claims of ambient condition superconductor languish in obscurity when they come out? It's definitely not for the strength of evidence!
    - marcosdumay 3y ago
      It's pretty much due to the strength of evidence. There is no conclusive proof that this is a superconductor on the replications, but the evidence has been on a different level from nearly everything else you'll find on Arxiv since the beginning.
lamontcg 3y ago
> top tier research will probably get published on youtube and arxiv in the future... "Top tier science" is already published on arXiv. The future happened back in 1995 or something like that. > Quantity, not quality is king. No, quality is still king. We don't need videos of flakes standing on their end we need someone who has been working with superconductors all their lives who is competent enough to measure zero resistance across the sample. If one of the wild theories is right we need someone who actually knows how to prove it and not just sound like they're intelligent by talking about 1-d superconductors.
marcosdumay 3y ago
> Quantity, not quality is king. Anomalies are drowned out by large samples. Yeah, that's not how it works. At practice, anomalies are hyped into everybody's attention, and large samples are drowned out by lack of novelty. You don't need questionable methods to get bad results, you just need quantity. All the things that are happening are great, and this may be an astonishing discovery. But don't put more certainty on it than it's warranted.
codethief 3y ago
> Instead of a few anonymous peer reviews, you get the entire planet to replicate. Notably, peer reviews typically don't involve reviewers trying to replicate the results. So we've essentially skipped peer review (at least temporarily) and went full speed ahead with the post-peer-review process.
Accujack 3y ago
>This is simply what post-peer-review looks like. No, this is a new face for peer review. Existing peer review processes have stagnated in the information age, possibly with good reason. There are a lot of advantages to more immediate review of papers and data, and also to bypassing the systems of qualifications presently used to choose who is a valid reviewer.

lumost 3y ago

Speed is correlated with execution quality. All else being equal, A lab which is better equipped - with better access to materials, and motivated and competent researchers will deliver before the equivalent lab which is deficient in any of the above. A twitter post isn't peer review, but it is a nice way of sharing part of the process. A video etc. will help others in their attempts to replicate.

moolcool 3y ago

> If I was an academic working on this, I would be following the Sagan Standard and triple checking my results before claiming anything. Counterpoint: It's better that a lot of people know "Hey, we might be onto something interesting here" early on.