5 ms·

Your argument doesn't hold economically nor based on the state of the status quo. Economically it is infeasible to politically play, social engineer, or bribe

by j-pb 4y ago

Your argument doesn't hold economically nor based on the state of the status quo. Economically it is infeasible to politically play, social engineer, or bribe thousands of reviewers in a publicly inspectable web of trust. It's easily done so and done in a secretive environment with a handfull of people that all know each other, i.e. the current state of publishing. The current paper review is a farce, you just go conference and review shopping till it gets accepted, papers get shitgunned into half a dozen journals with minor changes, and reviewers often have no clue about contents and just check the general form, wave papers from friendlies through, or block competitors based on personal vendettas and funding rivalries. Publishing and review in a decentralized anonymised web of trust secured by crypto is probably the only valid use case for crypto and blockchain foo that I know of.

I_DRINK_KOOLAID 4y ago
> Economically it is infeasible to politically play, social engineer, or bribe thousands of reviewers in a publicly inspectable web of trust Elon Musk hyperloop whitepaper pipedream...he made it in one night, probably on heavy drugs. It got press for 8 years, still getting press. A bastardised version of Hyperloop got hundreds of millions in public funding in Las Vegas and Los Angeles for what essentially amounts to a large underground car garage with skeets. The company selling this bastardised version just raised 700M in equity. "infeasible". When you open the floodgates of the court of public opinion...it becomes the game of the person who screams the loudest and is able to con people with the most confidence or has the loudest megaphone. History has shown us time and time again that the best governance system is a bunch of elder statesmen who are terrified of being judged harshly by their peers long after they are gone. For example people still say about Newton that his track record wasn't immaculate because he wasted time looking for hidden messages in the Bible or trying to turn bronze into gold. Same for Einstein, his track record isn't immaculate because he was wrong on quantum-mechanics. Elder statesmen who dream of having the track record of a Pythagoras, that's the perfect governance system, that just requires finding them, doesn't require crypto-tokens or blockchain or any other flavor du jour tech.
- j-pb 4y ago
  A public web of trust network based on asymetric crypto with weighted reputation scores between scientific entities, is an entirely different thing than trending on twitter. Besides, the "paper" you cite is not a paper at all, yet plenty of universities had their fun with the concept, so if anything it shows that it's possible to get people excited to do study reproducing work after all.
- userulluipeste 4y ago
  "History has shown us time and time again that the best governance system is a bunch of elder statesmen who are terrified of being judged harshly by their peers long after they are gone." Those elders have a hard time steering the ship in the storm of technological change. Faced with that, at best, they will most likely try avoid implementing anything radical (due to their bias for what's familiar to them) and will instead attempt to integrate the new with minimal change to existing system (which makes sense as a strategy, if that happens to work). At worst, they will ignore or actively resist change, betting on the likelihood that it's about some overhype at play and things needs to settle a bit before being taken seriously. As sound as this may be as a governing strategy (in general, not only for handling things in academia), it comes however with the inherent risk of inflicting debilitating effect to governed realm's progress. To avoid this, the governance has to include visionaries, and a governing body including only (although assumed visionary, yet just) elders, with a blindspot in judging much of what's new and unfamiliar, will not do.
throwawaygh 4y ago
> Economically it is infeasible to politically play, social engineer, or bribe thousands of reviewers in a publicly inspectable web of trust. You're replacing peer review with a social media following. A large social media following is absolutely something you can buy. Which, by the way, has already happened. The easiest way to get an R1 job is to build a large twitter following. Getting a top 5 CS position without at least a few thousand twitter followers is basically impossible these days. High-prestige academia is just a high-brow social media influencer these days. None of this is meant as a defense of the old system, btw.
- j-pb 4y ago
  That's why it's important to make it a web of trust and not a plain "upvote"-"downvote" system. E.g. publishing a paper that gets positive reviews gives you some form of trust token that you can spend on other papers in a positive review. Ideally there is no single global metric for trust, but trust is computed relatively based on who you trust, so that everybody gets a personalized quality and relevance score for each paper. In a way you could also personalise your definition of trust. E.g. do you want to rank papers highly that have positive reviews by a handfull of highly trustwothy individuals, or do you trust papers that have many positive reviews.
  - throwawaygh 4y ago
    That's just another popularity contest with all of the exact same perverse incentives. How do you think people get on conference committees / journal boards? We should stop worrying about popularity contests (citation counts, prestige-signalling conference committees/paper venues, etc.). When we decide whether to retain an academic as a visiting scientist, consultant, etc., I am always the voice in the room encouraging folks to disregard everything except for one question: after having read 3 or 4 of this person's most relevant papers, do we think they can help us with the specific problems we face? When I sit on grant review committees I ignore all the academic prestige shit and ask: is this a real 10-15-30 year problem for a field, does that field matter to the funding agency's objectives, and does the description of the plan of attack have a good chance of success? I don't need to evaluate popularity contests to answer those questions. Popularity contests are low-effort short-cuts for the low-brow scientist/program manager. Best to ignore them if you have real problems and need real solutions.
ClumsyPilot 4y ago
> Economically it is infeasible to politically play, social engineer, or bribe thousands If this were true, we would not have snake oil salesmen, pyramid scheme leaders, drug lords or youtube celebrities You just want to turn academics into yet another category of celebrities that must game social media. We have enough of that shit
- j-pb 4y ago
  We have that in academia already. It's games day in and day out. We need those games to be transparent at least. And a web of trust is not the same as social media.
  - ClumsyPilot 4y ago
    Its not what we have in academia. And please show me where do we have a functioning web of trust
    - j-pb 4y ago
      > Its not what we have in academia. If you are blind to this then you are either in a very sheltered environment or you're part of the problem. We haven't build such a system yet, which is exactly the critique OP had. The broken funding process based on citation count and journal impact deincentivizes building and participating in such a system. It's all self enforcing with the publishers reaping in billions. It took years till european academia finally started to rebell against elsevier, and even now open access gets sabotaged by golden path open access publishing.
      - ClumsyPilot 4y ago
        If web of trust was a workable system, i wouod have been used in some industry by now - engineering, law, anything. But you criticism is based on a fantastical benchmark that was never seen in practice
        4y ago

by j-pb 4y ago

I_DRINK_KOOLAID 4y ago

> Economically it is infeasible to politically play, social engineer, or bribe thousands of reviewers in a publicly inspectable web of trust Elon Musk hyperloop whitepaper pipedream...he made it in one night, probably on heavy drugs. It got press for 8 years, still getting press. A bastardised version of Hyperloop got hundreds of millions in public funding in Las Vegas and Los Angeles for what essentially amounts to a large underground car garage with skeets. The company selling this bastardised version just raised 700M in equity. "infeasible". When you open the floodgates of the court of public opinion...it becomes the game of the person who screams the loudest and is able to con people with the most confidence or has the loudest megaphone. History has shown us time and time again that the best governance system is a bunch of elder statesmen who are terrified of being judged harshly by their peers long after they are gone. For example people still say about Newton that his track record wasn't immaculate because he wasted time looking for hidden messages in the Bible or trying to turn bronze into gold. Same for Einstein, his track record isn't immaculate because he was wrong on quantum-mechanics. Elder statesmen who dream of having the track record of a Pythagoras, that's the perfect governance system, that just requires finding them, doesn't require crypto-tokens or blockchain or any other flavor du jour tech.

j-pb 4y ago
A public web of trust network based on asymetric crypto with weighted reputation scores between scientific entities, is an entirely different thing than trending on twitter. Besides, the "paper" you cite is not a paper at all, yet plenty of universities had their fun with the concept, so if anything it shows that it's possible to get people excited to do study reproducing work after all.
userulluipeste 4y ago
"History has shown us time and time again that the best governance system is a bunch of elder statesmen who are terrified of being judged harshly by their peers long after they are gone." Those elders have a hard time steering the ship in the storm of technological change. Faced with that, at best, they will most likely try avoid implementing anything radical (due to their bias for what's familiar to them) and will instead attempt to integrate the new with minimal change to existing system (which makes sense as a strategy, if that happens to work). At worst, they will ignore or actively resist change, betting on the likelihood that it's about some overhype at play and things needs to settle a bit before being taken seriously. As sound as this may be as a governing strategy (in general, not only for handling things in academia), it comes however with the inherent risk of inflicting debilitating effect to governed realm's progress. To avoid this, the governance has to include visionaries, and a governing body including only (although assumed visionary, yet just) elders, with a blindspot in judging much of what's new and unfamiliar, will not do.

throwawaygh 4y ago

> Economically it is infeasible to politically play, social engineer, or bribe thousands of reviewers in a publicly inspectable web of trust. You're replacing peer review with a social media following. A large social media following is absolutely something you can buy. Which, by the way, has already happened. The easiest way to get an R1 job is to build a large twitter following. Getting a top 5 CS position without at least a few thousand twitter followers is basically impossible these days. High-prestige academia is just a high-brow social media influencer these days. None of this is meant as a defense of the old system, btw.

j-pb 4y ago
That's why it's important to make it a web of trust and not a plain "upvote"-"downvote" system. E.g. publishing a paper that gets positive reviews gives you some form of trust token that you can spend on other papers in a positive review. Ideally there is no single global metric for trust, but trust is computed relatively based on who you trust, so that everybody gets a personalized quality and relevance score for each paper. In a way you could also personalise your definition of trust. E.g. do you want to rank papers highly that have positive reviews by a handfull of highly trustwothy individuals, or do you trust papers that have many positive reviews.
- throwawaygh 4y ago
  That's just another popularity contest with all of the exact same perverse incentives. How do you think people get on conference committees / journal boards? We should stop worrying about popularity contests (citation counts, prestige-signalling conference committees/paper venues, etc.). When we decide whether to retain an academic as a visiting scientist, consultant, etc., I am always the voice in the room encouraging folks to disregard everything except for one question: after having read 3 or 4 of this person's most relevant papers, do we think they can help us with the specific problems we face? When I sit on grant review committees I ignore all the academic prestige shit and ask: is this a real 10-15-30 year problem for a field, does that field matter to the funding agency's objectives, and does the description of the plan of attack have a good chance of success? I don't need to evaluate popularity contests to answer those questions. Popularity contests are low-effort short-cuts for the low-brow scientist/program manager. Best to ignore them if you have real problems and need real solutions.

ClumsyPilot 4y ago

> Economically it is infeasible to politically play, social engineer, or bribe thousands If this were true, we would not have snake oil salesmen, pyramid scheme leaders, drug lords or youtube celebrities You just want to turn academics into yet another category of celebrities that must game social media. We have enough of that shit

j-pb 4y ago
We have that in academia already. It's games day in and day out. We need those games to be transparent at least. And a web of trust is not the same as social media.
- ClumsyPilot 4y ago
  Its not what we have in academia. And please show me where do we have a functioning web of trust
  - j-pb 4y ago
    > Its not what we have in academia. If you are blind to this then you are either in a very sheltered environment or you're part of the problem. We haven't build such a system yet, which is exactly the critique OP had. The broken funding process based on citation count and journal impact deincentivizes building and participating in such a system. It's all self enforcing with the publishers reaping in billions. It took years till european academia finally started to rebell against elsevier, and even now open access gets sabotaged by golden path open access publishing.
    - ClumsyPilot 4y ago
      If web of trust was a workable system, i wouod have been used in some industry by now - engineering, law, anything. But you criticism is based on a fantastical benchmark that was never seen in practice
      - 4y ago