4 ms·
The Rank Amateur
- patio11 17y agoAmateurs are a useful check on the system because they will never submit a grant proposal to the committee chaired by the co-investigator of the paper they're reviewing, and hence can make aggressive, confrontational demands of his code like running it.
- JoachimSchipper 17y agoWhich is why almost every scientific publication does not tell the peer reviewers who the author is, and vice versa.
- jgrahamc 17y agoMy experience with reviewing papers was that it was pretty easy to figure out who the author was because each field contains a limited number of specialists. And looking at the references on a paper enables you to narrow it down. It's a bit like the situation with date of birth and CVs: you can figure out how old someone is, if you want to know that, by looking at their degree year.
- arethuza 17y agoGiven the hyper specialisation of most academic research, circle of people involved in an area can be pretty small. You probably don't even need to read the paper - just to know what kind of approach to what problem they are taking and you can make a good guess of the people/team involved.
- yummyfajitas 17y agoIn my experience (math, physics, CS), most journals are only single blind. The referee is anonymous, the author is not. Additionally, referees can sometimes give themselves away. The authors also fail to cite [Irrelevant Paper, Irrelevant Paper 2, Peripherally related paper 3]. These works should be cited and compared to the work being reviewed. "Irrelevant Paper", John Smith. Journal of Trivialities, 1995. "Irrelevant Paper 2", by John Smith and Steve Jones. Journal of followups to irrelevant papers, 1998. "Peripherally related paper 3", by John Smith and Phil Johnson. Journal of Blah Blah, 2002.
- scott_s 17y agoA minority of CS conferences are double-blind - I've only submitted to one that was. But it's difficult to maintain true author-blindness. If your work builds on previous work at all, it's difficult to maintain that. Further, I once reviewed a paper that was double-blind, and before I made it through the abstract I knew immediately where the paper came from. (Not the specific authors, but I knew it had to be from a particular company.)
- m_eiman 17y ago"The most effective people at finding errors in scientific reasearch are scientists: it was professional glaciologists, after all, who exposed the error in the IPCC 2007 case study of Himalayan glaciers." To exclude the amateur is to deny a large part of the history of science. Well, he's not saying that a scientist is always right and an amateur is always wrong. He's just saying that a scientist is more likely to find errors in research material, which I think sounds reasonable since they're supposedly trained to do it. Then there's e.g. the good ol' "seeing what you know is there instead of what's actually there" effect which works against those who've spent a lot of time with something. Good find on the bad data, btw! Real science is in the numbers, not in the propaganda surrounding them.
- jgrahamc 17y agoI do agree that that quote doesn't tell the whole story, if you read the professor's entire letter you'll see what I'm talking about.
- deleted 17y ago[deleted]
- randomwalker 17y agoDisclaimer: I'm a scientist, so maybe I'm biased. But then again, I'm often a vocal critic of the system myself, so maybe not. Science is actually admirably egalitarian. That's not one of the things I think is wrong with it. The heavily politicized branches might not be egalitarian; I have no experience with them. But in general, you don't need to be a member of any professional organizations or have any other credentials to submit your work to journals and conferences (which is how we transact our business.) Many journals employ double-blind reviewing, in which case even unintentional discrimination based on credentials is unlikely or impossible. Nevertheless, scientists might often give the impression of not wanting to let amateurs into the 'club'. Why is this? It is simply an issue of bandwidth. Think about this: for each amateur, like the author, who had something to contribute, how many do you think thought they had something to contribute but were mistaken? If you guessed a hundred thousand you'd be in the right ballpark. I've seen it in many different areas. For example, the number of people trying to submit "proofs" of P != NP, (or worse, P = NP) is just ridiculous. Some are well-intentioned although ignorant, and others are just cranks. Some fields of research attract more amateur claims than others, but whichever field you look at, the amateurs greatly outnumber scientists, and the vast majority of them are mistaken. So what do we do? We use a simple filter. If you speak our language, we'll listen to you. This is what appears to the lay public as anti-amateur. But it's not, really. All you have to do is to learn some simple definitions and terminology set forth in a straightforward way in previous papers, to prove that you've done your homework. Then write up what you have to say and we'll be happy to give it a read. Sure, like every heuristic, it's not perfect; sometimes there are false negatives. But there really isn't an alternative. Without a filter, all we'd ever be doing is debunking crackpot theories. I acknowledge that much of this probably doesn't apply to climate change "science," which is a special case. But there's been a lot of criticism of science in general, and I wanted to set the record straight. It's important to distinguish between what's really broken and what appears broken, otherwise we risk throwing the baby out with the bathwater.
- ahk 17y agoYour criticism of current scientific practice does not go far enough. Recent issues like the refusal to grant access to climate data cannot be so easily brushed off with a "not enough bandwidth" complaint. We have the internet now. There is no reason to not put up a site with data & code when you publish your paper and just point all requests to go to the site. Scientists need to start using these tools like the rest of us and stop hiding behind such false excuses.
- glymor 17y agoAmateurs scientists are still scientists, using similar methods and acting in a constructive way. If nothing else many eyes effect makes them useful - particularly in astronomy. Journalists attempt to generate controversy to increase page views. Myles Allen was correct that corrections to climate models have come from scientists amateur or otherwise not journalists writing copy.
- jgrahamc 17y agoI agree about the journalists bit, but when he says in his letter includes the following: John Christy took a lot of heat over the satellite data, but nothing remotely like what is being turned on Phil Jones. It would have been romantic if John's error had been uncovered by journalists combing through stolen emails, or members of the public issuing freedom of information requests. But it wasn't. It was found by the US government funding a painstaking independent analysis of the satellite record, with John's co-operation, just as Phil has said he would be happy to co-operate with an impartial and scientific re-analysis of the surface temperature record, if anyone wants to fund such a thing. The implication is that none of these 'members of the public' are any good, the only thing worth considering is funded research.
- glymor 17y agoI agree he's too broad (and in other places as well). I had assumed it was just the heat of the argument. I also wonder about him invoking his professional capacity. It implicitly makes him a representative of scientists; in acting so he really should have been more nuanced.
- pbiggar 17y agoI'm pretty sure Einstein had a PhD (I believe it was on Brownian motion). Hardly a rank amateur. Patent offices don't hire Joe Soap to review patent application.
- david927 17y agoThis is an important distinction, and the other amateurs always came early in the field's history when it was a simpler science. Climate science is not in its infancy. It's broad, deep and complex. I desperately hope you're not implying that we should give credence to the din of crackpot circus clowns who think dramatic climate change isn't really happening and that Al Gore is personally placing space heaters on the glaciers and polar icecaps.
- david927 17y agoI'll take those downvotes as a "Yes, Al Gore is personally placing space heaters on glaciers."
- yummyfajitas 17y agoClimate science is broad, and a few parts are deep. Nevertheless, amateurs such as JGC and Steve McIntyre have identified and corrected errors in the field. You are being downmodded because you are nonsensically comparing them to deranged conspiracy theorists. You do so without evidence, most likely for political purposes.
- david927 17y agoYou are being downmodded because you are nonsensically comparing them to deranged conspiracy theorists. You do so without evidence, most likely for political purposes I have no political purpose -- and I'm not sure how politics play in watching shipping lanes open up on the north pole. I'm glad if they find errors. But correct me if I'm wrong, it hasn't change anything. And this is what concerns me: that it raises a hand for debate. The debate is over. We need to act and we need to have started yesterday. I feel like I'm watching a woman get raped, but instead of people stopping it, they want to talk about if it's really "rape" if the man wears a condom. We are at a tipping point and we can, and should, cut carbon emissions. Moving to alternate energy sources is an important step that we will make eventually anyway, but to do it sooner means that the poorest of the world will not get quite as screwed in the end.
- nico 17y agoHere's another example, James Ellis, who came up with the idea of public key cryptography: http://www.wired.com/wired/archive/7.04/crypto_pr.html http://www.wired.com/wired/archive/7.04/crypto_pr.html (HN submission: http://news.ycombinator.com/item?id=1064931 http://news.ycombinator.com/item?id=1064931)
- jgrahamc 17y agoI'm not sure he counts as an amateur. He was employed to do this sort of thinking, it was just done in secret.
- lazyant 17y agoI'm getting tired of this "Einstein also failed a test in university" meme (usually suggesting in was in physics or math). From what I remember from a biography he always did excellent in physics and math, and in that famous entrance exam (that I think he took a year earlier than usual) he only failed in French, which he didn't care much for.
- jgrahamc 17y agoDid I claim that? It's entirely factual to say that he didn't get into ETH Zurich, had a hard time getting a research job and did his best work while employed by the patent office.
- lazyant 17y agoNote that I agree with the general thesis of your article. While you posted a fact (the failed admission test), that fact is a cherry-picked outlier that gives the opposite impression of Einsteins actual academic achievements (especially more so if you omit the rest of the circumstances). It's like saying the fact that I won my Physics School chess championship in 1996 and leaving it there (true) without mentioning that I'm actually a mediocre player that won because many of my opponents didn't show up. My rant was more general; I'd swear I've seen in a couple of movies the "Einstein flunked a physics test".
- DanielBMarkham 17y agoNever underestimate what a fresh set of eyeballs can do for a problem, even a highly sophisticated and nuanced one. Good hackers work in all sorts of fields they are not experts in. I can't tell you the number of times I've seen "breakthroughs" with technical problems simply because the smart group with all the jargon had to explain themselves and their situation to the outside hackers just trying to make a computer program help them out. It's extremely easy for groupthink to take over small groups of specialists, and it happens most of the time without any of them realizing it. By the way, just so we don't feel so cocky, this also happens all the time in groups of programmers in a project who take hidden assumptions and make them into unwritten law without anybody questioning them either. The amateur, outsider, or programmer has a vital role in technical groups and efforts of all stripes.
- edw519 17y agoThe most important lesson my mentor ever taught me: We went to a client to work on Problem X. He quickly determined that solving Problem X would achieve nothing. Problem Y was the real problem, but was way outside our areas of expertise. So what did he do? He slept 4 hours a night for the next 2 weeks studying everything he could find about Problem Y. He reviewed reports, industry literature, called experts, and talked to as many people in the company who knew anything about that subject area. Within 2 weeks, he presented a brilliant solution that no one had ever considered but was instantly understandable by their experts. (That solution included work done by us and we had a great client relationship for years.) Later, I asked him why he tried to accomplish something so difficult with such a seemingly tiny possibility of success. I'll never forget his reply... "I didn't know that I couldn't do it, so I did it."
- neilk 17y agoIt's a charming anecdote, but don't take this as a story of individual brilliance. Read it as faith and perseverance triumphing over doubt, and conquering problems with appropriately applied work. Note the crucial elements. He talked to a lot of experts in field Y. This is not a "lone genius"! He had the freedom and incentives to work on this almost uninterrupted. He would have spent at least 100 hours, which is worth at least as much as two undergraduate courses. Also, he was an expert in related field X, which must have helped. Furthermore, he because he knew the practical problem, he could cut down the search space. He didn't have to investigate everything about field Y, just stuff applicable to him. If you consider all that, the solution doesn't look so miraculous. What's more interesting to me is that I think even ordinary people could be achieving that kind of result all the time if they had the right background, incentives, and support.
- ZeroGravitas 17y agoI'm not sure the amateur-professional dichotomy is useful. They put the data out there and someone found an error. Someone else might spot an anomaly that leads to some important discovery. Whether that someone is a professional scientist, a professional programmer, a statistics student or a crank is mostly irrelevant. Putting stuff out for re-use can (if planned for) be cheap and easy. There's no reason in this day and age, not to do it. That's what's important here and pitting professional against amateur doesn't really move you forward on that issue.
- arethuza 17y agoI have met some people with Chartered Engineer/PE status who program - not many though. They are the only people I would call "professional programmers".
- araneae 17y agoWell, it used to be that being a scientist wasn't a profession at all. Most science was done by the independently wealthy. In many fields, today, to be an amateur you need to be independently wealthy as well. Most science requires a lot of expensive equipment. The "amateurs" he spoke of were using techniques that required little money (data analysis, pen and paper math)