3 ms·

It turns out for most complex analytical procedures there are a million more ways to do something wrong than right. That's why as much as I like the idea of pub

by kurthr 6y ago

It turns out for most complex analytical procedures there are a million more ways to do something wrong than right. That's why as much as I like the idea of publishing failures along with successes, it just doesn't make sense. However, as you mention, that doesn't mean that something is irreproducible... get the right grad student to show you how (hopefully they can actually comment on your procedure and help, but pharma/bio-chem is notoriously touchy). Like cooking, good experimental method and helpful documentation (recipes) take both temperament and training. http://pages.cs.wisc.edu/~kovar/hall.html http://pages.cs.wisc.edu/~kovar/hall.html "Electron Band Structure In Germanium, My Ass", is a great example of this... I'm pretty sure they had a bad contact (using indium solder?) when they ran the experiment. When you're getting garbage like that... stop and see if you can reproduce a single result, and until you can, something about your method/equipment/model sucks.

nordsieck 6y ago
> It turns out for most complex analytical procedures there are a million more ways to do something wrong than right. That's why as much as I like the idea of publishing failures along with successes, it just doesn't make sense. However, as you mention, that doesn't mean that something is irreproducible... get the right grad student to show you how (hopefully they can actually comment on your procedure and help, but pharma/bio-chem is notoriously touchy). Like cooking, good experimental method and helpful documentation (recipes) take both temperament and training. This sounds like a complicated way of saying the procedures for some scientific research are underspecified.
- kurthr 6y ago
  Well, yes... the problem is that if you're not reasonably proficient and observant then entire books worth of specification will be insufficient. Intel deals with this in their fabs by Copy Exactly! Not just the equipment and recipes, but the floor wax, the paint, the roof, soda machines, cafeteria... everything but the people who don't go inside except to fix stuff. So, if you're in software enjoy the fact that you have the wonder of a compiler, an OS, and a CPU you can (mostly) trust. The rest of the physical world dreams of that level of control and repeatability for anything like the tiny incremental cost. Other than math and theoretical physics that's as close as you'll get to fully specified.
jiggawatts 6y ago
That's why I've immensely enjoyed the content on the NileRed YouTube channel. I don't even study chemistry, I mostly have a physics background! The way the host shows mistakes, troubleshooting processes, recovery attempts, and even the outright failures is so refreshing after endless canned presentations focused on the final success, ignoring years of trial and error. Often, it's the trial and error part that's actually important, the end result not so much. The journey, not the destination, if you know what I mean. https://www.youtube.com/channel/UCFhXFikryT4aFcLkLw2LBLA https://www.youtube.com/channel/UCFhXFikryT4aFcLkLw2LBLA PS: I actually did the semiconductor conductivity experiment at University, and it was a blast! It was supposed to be a 12-hour experiment, most of which was collecting the data by hand and then analysing the curve. Fitting two nonlinear functions using fancy graph paper sounded awful dull. Two hours of playing with liquid nitrogen followed by ten hours of paperwork? No thanks! Like the guy complaining in that blog, I swiped the (at the time) fancy digital oscilloscope from the senior lab, output the data to a floppy, and did a nonlinear curve fit in Mathematica. Because it could fit both regimes at once, I got about one digit more accuracy than the reference text provided. The professor gave me 6 out of 5 for my score, and I finished the 12 hour lab in 4 hours. It was my first real life lesson in the power of computers to solve physical problems.

by kurthr 6y ago

nordsieck 6y ago

> It turns out for most complex analytical procedures there are a million more ways to do something wrong than right. That's why as much as I like the idea of publishing failures along with successes, it just doesn't make sense. However, as you mention, that doesn't mean that something is irreproducible... get the right grad student to show you how (hopefully they can actually comment on your procedure and help, but pharma/bio-chem is notoriously touchy). Like cooking, good experimental method and helpful documentation (recipes) take both temperament and training. This sounds like a complicated way of saying the procedures for some scientific research are underspecified.

kurthr 6y ago
Well, yes... the problem is that if you're not reasonably proficient and observant then entire books worth of specification will be insufficient. Intel deals with this in their fabs by Copy Exactly! Not just the equipment and recipes, but the floor wax, the paint, the roof, soda machines, cafeteria... everything but the people who don't go inside except to fix stuff. So, if you're in software enjoy the fact that you have the wonder of a compiler, an OS, and a CPU you can (mostly) trust. The rest of the physical world dreams of that level of control and repeatability for anything like the tiny incremental cost. Other than math and theoretical physics that's as close as you'll get to fully specified.

jiggawatts 6y ago

That's why I've immensely enjoyed the content on the NileRed YouTube channel. I don't even study chemistry, I mostly have a physics background! The way the host shows mistakes, troubleshooting processes, recovery attempts, and even the outright failures is so refreshing after endless canned presentations focused on the final success, ignoring years of trial and error. Often, it's the trial and error part that's actually important, the end result not so much. The journey, not the destination, if you know what I mean. https://www.youtube.com/channel/UCFhXFikryT4aFcLkLw2LBLA https://www.youtube.com/channel/UCFhXFikryT4aFcLkLw2LBLA PS: I actually did the semiconductor conductivity experiment at University, and it was a blast! It was supposed to be a 12-hour experiment, most of which was collecting the data by hand and then analysing the curve. Fitting two nonlinear functions using fancy graph paper sounded awful dull. Two hours of playing with liquid nitrogen followed by ten hours of paperwork? No thanks! Like the guy complaining in that blog, I swiped the (at the time) fancy digital oscilloscope from the senior lab, output the data to a floppy, and did a nonlinear curve fit in Mathematica. Because it could fit both regimes at once, I got about one digit more accuracy than the reference text provided. The professor gave me 6 out of 5 for my score, and I finished the 12 hour lab in 4 hours. It was my first real life lesson in the power of computers to solve physical problems.