3 ms·

> It seems like a strange thing to take someone with a long and respected career and subject them to what would essentially be a Western blot and photomicrograp

> It seems like a strange thing to take someone with a long and respected career and subject them to what would essentially be a Western blot and photomicrograph audit before offering them a big position. This is absolutely something that we should routinely be doing, though.

majormajor 2y ago
It's pretty similar to the level of distrust in the software engineering job interview process. Pick your poison, to some extent. Better would be to not have to do it after-the-fact, but to vet better at every intermediate step, but it's hard. Just a very difficult people problem.
- cdaringe 2y ago
  Agreed. There!s uproar over coding interviews, which makes no sense to me. We give easy peasy code reviews to smoke check claimed skills. 4 of 5 candidates do absolutely terribly on very easy stuff, relative to their claimed skillset. No, our bar isn’t high—fraud (resume fraud) is sadly very real.
BigGreenJorts 2y ago
yeah it sounds a little bit absurd to me. It's just basic due diligence. You don't not run a background check on a potential employee just bc their resume looks good and they got a reference. In those cases you still go, "Annoying we have to wait because we want this person on board NOW and it's a fairly shallow investigation that 99% of the time doesn't reveal anything even if there is something, but it's the standard procedure."
dr_kiszonka 2y ago
I agree. To their defense, he had a few hundred papers published before joining, according to PubMed, and was a leader in his field: https://pubmed.ncbi.nlm.nih.gov/?term=Masliah%20E&filter=dates.1000%2F1%2F1-2015%2F12%2F30 https://pubmed.ncbi.nlm.nih.gov/?term=Masliah%20E&filter=dat... My concern is that with AI getting better and easier to use (e.g., in Photoshop) fraud will be extremely hard to detect. We might need to re-think how research is done and results verified.
- fluidcruft 2y ago
  Maybe we need instruments that sign results cryptographically and use a blockchain mechanism to establish provenance. We should have cameras that can establish that published images have not been modified (or at least provide raw and adjusted pairs--digital radiology has the concept of a "presentation state" that I think could work). In theory at least research should be auditable to a lab notebook. The problem with photos and such is you can't tell if it was modified before it was pasted to the page and large datasets just can't be put in to paper. And electronic notebooks I've used tend to be even more annoying than paper (too rigidly formatted and not adaptive to workflow optimization but it's difficult to explain). Anyway those sorts of things that establish provenance should also protect against deep learning. You may be able to create fake data, but can you deep fake data that was signed by Nikon device #XYZ with cryptographicly confirmed hashes published to a blockchain 3 years ago at the time the data was generated?

majormajor 2y ago

It's pretty similar to the level of distrust in the software engineering job interview process. Pick your poison, to some extent. Better would be to not have to do it after-the-fact, but to vet better at every intermediate step, but it's hard. Just a very difficult people problem.

cdaringe 2y ago
Agreed. There!s uproar over coding interviews, which makes no sense to me. We give easy peasy code reviews to smoke check claimed skills. 4 of 5 candidates do absolutely terribly on very easy stuff, relative to their claimed skillset. No, our bar isn’t high—fraud (resume fraud) is sadly very real.

BigGreenJorts 2y ago

yeah it sounds a little bit absurd to me. It's just basic due diligence. You don't not run a background check on a potential employee just bc their resume looks good and they got a reference. In those cases you still go, "Annoying we have to wait because we want this person on board NOW and it's a fairly shallow investigation that 99% of the time doesn't reveal anything even if there is something, but it's the standard procedure."

dr_kiszonka 2y ago

I agree. To their defense, he had a few hundred papers published before joining, according to PubMed, and was a leader in his field: https://pubmed.ncbi.nlm.nih.gov/?term=Masliah%20E&filter=dates.1000%2F1%2F1-2015%2F12%2F30 https://pubmed.ncbi.nlm.nih.gov/?term=Masliah%20E&filter=dat... My concern is that with AI getting better and easier to use (e.g., in Photoshop) fraud will be extremely hard to detect. We might need to re-think how research is done and results verified.

fluidcruft 2y ago
Maybe we need instruments that sign results cryptographically and use a blockchain mechanism to establish provenance. We should have cameras that can establish that published images have not been modified (or at least provide raw and adjusted pairs--digital radiology has the concept of a "presentation state" that I think could work). In theory at least research should be auditable to a lab notebook. The problem with photos and such is you can't tell if it was modified before it was pasted to the page and large datasets just can't be put in to paper. And electronic notebooks I've used tend to be even more annoying than paper (too rigidly formatted and not adaptive to workflow optimization but it's difficult to explain). Anyway those sorts of things that establish provenance should also protect against deep learning. You may be able to create fake data, but can you deep fake data that was signed by Nikon device #XYZ with cryptographicly confirmed hashes published to a blockchain 3 years ago at the time the data was generated?