And what's wrong with that? That only makes it easier to spot the errors in their research.

Or harder, especially if you use the existing code as a crutch when recreating the experiment. You're much more likely to overlook a bug and copy it than perfectly recreate it from scratch.

Or worse yet, if you just run experiment.sh, see the results are the exact same, and declare that you recreated the results.

If someone goes looking through the code, it's a terrific improvement. But I work with astronomers and I haven't seen that tendency here. I expect this is changing across all science, but it's probably happening faster in biology.

Two "clean room" implementations that achieve the same result is much, much stronger evidence than any degree of code reviews on one piece of code (or reviews of one experiment). Independent reproducibility is a cornerstone of science.

I absolutely agree. I also have about zero faith in untrained scientists ability to perform a clean room implementation. (That's neither here nor there though.)

Good point, I hadn't considered that. But isn't the source code a part of the methodology used? As in, if someone doesn't review the source code, then he's skipping a certain part of the methodology?

Yes, logically that's true. Unfortunately, in the past programming wasn't particularly respected—remember Admiral Hopper discovered bugs—prior to that everyone had apparently assumed their programs would just be a small matter of coding or something. The scientists I know continue to regard a program as this necessary inconvenience, I suppose in part because the math is the reality.

So yes, I agree, but there's going to need to be some education and cultural change before we get there, and I think fields that have embraced a computational subfield have a big head start.