Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There is a reproducibility crisis in psychological research. The main reason for this is that the experimental methods they use are not robust (although they might be rigorous). A second reason for this crisis is that the things they are measuring are often hard to actually define (eg is there a definition of 'conservative views' that we can all agree on, and does any such categorization in an experiment really mirror external reality?).

In reality what happened with the first paper is the editors of the journal sent it out for review, and whoever reviewed it was happy that it met the methodological standards considered sufficient at the time. This is not the same as being happy that it was true. You might consider that 'messed up' (and you are right, it is messed up), but that was how the sausage was made in psychology research in 2008. There is no galactic council that imposes standards of evidence in a journal. They rely on the reviewers, and the reviewers are active researchers in the field.

As an aside, publication in a top journal is not a guarantee that results are water tight, and should never be considered as such. If you want to believe a result, you need to read the paper and think about it carefully. Even then, this approach is susceptible to outright fraud, which also does happen.



If we have a reproducibility crisis in psychological research, it seems to me like one of the best things we can do to address it is to incentivize people to try reproduce other group's findings. It's not sexy and it won't earn headlines, but it is important. The absolute best way to kill such an effort is for a journal that published a bad study to refuse to even consider publishing a study that refutes it.


You’ve misdiagnosed the problem. The best thing we can do is incentivize research psychologists to produce robust research in the first place. This is unfortunately at odds with publishing lots of papers, making money off pop-psych books and giving TED talks.


wait, what?

We're allowed to question what a peer-reviewed article published in a top-flight journal says?

There might be a possibility that the science in Science might be wrong, and we need to work that out for ourselves?

This contradicts a lot of the messaging about science these days...


No, you're not allowed to question.

You're required to question!

-At least if you actually want to derive any benefit from the process.

Science is most assuredly a "use your (own) brain" kind of endeavor. ;-)


Unless you try questioning Climate Science, of course. That science is totally settled and unless you have four papers published in a climate-specialist journal you are not qualified to question it.

Climate Science isn't alone in this, though. Archeology also requires a PHD before you can comment on the purpose of any artifacts dug up by archeologists, even though archeology degrees don't teach anything about (for instance) textile production.

And then there's the various interesting parts of Social Sciences, which often require you to have a particular identity or experience before you can question any part of their conclusions.

I agree that Science should be a "use your own brain" kind of endeavour, but increasingly it is not, unless you have the appropriate letters after your name.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: