Peer-review activists push psychology journals towards open data ""
Posted on: March 1, 2017

Psychologist Gert Storms says he won’t review papers if authors don’t share underlying data, or explain why they can’t.

An editor on the board of a journal published by the prestigious American Psychological Association (APA) has been asked to resign in a controversy over data sharing in peer review.

Gert Storms — who says he won’t step down — is one of a few hundred scientists who have vowed that, from the start of this year, they will begin rejecting papers if authors won’t publicly share the underlying data, or explain why they can’t.

The idea, called the Peer Reviewers’ Openness Initiative, was launched by psychologists hoping to increase transparency in a field beset by reports of fraud and dubious research practices. And the APA, which does not ask that data be made available to peer reviewers or shared openly online, seems set to become an early testing ground for the initiative’s influence. With Storms’ situation still unresolved, the society’s council of editors will discuss whether it should change its policies at a meeting in late March.

Storms, a psychologist at the Catholic University of Leuven in Belgium and a consulting editor for the APA’s Journal of Experimental Psychology: Learning, Memory, and Cognition, accepted an invitation last year to review a study for the journal, and pointed out his new open-data policy. The journal’s editor, Robert Greene, wrote back to say that Storms’s stance set “a terrible precedent” because it was unfair to the author of the paper and opposed the APA’s policies and the guidelines followed by other reviewers. “Given that your policy conflicts with that of the journal, I think that it’s best that you step down from the editorial board,” he wrote.

Storms refused, writing that he would continue to do what he thought was necessary to “prevent sloppy science”. And he forwarded his correspondence to other editors at the journal. Two of them, Robert Hartsuiker and Marc Brysbaert, both psychologists at Ghent University in Belgium, wrote to Greene saying that they, too, would quit if Storms was forced to resign. “The policy of asking people to leave rather than inviting a discussion and getting critical voices — I found that quite inappropriate,” said Hartsuiker.

Greene, a psychologist at Case Western Reserve University in Cleveland, Ohio, notes that Storms’ stance is inconsistent with APA rules and that all submissions to the society’s journals ought to be treated in the same way. “At this point we’re just letting things lie and I am not removing [Storms] from the editorial board. We’ll see what happens at the council of editors,” he adds.

The APA’s journals publisher, Rose Sokol-Chang, declined to comment on Storms’ case. “While we support open sharing of data when it can be ethically shared, we leave the decision of whether to do so to the author,” she added.

Psychology’s data strife

The conflict marks the latest effort by some psychologists to change their discipline’s policies on data sharing. The APA, in common with many publishers in the field, asks authors to make their data available to others after publication. But as far back as 2006, a study found1 that 73% of psychologists were unwilling or unable to do so, even though they had agreed to share.

Calls for change gathered force after 2011, when the full extent of years of research fraud by Dutch psychologist Diederik Stapel came to light. In that case, an investigative committee noted that Stapel often refused to share his scientific data with colleagues, including his co-authors.

Further fraud scandals followed, and psychologists have since taken the lead on efforts to revisit earlier work, questioning statistical data and showing that some textbook findings in the field are hard to replicate. Simine Vazire, a psychologist at the University of California, Davis, says that data should be made available on publication and also to reviewers, even if it is on a confidential basis. Without such transparency, assessing a study “is like buying a used car without being able to look under the hood”, she says.

But by 2012, only 38% of researchers publishing in four APA journals shared their data when asked, Storms and others reported in a 2015 paper2 titled ‘Are we wasting a good crisis? The availability of psychological research data after the storm’.

Eric-Jan Wagenmakers, a psychologist at the University of Amsterdam and one of the founders of the Peer Reviewers’ Openness Initiative, says he has surveyed 600 researchers in the field to understand barriers to data sharing. The main explanations that they gave were: data sharing is an uncommon practice in the field; researchers prefer to share data only on request; it is time consuming; and researchers have never learned how to share data properly. Wagenmakers’ survey results have not yet been published.

Despite prolonged pressure, the APA hasn’t changed its data policies for years, says Jelte Wicherts, a psychologist at Tilburg University in the Netherlands who co-authored the 2006 review and has since criticized the APA’s policies and psychology’s data-sharing standards. “My hope is they will change their view relating to the openness of data,” he says.



Share This:

Leave a Reply

Your email address will not be published. Required fields are marked *

CommentLuv badge