Skip to main content

New system will measure journals' research transparency

A new ranking system for academic journals measuring their commitment to research transparency will be launched next month -- providing what many believe will be a useful alternative to journal impact scores.

Under a new initiative from the Center for Open Science, based in Charlottesville, Va., more than 300 scholarly titles in psychology, education and biomedical science will be assessed on 10 measures related to transparency, with their overall result for each category published in a publicly available table.

The center aims to provide scores for about 1,000 journals within six to eight months of the site’s launch in early February.

Among the measures of assessment are whether the publications ask authors to share their raw data or if they set standards for research design disclosure.

Other categories cover whether journals encourage the replication of studies and whether authors are required to preregister their experiments before data collection.

On data sharing, for example, journals would be given a one-out-of-three score for publishing a data availability statement, two for requiring authors to share data (subject to exceptions) and a full three out of three for providing enough data to enable full replication.

Journals will also receive credit if they offer the option of peer review before any research is undertaken -- a published format known as “registered reports,” used by more than 200 journals, in which assessors focus on research design rather than end results.

The new ranking system comes amid concerns over the difficulty of reproducing some of science’s most high-impact papers owing to large amounts of missing or withheld methodological data. Last week Brian Nosek, the center’s director, told Times Higher Education that a forthcoming paper on cancer research replication would be a “wake-up call” for science given how the weak methodology sections of most papers made reproduction impossible.

David Mellor, the center’s director of policy initiatives, said he believed the new table would “promote those who are doing the most to encourage open science.”

“We want to recognize those who are taking difficult steps to address bad practices,” said Mellor, who hoped journals would “require or reward open science practices that are not as common as they should be.”

Mellor added that he hoped the new scorecard system -- in which scores are decided by the center’s assessment teams based on publicly available information -- would provide an alternative to the current journal ranking system, which is based on citations per paper, known as journal impact.

“There is a clear pecking order based on impact, but this is really about a paper’s novelty or prestige, rather than [adhering to] core scientific values,” said Mellor.

Editorial Tags: 
Is this diversity newsletter?: 
Newsletter Order: 
0
Disable left side advertisement?: 
Is this Career Advice newsletter?: 
Magazine treatment: 
Display Promo Box: 


Udimi - Buy Solo Ads from Inside Higher Ed https://ift.tt/2uL28VB
via IFTTT

Comments

Popular posts from this blog

Author discusses book on grad school

Graduate school is a great mystery to students, and to some faculty members, says Jessica McCrory Calarco, the author of A Field Guide to Grad School: Uncovering the Hidden Curriculum (Princeton University Press). Calarco is an associate professor of sociology at Indiana University. She believes many faculty members (as well as graduate students, of course) will benefit from her book. She responded to questions via email. Q: How did you get the idea to write this book? Why did the issue speak to you? A: This book started as a tweet . Or, rather, as a series of tweets about the hidden curriculum of higher ed. Ph.D. student Kristen K. Smith had tweeted about the need to better educate undergrads about grad school opportunities, and it made me think about how opportunities in academe are often hidden from grad students, as well. Reflecting on my own experiences in grad school, I thought about the many times I'd found myself embarrassed because of what I didn't know -- the ...

Bad Education: A Movie Review

"It's not having what you want," quips Roslyn Assistant Superintendent Pam Gluckin in her Long Island accent, "it's wanting what you got." And what educators got from HBO's Bad Education was a harrowing detail of a pair of school administrators gone rogue with the school district's treasury, sacking $11.2 million before they were caught... by [...] from The Educators Room https://ift.tt/3d5LaSu via IFTTT

Tips and Strategies for Remote Learning (March 2020)

For the past four weeks, starting two weeks before our Spring Break last week, teachers, administrators and staff at our school have been preparing for “remote school” for students as a result of the coronavirus / COVID-19 crisis. On March 19, 2020, our Head of School sent out a letter to our parents including information about our remote learning plans and FAQs for parents . If your school is currently preparing for remote learning ( as all Oklahoma public school districts now appear to be , pending tomorrow’s state school board meeting vote) I encourage you to review these publicly shared documents and information. As our school’s “Technology Integration and Innovation Specialist” this year, I’ve been building an instructional site for our teachers to support remote learning, on support.casady.org , which is openly licensed CC-BY . This collaborative effort with many other teachers and members of our school staff has provided a good opportunity to curate as well as present instruc...