Skip to main content

New system will measure journals' research transparency

A new ranking system for academic journals measuring their commitment to research transparency will be launched next month -- providing what many believe will be a useful alternative to journal impact scores.

Under a new initiative from the Center for Open Science, based in Charlottesville, Va., more than 300 scholarly titles in psychology, education and biomedical science will be assessed on 10 measures related to transparency, with their overall result for each category published in a publicly available table.

The center aims to provide scores for about 1,000 journals within six to eight months of the site’s launch in early February.

Among the measures of assessment are whether the publications ask authors to share their raw data or if they set standards for research design disclosure.

Other categories cover whether journals encourage the replication of studies and whether authors are required to preregister their experiments before data collection.

On data sharing, for example, journals would be given a one-out-of-three score for publishing a data availability statement, two for requiring authors to share data (subject to exceptions) and a full three out of three for providing enough data to enable full replication.

Journals will also receive credit if they offer the option of peer review before any research is undertaken -- a published format known as “registered reports,” used by more than 200 journals, in which assessors focus on research design rather than end results.

The new ranking system comes amid concerns over the difficulty of reproducing some of science’s most high-impact papers owing to large amounts of missing or withheld methodological data. Last week Brian Nosek, the center’s director, told Times Higher Education that a forthcoming paper on cancer research replication would be a “wake-up call” for science given how the weak methodology sections of most papers made reproduction impossible.

David Mellor, the center’s director of policy initiatives, said he believed the new table would “promote those who are doing the most to encourage open science.”

“We want to recognize those who are taking difficult steps to address bad practices,” said Mellor, who hoped journals would “require or reward open science practices that are not as common as they should be.”

Mellor added that he hoped the new scorecard system -- in which scores are decided by the center’s assessment teams based on publicly available information -- would provide an alternative to the current journal ranking system, which is based on citations per paper, known as journal impact.

“There is a clear pecking order based on impact, but this is really about a paper’s novelty or prestige, rather than [adhering to] core scientific values,” said Mellor.

Editorial Tags: 
Is this diversity newsletter?: 
Newsletter Order: 
0
Disable left side advertisement?: 
Is this Career Advice newsletter?: 
Magazine treatment: 
Display Promo Box: 


Udimi - Buy Solo Ads from Inside Higher Ed https://ift.tt/2uL28VB
via IFTTT

Comments

Popular posts from this blog

Author discusses book on grad school

Graduate school is a great mystery to students, and to some faculty members, says Jessica McCrory Calarco, the author of A Field Guide to Grad School: Uncovering the Hidden Curriculum (Princeton University Press). Calarco is an associate professor of sociology at Indiana University. She believes many faculty members (as well as graduate students, of course) will benefit from her book. She responded to questions via email. Q: How did you get the idea to write this book? Why did the issue speak to you? A: This book started as a tweet . Or, rather, as a series of tweets about the hidden curriculum of higher ed. Ph.D. student Kristen K. Smith had tweeted about the need to better educate undergrads about grad school opportunities, and it made me think about how opportunities in academe are often hidden from grad students, as well. Reflecting on my own experiences in grad school, I thought about the many times I'd found myself embarrassed because of what I didn't know -- the

Guest Blog: Where Does the Bizarre Hysteria About “Critical Race Theory” Come From?—Follow the Money!

Blog:  Just Visiting Guest Blog: Where Does the Bizarre Hysteria About “Critical Race Theory” Come From?—Follow the Money! By Isaac Kamola Trinity College Hartford, CT There are now numerous well-documented examples of wealthy right-wing and libertarian donors using that wealth to transform higher education in their own image. Between 2005 and 2019, for example, the Charles Koch Foundation has spent over  $485 million  at more than 550 universities. As demonstrated by Douglas Beets and others, many of these grants include considerable  donor influence  over what gets taught, researched, and even who gets hired. It should therefore come as no surprise that conservative megadonor, Walter Hussman Jr.,  lobbied hard  to deny the Pulitzer-prize winning journalist Nikole Hannah-Jones a tenured professorship at the UNC journalism school that bears his name. Nor that her offer of tenure, awarded through the normal channels of faculty governance, was ultimately  revoked   by a far-

Live Updates: Latest News on COVID-19 and Higher Education

Image:  Woman Charged With Faking Positive COVID-19 Test From U of Iowa   Nov. 5, 6:14 a.m. A lawyer in Colorado has been charged with faking a positive COVID-19 test from the University of Iowa to get out of a court appearance, The Gazette reported.   Emily Elizabeth Cohen was booked Tuesday on a detainer from the Boulder County Sheriff’s Office, shortly after she tweeted that the Colorado court system “just had me arrested alleging I lied about having COVID. Tweeting from cop car.”   The Boulder Daily Camera reported that Cohen is scheduled for a 10-day trial in Boulder County in Colorado starting Dec. 6 for 11 felony counts stemming from allegations she collected fees from immigrant families before losing contact with them without producing visas or work permits.   -- Scott Jaschik Judge Permits Suit Against Montana State to Go to Trial Nov. 3, 6:18 a.m. A Montana judge has ruled that a suit against Montana State University over the shift to online education