Predatory open access journals seem to be a hot topic these days. In fact, there seems to be kind of a moral panic surrounding them. I would like to counter the admittedly shocking and scary stories around that moral panic by pointing out that perhaps we shouldn't be worrying so much about a fairly small number of admittedly bad actors and that we should be more concerned with the larger issues around the limitations of peer review and how scientific error and fraud leak through that system.
I'm hoping my methodology here will be helpful. I hope to counter the predatory open access (OA) journal story with a different and hopefully just as compelling narrative. Fist of all, after gathering together some of the stories about predatory OA journals, I will present some of what's been written recently about issues in scientific peer review, it's problems and potential solutions.
Then I'll be presenting a more direct counter narrative to the predatory one. First of all, I'll present some information about the fantastic resource Retraction Watch. Then I'll present some concrete case studies on how traditional peer reviewed commercial publishing fails in all the same way that supposedly predatory publishing fails.
Finally, using the incredible work of Walt Crawford and others, I'll gather some resources that will further debunk the whole "predatory" open access moral panic and further suggest that perhaps it isn't the bogus OA journals that are the main source of "predatory" publishing, but rather that the big commercial and society publishers perhaps deserve that label more.
I want to be perfectly clear. My issue isn't with the necessity of peer review and it's importance in science. Issues like climate change and vaccination panics highlight why trusting in peer reviewed science is most responsible thing to do. After all, “Research misconduct accounts for a small percentage of total funding”. I think it's probably safe to say that at the end of the day, peer review and scientific publishing work fairly well as far as fraud and general quality levels go.
Both peer review in particular and the scholarly communications ecosystem in general are human systems with all the potential for the full range of human weaknesses that implies: folly, error, bias, fallibility and bad faith. This post will explore some of the dimensions of folly, error, bias and bad faith in scholarly communication.
Let's start our adventures with some media stories and cases studies of bad faith -- true predatory open access journals.
- Nov 2014. Fringe activists learn to use 'predatory' science journals
- Nov 2014. Sham Journal Accepts Totally Absurd But Completely Appropriate Paper (The Get Me Off Your Fucking Mailing List paper.)
- Dec 2014. A paper by Maggie Simpson and Edna Krabappel was accepted by two scientific journals (More here)
- Jan 2015. Why a fake article titled "Cuckoo for Cocoa Puff" was accepted by 17 medical journals
Predatory journals are a real problem, of course, as we can see from the list above. However, I think the moral panic about their extent and impact tends to be exaggerated. I would really love to see more balance in reporting about predatory journals that contrast the real issues with scam journals with what I think are the far more pressing issues in scholarly communications. In other words, the flaws and limitations in the peer review system and the far more "predatory" traditional system of scholarly publishing that's controlled by the big commercial and society publishers. It's those publishers that are the leeches affecting the system.
These stories and anecdotes about predatory journals tend to acquire the mythic stature of the stories and anecdotes about vaccination that drive the anti-vaccine movement. Those tragic, personal stories take on a weight and social impact that's disproportional to the actual scientific and statistical significance.
Time to explore bias and human fallibility a little bit. Here are some resources about the general state of peer review, talking in general about the issues around peer review and the potential for reform. This list is meant to contrast the moral panic about "predatory" open access journals with a sober discussion of the strengths and weaknesses of peer review across all of science publishing, not just some fairly specific issues with a limited number of open access journals.
General Resources on Quality in Scientific Publishing, particularly on Issues with and Reform of Peer Review
- Jul 98. Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports: A Randomized Controlled Trial
- Aug 2005. Why Most Published Research Findings Are False
- Dec 2008. Exploring Scientific Misconduct: Isolated Individuals, Impure Institutions, or an Inevitable Idiom of Modern Science?
- May 2009. Three myths about scientific peer review
- Aug 2011. High-profile titles lead the field in number of retractions
- Feb 2013. Opinion: Scientific Peer Review in Crisis
- Jul 2013. Why Has the Number of Scientific Retractions Increased?
- Apr 2014. What Can We Do About Junk Science?
- Jun 2014. The perils of Science by press release: ‘overly exaggerated presentation of research findings’
- Jul 2014. The Guardian view on the end of the peer review
- Oct 2014. Science fiction? Why the long-cherished peer-review system is under attack
- Nov 2014. The Open Science Peer Review Oath
- Nov 2014. Measuring the effectiveness of scientific gatekeeping
- Nov 2014. Impact Challenge Day 19: Establish your expertise with Open Peer Review
- Dec 2014. Peer review could reject breakthrough manuscripts, study shows
- Dec 2014. Peer review isn’t good at “dealing with exceptional or unconventional submissions,” says study
- Dec 2014. Peer review — reviewed: Top medical journals filter out poor papers but often reject future citation champions
- Dec 2014. Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot
- Dec 2014. PLOS is anti-elitist! PLOS is elitist! The weird world of open access journalism.
- Feb 2015. The Ethics of Authorship: Is Ghostwriting Plagiarism?
- Feb 2015. Scandals prompt return to peer review and reproducible experiments
- Beyond Beall’s List: Better understanding predatory publishers
- Mar 2015. We need post-publication peer review of journals
- Mar 2015. Why you can't always believe what you read in scientific journals (Pubpeer & post-publication peer review)
- Mar 2015. Springer and Université Joseph Fourier release SciDetect to discover fake scientific papers (More info)
- Mar 2015. The Extent and Consequences of P-Hacking in Science ("One type of bias, known as “p-hacking,” occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science.")
- Mar 2015. Amid a Sea of False Findings, the NIH Tries Reform
- 2015. Data fraud in clinical trials
- xxxx. COPE statement on inappropriate manipulation of peer review processes (Committee on Publication Ethics, undated)
(The Shit My Reviewers Say tumblr is a lighter-side exploration of some of these issues.)
More importantly perhaps, there is another set. There is no shortage of fairly well publicized cases of significant retractions or scientific fraud that got past the peer review process in traditionally published, peer reviewed journals, mostly from the big commercial or society publishers. In other words, where peer review was the issue, not the subscription model.
If you want to know about the failing of the big publishers when it comes to quality control or about researcher perpetrating scientific fraud, Retraction Watch is the definitive site on the web.
Resources by and about Retraction Watch
- Aug 2011. Is it time for a Retraction Index?
- Dec 2012. Elsevier editorial system hacked, reviews faked, 11 retractions follow
- Jun 2014. “Barriers to retraction may impede correction of the literature:” New study
- Jul 2014. Retractions are coming thick and fast: it's time for publishers to act
- Nov 2014. Publisher discovers 50 manuscripts involving fake peer reviewers
- Nov 2014. The Peer Review Scam: How authors are reviewing their own papers
- Nov 2014. Publishing: The peer-review scam
- Dec 2014. Science journals screw up hundreds of times each year. This guy keeps track of every mistake
- Dec 2014. Retraction Watch is growing, thanks to a $400,000 grant from the MacArthur Foundation
- Dec 2014. Are companies selling fake peer reviews to help papers get published?
- Dec 2014. Why MacArthur is Backing a Popular Blog About Flawed and Fraudulent Science
- Dec 2014. The Top 10 Retractions of 2014
- Jan 2015. Hall of Shame
- Mar 2015. Yes, we are seeing more attacks on academic freedom: guest post by historian of science and medicine
The site Science Fraud was taken down by various legal threats. While it existed, it was an amazing resource for uncovering practices such as falsified images or tables. Some posts are retrievable via the Internet Archive.
SCIgen is a website that allow anyone to automatically generate a bogus paper. It is often used to generate garbage papers for predatory open access journal stings. SCIgenDetection is one site that detects SCIgen papers. The SCIgen page has a number of examples and other resources related to automatically generated bogus scientific pages.
Springer has recently teamed up with Université Joseph Fourier to release the a generalized open source software package SciDetect which tries to detect fake scientific papers such as those generated by SCIGen.
And yes, the main event where we explore a different dimension of bad faith and human folly and weakness. This time on the side of the supposedly "good guys."
Bellow are examples where big commercial or society traditional, subscription-based peer review have fallen short, either due to careless or insufficient review or fraud on the part of scientists. Of course, peer review will rarely catch genuine fraud as the books are cooked. But even fraud cases demonstrate the limits of peer review across all scholarly communication, not just in "predatory" open access journals.
I would like to emphasize that this list is extremely selective. I'm mostly only highlighting particularly egregious examples that have made their way into the mass media or onto popular blogs. As above, for much much more, please visit Retraction Watch for more complete coverage. For example, The top 10 retractions of 2014.
This list is meant to contrast in number and severity to the list of examples of "predatory" open access publishing crisis and stings above.
Failure in Scholarly Communications Ecosystem through Stupidity, Error or Fraud
- 1989. Fleischmann–Pons experiment cold fusion controversy
- Apr 1994. Doctor Says He Falsified Cancer Data to Help Patients (Roger Poisson fakes trial data)
- Mar 1999. A Doctor's Drug Trials Turn Into Fraud (Robert Fiddes fakes lab data, gets published)
- Feb 2000. Breast Cancer Researcher Admits Falsifying Data (Werner Bezwoda fakes data & gets results published)
- Jun 2001. How a cancer trial ended in betrayal (Harry W Snyder Jr & Renee Peugot fake data for profit, get published)
- Sep 2002. Scientific fraud found at Bell Labs by Linda A. Johnson (Jan Hendrik Schon's fraud at Bell Labs, numerous papers involved)
- Jan 2006. Lancet study faked: Investigation to probe all research conducted by scientist accused of fabricating results from 900 research participants (Jon Sudbo fakes out The Lancet, gets published)
- Nov 2008. Elsevier Math Editor Controversy (November 2008 issue of Chaos, Solitons & Fractals has five articles by notorious editor Mohamed El Naschie. He has over 300 articles in that journal overall.)
- May 2009. Elsevier published 6 fake journals (Elsevier journals sponsored by pharma companies)
- Oct 2009. Disgraced Cloning Expert Convicted in South Korea (Hwang Woo-suk is a famous case of fraud in stem cell research)
- Feb 2010. The Lancet retracts Andrew Wakefield’s article (Notorious "vaccines cause autism" paper by Andrew Wakesfield is retracted by Lancet)
- May 2011. Updated Cell Phone Study Deeply Flawed, Say Experts (British Medical Journal publishes flawed paper saying cell phone radiation causes cancer)
- Jul 2011. Marc Hauser resigns from Harvard (Hauser resigns due to scientific misconduct)
- Nov 2011. Report finds massive fraud at Dutch universities (Diederik Stapel fakes data in dozens of social-psychology papers, gets published)
- Jul 2012. Anesthesiologist Fabricates 172 Papers (Yoshitaka Fujii gets an unbelievable number of papers past mostly traditional peer review)
- Sep 2012. Top Canadian scientist and award-winning student caught in 'blatant plagiarism' of text (Dongqing Li and Yasaman Daghighi publish plagiarized paper in Springer journal Microfluidics and Nanofluidic, which Li edits)
- Retraction count grows to 35 for scientist who faked emails to do his own peer review (Journal of Enzyme Inhibition and Medicinal Chemistry author Hyung-In Moon did his own peer review)
- Feb 2013. Arsenic-based life paper: peer review process comes to light #arseniclife (The journal Science publishes a paper they hype to the moon on arsenic-based life that is shredded in post-publication social media commentary)
- Jun 2013. A Personal Academic Journal: Why is a major academic publisher printing a journal that seems a lot like the newsletter of the editor’s fan club? (SAGE journal Nursing Science Quarterly seems to be mostly about it's editor, Rosemarie Rizzo Parse)
- Feb 2014. Publishers withdraw more than 120 gibberish papers (IEEE and Springer withdraw 120+ computer generated gibberish conference papers)
- Mar 2014. Want to make sure your paper gets published? Just do your own peer review like this researcher did (Yongdeng Lei faked peer review for paper in Environmental Management, among others from Springer etc.)
- May 2014. The So-Called Sting (The journal Science runs a sting on various open access journals to see how their peer review holds up. However, they design their experiment so poorly that it would have failed any reasonable peer review process if they had published it as an article rather than "news")
- Jun 2014. Facebook’s Unethical Experiment: It intentionally manipulated users’ emotions without their knowledge (PNAS publishes paper that should never have passed internal ethics/peer review)
- Jul 2014. ISU loses $1.4 million in fraud case (Grant to study HIV vaccine rescinded due to Dong-Pyou Han's misconduct in faking results)
- Jul 2014. STAP stem cell papers officially retracted as Nature argues peer review couldn’t have detected fatal problems (Nature acknowledges limits of peer review in rush to publish papers in hot field)
- Jul 2014. SAGE Publications busts “peer review and citation ring,” 60 papers retracted (Huge scandal at Journal of Vibration and Control)
- Nov 2014. This Is What Happens When No One Proofreads an Academic Paper (Wiley journal Ethology has embarrassing lapse in "value added" service)
- Nov 2014. University of Regina prof investigated for allegedly plagiarizing student's work (plagiarized paper slips into ICE journal Environmental Geotechnics)
- Dec 2014. Elsevier retracting 16 papers for faked peer review (Author Khalid Zaman orchestrated series of fake peer reviews in Elsevier journals Economic Modelling, Renewable Energy, and Renewable and Sustainable Energy Reviews)
- Dec 2014. For Sale: “Your Name Here” in a Prestigious Science Journal (Springer journal Diagnostic Pathology may have copied articles, researchers used "paper writing" service which copied & pasted)
- Feb 2015. Journals without editors: What is going on? (unusually high percentage of papers in Elsevier journal Research in Autism Spectrum Disorders are by editor Johnny Matson)
- Feb 2015. What pushes scientists to lie? The disturbing but familiar story of Haruko Obokata (Fraudulent papers published in Nature.)
- Mar 2015. Major publisher retracts 43 scientific papers amid wider fake peer-review scandal (Springer-owned BioMed Central retracts papers due to faked peer review)
- Mar 2015. The games we play: A troubling dark side in academic publishing (Elsevier journals Research in Developmental Disabilities & Research in Autism Spectrum Disorders accused of gaming citation counts)
As noted above, this is the tip of the iceberg. Please see Retraction Watch for the rest of the iceberg.
And here are some books about academic fraud.
- 1983. Betrayers of the Truth by William Broad, Nicholas Wade
- 2004. The Great Betrayal: Fraud in Science by Horace Freeland Judson
- 2009. Plastic Fantastic: How the Biggest Fraud in Physics Shook the Scientific World by Eugenie Samuel Reich
- 2010. On Fact and Fraud: Cautionary Tales from the Front Lines of Science by David Goodstein
- 2014. Faking Science: A True Story of Academic Fraud by Diederik Stapel
And as a bit of a desert, let's take a brief look at who we should perhaps be considering predatory journals, those big commercial and society journals that soak the library world for every penny of obscene profit.
Oh yes, some resources from this blog and beyond that highlight some of the issues with the big, traditional journals, some of which are society, some of which are commercial. And finally, some resources about the real predatory publishing, the big commercial and society publishers who control so much of scholarly publishing.
This list is extremely partial. Please feel free to add your own suggestions in the comments.
- Mar 2014. Elsevier: bumps on road to open access: Academic seeks to gather examples of cases where open access article fees have been paid but content remains behind a paywall
- Apr 2014. The Sad Case of Jeffrey Beall
- Jun 2014. Around the Web: Your university is definitely paying too much for journals
- Oct 2014. Around the Web: Big Deals ‘R Us, or, Libraries in the lobster pot
- Mar 2015. Know the Difference: Scientific Publications versus Scientific Reports (American Chemical Society takes a swipe at PLOS ONE style peer review, which assess scientific validity rather than attempting to judge impact)
- Mar 2015. Elsevier Appears To Be Slurping Up Open Access Research, And Charging People To Access It
- Mar 2015. Wrongly paywalled articles: a recap of what we now know
- Mar 2015. Wiley are charging for access to thousands of articles that should be free
- Apr 2015. The Economics of Open Access
Hearing complaints and panic about predatory open access journals? Send them here for a hopefully more complete and honest picture.
(As usual, if I've mis-characterized or misunderstood any of the incidents or if I'm missing any significant items for any of the lists above, please let me know in the comments or by email at jdupuis at yorku dot ca. Hey, think of this as post-publication open peer review on this blog post. The wave of the future!)