Back in 1974, the famed physicist Richard Feynman gave a commencement speech at the California Institute of Technology in which the term “Cargo Cult Science” was popularised as a way of describing science that was not really science at all. What we appear to have now is the emergence of Cargo Cult Peer Review. It looks like peer review, it smells like Peer Review, but it is just an illusion and not the real thing, so it fools many.
The Disruption of Scientific Publication
The publication of scientific papers is self-regulated via what is known as the peer-review process. Having concluded a scientific study and written up all the details, you publish. Traditionally scientific articles appeared within reputable scientific journals that specialised. Prior to publication what gets published goes through a peer review process, hence the quality of the journal’s output is maintained.
Reputation mattered because the publishers wanted academic institutions to subscribe. The libraries that subscribe need to pick and choose what to subscribe to, hence they would pay attention to what their academic readers needed.
This traditional model has been disrupted by the arrival of Open Access journals. The authors of scientific papers are instead charged a fee and so anybody can now gain access without a subscription. This makes rather a lot of sense because the dissemination of scientific information should be open and not locked away behind a paywall. The previous paywall model might of course have worked if the fees had been reasonable, but they are not. Instead in many cases they are quite extortionate.
Unfortunately, some see the open access fee model as an opportunity and so we also have seen an explosion in the number of vanity journals that will for a fee publish anything at all. Since reputation is not a factor because the only source of income is the publication fees, then the quality of many of these journals renders them completely and utterly useless.
Think of it as Cargo Cult Scientific Publication, or to be a bit more accurate, Cargo Cult Peer Review. They have the look and feel of being a real scientific journal. There is often lip service to a peer-review process and also an editorial board, but if you pay a fee then you will be published.
That whooshing sound just now was the quality flying vertically downwards past you and dropping through the floor.
Sham vs Real Scientific Publications
How can you tell what is quality and what is not?
University of Colorado librarian Jeffrey Beall, used to keep a list of the predatory journals. When he first started that it consisted of just 18 publishers to keep away from, but that soon rapidly grew to a list of thousands. Unfortunately about a year ago it was all wiped without any warning.
What happened?
He was threatened and bullied into silence …
@CabellsPublish stands behind close personal friend @Jeffrey_Beall who was forced to shut down blog due to threats & politics #academicmafia
— Lacey E. Earle (@lacey_earle) January 17, 2017
In some respects it is not a surprise because some with quite a bit of political and financial clout would have looked upon his list as a threat to their source of income.
Do we have a major problem now?
Cory Doctorow recently highlights within an article how reputation is not a consideration for some of these predatory journals. One scientist has managed to get his dog on to the editorial board of seven journals …
By day, “Olivia Doll” sits on the boards of seven academic journals; by night, she’s a Staffordshire terrier named Ollie, owned by Mike Daube, a public health expert in Perth, Australia.
Daube ginned up an intentionally absurd CV for his dog, claiming that she was senior lecturer at the Subiaco College of Veterinary Science and past associate of the Shenton Park Institute for Canine Refuge Studies (e.g., she was a shelter dog) and laying out research interests that included t”he benefits of abdominal massage for medium-sized canines” and “the role of domestic canines in promoting optimal mental health in ageing males”; then he submitted it the to boards of scammy predatory journals and every single one of them added her to their editorial boards.
The Global Journal of Addiction and Rehabilitation Medicine even made her associate editor.
These journals are parasitic on the open access science movement; they charge desperate researchers thousands of dollars for “peer review” of their papers, then give them shovelware credits they can put on their CVs as they seek positions and promotion.
Since doing this, his dog has been given some real research to peer-review and that is heartbreaking from Daube because he knows that the con being run by these journals has successfully scammed real researchers.
Being curious I checked to see who is on the editorial board of the pseudo-journal mentioned … and yes, this guy’s dog is still an associated editor on the board of that predatory publication.
OK, come now, you know you want to meet this amazingly smart editor, so here you go …
Has Beall’s List been completely wiped?
No, an archived copy still exists, you will find that here at this link. It is no longer being updated, but it is still a valuable resource as a starting point.
Even More Disturbing is this
Charles Seife writes about how some of the big names are also getting in on the game of faking peer reviews.
There are of course the blatantly obvious fake articles such as “Cuckoo for Coco [sic] Puffs” and “Get Me Off Your Fucking Mailing List.”, that even a glance at the paper should leave any editor thinking WTF, but the more subtle yet wholly fake do also exist as well.
About a decade ago some clever MIT students coded up SCIgen. This will toss together sciency sounding phrases randomly and also stir in a few pictures and references to give it the look and feel of a real paper. A proper peer-review would rapidly pick up on it being gibberish, but ….
…for the better part of a decade, SCIgen-written papers have been appearing in conference proceedings and computer science journals all around the world, exposing bogus peer review wherever it crops up. (It’s not hard to find oodles and oodles of examples if you know what to look for.) So, last year, when Springer and IEEE had to withdraw more than 120 articles written by SCIgen, it was a clear demonstration that fake peer review was a big deal in major mainstream scientific publications, not just predatory ones. (Springer, with more than 2,000 journals in its portfolio, is a juggernaut like Elsevier; IEEE is the leading society for electrical engineers and computer scientists and a major publisher in its own right.)
I believe the technical term for that is …”yikes”. (well OK, I had another anglo-saxon phrase in mind, but I censored that).
If you think the above is bad, then it gets even worse …
It’s not just an issue in computer science. In 2013 science journalist John Bohannon created a biomedical equivalent of SCIgen and submitted more than 300 obviously bogus papers to publishers around the world. More than half were accepted. Though Bohannon’s sting was mostly aimed at predatory publishers, some reputable houses were caught in the dragnet, such as Dove, Sage, and Elsevier.
… and even worse …
More recently I exposed more than 100 journal articles that looked like they were products of a scientific version of Mad Libs; they tended to have obvious signs of plagiarism or poor scholarship of various sorts. A rigorous peer-review process should have caught these problems. Yet these papers appeared in journals published by PLoS, Nature Publishing Group, Wiley, Taylor & Francis, BioMed Central, and other publishing powerhouses.
… and also there is more where that came from (see here).
So yes, what can I say except to welcome you to the age of Cargo Cult Peer Review.
Bottom Line
Peer Review is still the gold standard, and while not ideal, most still strive to do the best they can. Yes, some rubbish gets through, and yes there is a rising tide of predatory publishers, and yes the big name publishers need to get their act together if they wish to maintain their reputation, but the scientific endeavour still marches forward, warts and all.
Technology has created these issues, and so in time it will also be the solution. Tools and algorithms to sniff out the obviously fake SciGen or similar will be increasingly deployed, and the fake publishers will find their references are automatically purged from Scopus and Web of Science.
This might indeed be an era of cargo cult peer review via the emergence of predatory journals, but the real peer review process not only persists, it still thrives. We just need to step up to the mark and be smarter about how we filter out the gibberish.