Keeping an Eye on Wayward Studies
[Flatow] Earlier this week a minor journal, the Journal of Vibration and Control, issued a retraction notice for 60 papers, that is six zero, 60 papers after an investigation showed that there was what has been called a peer review ring, fraudulently moving publications into print. Last week, the journal Nature retracted two papers that had gotten a lot of press. You may remember one involved a method to make ordinary skin cells into something approaching stem cells through the use of an acid bath. Those papers the journal said contained serious errors, some of which constituted misconduct. Nature said that its reviewers had taken parts of the papers on trust and could not have been expected to catch the errors. So what's what's going on? Is the publishing process so broken? Joining me now is Ivan Oransky. He wears many hats, but with us today he is wearing his hat as co-founder of the Retraction Watch Blog, which broke the peer review ring story. Welcome back to program, Ivan.
[Oransky] Thanks, Ira. Always great to be here.
[Flatow] What is going on here?
[Oransky] Ah, well you know the vibration story . . . at the Vibration Control Journal, it's pretty interesting, there seems to be a lot of shakiness at this journal and maybe not too much control. I think that's kind of the headline. I think what all these stories speak to is just this tremendous pressure that scientists are under to get grants, to get tenure, to get promotions, to get prizes and all of it depends on papers, depends on publishing papers. And so, if everything you know and everything in your career depends on something, you're going to do whatever it takes to get there. Sometimes that just means, and most of the time, luckily, it just means working harder, working better, being smarter about things. Unfortunately in some cases as we've seen with these particular cases, and a number of others over the years, it means doing things that really aren't kosher, that are fraudulent, that are considered misconduct.
[Flatow] And doing it on purpose?
[Oransky] It's hard to say in some cases. Unless people actually come out and confess which is not typical, because of course, no one confesses to big crimes like murder or theft if they don't have to. But in a number of cases people have actually confessed, we've learned that yes, you know I felt this pressure, I actually knew it was wrong, or had good sense was wrong, and they did it anyway.
[Flatow] Let's talk about this a peer review ring. How did that work?
[Oransky] Sage Publishers, who is the publisher of this journal, Vibration and Control, they're calling it this a peer review and citation ring, which connotes that there were a number of people involved. It is actually unclear, I should say. whether it was more than one person involved. What happened was this engineering researcher in Taiwan named Peter Chen submitted a whole bunch of papers to this journal because he knew that their peer review system would ask him for suggested reviewers. In other words, people who would than look at the paper and the determine whether or not it was worth publishing. So he ended up signing himself into this peer review system multiple times, as many perhaps 130 times. In other words he created a hundred thirty fake email addresses and aliases. So he ended up in at least one case they found peer reviewing his own paper. It's like Ira Flatow is writing the reviews of Science Friday. Wouldn't that be wonderful, except you're making it look like it was Ivan Oransky writing the reviews of Science Friday saying that it's wonderful.
[Flatow] Let's talk about a couple of the famous cases. Let's talk about one of the big ones, retracted in Nature. We had talked about the research, the one with stem cells, we actually covered that when it was first published. What went wrong on that one? It looked so exciting and it was almost too good to be true . . . it was too good to be true.
[Oransky] Well, this is sort of the theme. That a lot of things that seem too good to be true might actually turn out to be too good to be true. In this case here is a paper published in Nature earlier this year, actually two papers published in Nature this year, of a really quick and easy way to make stem cells. And this obviously is of great interest a lot of people: basic scientists, clinicians, all sorts of people. but right after it was published people started raising questions. They said you know it doesn't even make sense. These images don't look right or there is something funny about them. Some people actually tried to do the work in their own labs, they tried to replicate, to reproduce to work. They couldn't. That started raising a lot of questions. At that point, and this was all done in public, and I think that that's actually an important lesson here, an important theme of what's been happening. A lot of this happening on web sites, in public, everyone can see what's going on . . .all those problems triggered a major investigation at the institution where this person worked. And, lo and behold, not only were there certain problems that might have been explained away by error and sort of something, just carelessness, but actually there was misconduct involved according to their report and that prompted the retraction of both papers.
[Flatow] And then a few weeks ago researchers announced, and I remember this one, that two extra-solar planets that had been said to be in the right position from the Sun to support life, sort of in the Goldilocks spot, probably didn't even exist at all. Is that is that the same kind of thing?
[Oransky] No. In that case it was just science, basically I think, and maybe this is a little over simplification, but, basically working the way it should. You know science is a process and I think that often and we lose sight of that. I have to blame, I'm sorry to have to do this on your airways, Ira, I have to blame us, as journalists, as well. We get very excited about things, we like to write about things, as if they were finite because it's sort of an easier story, it'll sell better. In this case, oh well, this is what scientists said, and I think was four years ago that they actually said it. And then 4 years later people again, trying not to experiment per se, but to look at the data again, look at what they were looking at and seeing. They determined, you know what, it probably wasn't that, and so it's just science over turning itself. You know, John Maddox who was the editor of Nature decades ago, had a very famous quote. I'm going to paraphrase here. A reporter asked him: John how much of what's in Nature every week is wrong? The reporter was really trying to get his goat and trying to get him to really be a little bit snarky or sarcastic. John said in a very straightforward way: you know, everything. Everything is wrong. The reporter was taken aback. What he meant by that is, over time, the vast majority of what we think we know now turns out to be wrong. Sometimes it turns out to be just a little bit wrong. Sometimes turns out to be completely wrong. That's not fraud. That's just how science works.
[Flatow] I'm Ira Flatow and this is Science Friday from PRI Public Radio International talking with Ivan Oransky about fraud in papers. There's another big science finding: the ripples of the Big Bang. Remember that one when that came out, supposedly scientists had seen the ripples. And after further thinking about it, somebody said hey, it might have been just dust, you know, dust in the way. That's why, when I heard about that I was thinking back to the Large Hadron Collider and the Higgs boson discovery. It seemed, at the time, people were trying to goad, I think, scientists into making an absolute claim about it being discovered. But they were sort of hedging their bets a little, waiting for more data to come in, which I think was laudable of them.
[Oransky] Absolutely. I think scientists should be forthright and I gotta be clear here. Most of them are, most very forthright about what a process science is. It's just that they get pushed into these situations, and again, if you're standing in front of a press conference and the mike is on, the cameras are on you, and all that and reporters ask you something maybe a little uncomfortable, something definitive, but you know that that's what they want and are going to keep asking? We're human beings, let's be honest about that. On the other hand there are a lot of pressures, never mind reporters, never mind press agents and people like that, just the way that science is funded. If your group doesn't say something definitive, maybe another group will and another group is going to get that grant. I think we have to be realistic about that, and try to solve that problem somehow, I don't necessarily have the answer, so that people don't feel that pressure. Because you're absolutely right, saying something definitive, when you don't have something definitive, really it warps everyone's understanding of science and I think in the long run it really kind of makes people distrust science more than they really should.
[Flatow] But the fact that these papers were retracted is sort of on the positive side of the ledger is it not? It illustrates the self correcting method of science.
[Oransky] Absolutely, I think that the Nature papers were retracted within about 6 months, which sounds like a long time to some people I suppose now that we alll live online but actually it's quite quick especially for a big finding, big journal, that required investigation. You know retractions may be a sign of health. Adam Marcus and I who co-founded Retraction Watch four years ago, we went into this for a number of reasons, but one of the things we were sort of playing with, one of the ideas we were playing with is that retractions could actually mean that science is, as you suggest, working pretty well, that the self correction nature of science is working well. We may not have as many retractions as there actually should be yet. A lot of them are going to be for honest error and again, that is different in the cases that we've just been discussing. In the case of the Nature stem cells that did involve some fraud and some misconduct. In the case of the Vibration and Control Journal that involved misconduct, really publishing misconduct, rather than the traditional kind of fraud, if you will. But those were retracted actually quite quickly and in those cases people were fairly forthright. I think there was a little bit of teeth grinding and a lot of stubbornness at certain points in both cases particularly in the Nature case, but science should just be transparent at the end of the day and retractions are an important part of self correction so they should talk about them . . . talk about how important they are in the process.
[Flatow] Well, Ivan, thank you for coming on and talking about it with us.
[Oransky] Thank you. Always a pleasure.
[Flatow] You're welcome. Ivan Oransky co-founder of the Retraction Watch Blog also Vice President and editorial director of MedPage Today.