For years, a major LGBT advocacy group had been using a particular canvassing technique that they had developed, which they felt was most successful at persuading people to become more accepting of the LGBT community. However, the technique had never undergone any rigorous study, so they reached out to researchers to test it.
The first researchers, studying same-sex marriage in California, discovered that, indeed, the canvassing techniques worked, at least when the canvasser was himself an LGBT individual, and the persuasive effects lasted at least nine months (the furthest out they tested in follow-up interviews). Everyone was very excited. The paper was published in Science in December 2014.
Then, another research team decided to test the same technique as it related to transgender issues, so they studied the first paper in exquisite detail. They found that the numbers being reported simply didn't make sense. They contacted the survey company supposedly used for the data collection, and were told that they had not actually been hired for any such thing. Although the lead author (a grad student at the time) continues to deny that he faked the data, the case was strong enough that his co-author (an older professor who was a mentor of both researchers in the second team) voluntarily asked that the paper be retracted.
The second team actually found that persuasive effect claimed in the first paper did exist, but there were some key differences. For example, they found that the effect was the same, regardless of whether or not the canvasser was actually transgendered, while the original paper claimed the canvasser had to be a member of the particular minority community to have an effect. In other words, if the original researcher had bothered to actually do the research, he would have found the effect he claimed to.
This makes an interview from last year with the co-author of the original paper (the one who was not in charge of the data) especially interesting, since he used the experience to tout the self-correcting nature of science. His co-author, he felt, should have known that others would try to replicate his findings, and eventually he would be found out. A certain amount of trust is assumed when doing research - that is, you start from the assumption that your colleagues are honest - but those assumptions will eventually themselves be tested when people try to replicate your findings, take your research in new directions, etc.
Ultimately, that's one of the most powerful aspects of rigorous scientific research. While it's possible for dishonest researchers to game the system, and defeat the peer review that is designed to prevent bad research from getting published in the first place, the nature of scientific study virtually guarantees they will be found out eventually (in the case cited here, the fraudulent study didn't even make it six months).
The first researchers, studying same-sex marriage in California, discovered that, indeed, the canvassing techniques worked, at least when the canvasser was himself an LGBT individual, and the persuasive effects lasted at least nine months (the furthest out they tested in follow-up interviews). Everyone was very excited. The paper was published in Science in December 2014.
Then, another research team decided to test the same technique as it related to transgender issues, so they studied the first paper in exquisite detail. They found that the numbers being reported simply didn't make sense. They contacted the survey company supposedly used for the data collection, and were told that they had not actually been hired for any such thing. Although the lead author (a grad student at the time) continues to deny that he faked the data, the case was strong enough that his co-author (an older professor who was a mentor of both researchers in the second team) voluntarily asked that the paper be retracted.
The second team actually found that persuasive effect claimed in the first paper did exist, but there were some key differences. For example, they found that the effect was the same, regardless of whether or not the canvasser was actually transgendered, while the original paper claimed the canvasser had to be a member of the particular minority community to have an effect. In other words, if the original researcher had bothered to actually do the research, he would have found the effect he claimed to.
This makes an interview from last year with the co-author of the original paper (the one who was not in charge of the data) especially interesting, since he used the experience to tout the self-correcting nature of science. His co-author, he felt, should have known that others would try to replicate his findings, and eventually he would be found out. A certain amount of trust is assumed when doing research - that is, you start from the assumption that your colleagues are honest - but those assumptions will eventually themselves be tested when people try to replicate your findings, take your research in new directions, etc.
Ultimately, that's one of the most powerful aspects of rigorous scientific research. While it's possible for dishonest researchers to game the system, and defeat the peer review that is designed to prevent bad research from getting published in the first place, the nature of scientific study virtually guarantees they will be found out eventually (in the case cited here, the fraudulent study didn't even make it six months).
