13 Feb Experimenter Effects and Replication in Psi Research
An important unresolved question in scientific research in general, and in psi research in particular, is the effect that the expectancy of the experimenter has on experiment outcomes. I am pleased to report that two distinguished colleagues and I have received funding to conduct a new experiment to test the hypothesis that the expectancy of the experimenter will have an impact on the outcome of a psi task. We are now recruiting teachers to help us recruit student experimenters. Here is a brief history of this field of investigation to give a sense of what we’re investigating. I’ll conclude with details about the new experiment at the end of this post.
Psi research, the study of ESP and psychokinesis (mind over matter), is a controversial topic in science. Although proponents of psi can point to an extensive body of evidence for phenomena such as telepathy and precognition, most academic psychologists still do not believe that they are likely to exist (Bem, 2011). Central to the debate is the issue of replication. Can independent investigators replicate reportedly successful psi experiments?
An important factor in determining the success or failure of attempted replications appears to be the experimenter’s orientation toward the phenomenon under investigation—attitudes toward and expectations about the outcome of the experiment. In mainstream psychology, Rosenthal has demonstrated experimenter expectation effects in more than 300 studies, including studies in classroom and clinical settings (Rosenthal, 1978).
Experimenter effects have been observed in psi research for more than 70 years, with some researchers actually suggesting that evidence of psi is due less to gifted subjects than to the person who does the testing. Stanley Krippner, a humanistic psychologist, summarized findings showing differences among experimenters (e.g. Ramsey and Cabibbo, 1975), among data collectors (Johnson, et al. 1972), different reciprocal attitudes between experimenter and subject (Nash 1968), and differences across time by the same experimenter (Rivers 1950).
Although the experimenter effects are usually attributed to conventional sensory communication, some researchers have suggested that some of the variation may actually be caused by psi. For example, in one study, subjects did better at guessing psi targets prepared by a psi proponent than on those prepared by a psi skeptic.
There is also evidence that an experimenter can remotely influence a subject’s responses through the mediation of psi. For example, my own work with William Braud (1997) reported that experimenters could influence subjects’ physiology from a distance. Using this same protocol, psi-skeptic Richard Wiseman and I collaborated in three attempted replications using the same subject pool and procedures. I obtained significant psi effects in two of the three experiments, but Wiseman failed to get an effect in any of them (Schlitz et al., 2005).
Retroactive Facilitation of Recall: A Replicable Protocol?
One area of psi research that has received recent attention is the study of precognition (knowledge of the future). Experimental tests of precognition have been reported for more than half a century, and a meta-analysis (evaluation across experiments) found that 309 precognition experiments yielded a small but significant success rate (Honorton and Ferrari, 1989).
In an article published in a leading psychology journal, psychologist Daryl Bem reported 9 experiments that tested for retroactive influence by “time-reversing” well-established psychological effects so that the individual’s responses were obtained before the putatively causal stimulus events had occurred (Bem, 2011). All but one of the experiments yielded statistically significant results, and results across all 9 experiments were statistically significant overall.
To encourage exact replications of his experiments, Bem created software that runs them and a detailed instruction manual, including the experimenter’s script. As a result, several replications have now been attempted, most of them adopting the protocol of Experiment 9, “The Retroactive Facilitation of Recall.” In this experiment, participants are first shown a set of words and then given a free-recall test of those words. They are then given a set of practice exercises on a randomly selected subset of those words. The psi hypothesis is that the practice exercises will retroactively facilitate the recall of those words and, hence, subjects will have recalled more of the to-be-practiced words than the unpracticed words. Bem later confirmed this observation, although other investigators have gotten mixed results. Controversy over the research has been featured in the media and scientific discussion across the world.
A particularly striking set of findings implicating experimenter effects emerged from an attempted replication of the experiment by Subbotsky, a developmental psychologist in the UK. In his own experiments, Bem ran a set of control sessions in which the computer selected the subsets of practiced and non-practiced words but practice exercises were not actually administered. Bem reported non-significant findings on these control sessions: Experimental participants recalled words from the two subsets equally well. Subbotsky’s (2012) study successfully replicated the retroactive facilitation effect for the experimental sessions, but he found a significant reversal on the control sessions: Experimental participants recalled significantly more of the non-practice words than the to-be-practiced words. The main difference between Bem’s and Subbotsky’s control sessions was that Bem’s experimenters were blind with respect to whether they were running an experimental or a control session, whereas Subbotsky’s experimenter knew when he was running a control session. These findings suggest that the retroactive-facilitation-of-recall experiment is a promising vehicle for a more systematic examination of experimenter effects in psi research.
In a new experiment, recently funded by the Bial Foundation in Portugal, I am working with Daryl Bem and Arnaud Delorme to test the hypothesis that the expectancy of the experimenter will have an impact on the outcome of a psi task. In particular, we will explore one of the most controversial aspects of psi research: precognition. The protocol involves three levels: (1) teachers who will recruit student experimenters, (2) student experimenters who will receive a standardized training in the experimental procedure, (3) participants in the study who participate in the psi task. We are now recruiting teachers who can help us recruit student experimenters. In this process, they will help us test this remarkable hypothesis that challenges the foundation of scientific assumptions about objectivity and replication. For more information, or if you are a teacher who can help us recruit student experimenters, please write to me at mschlitz@noetic.org.
References
Bem, D. Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect. Journal of Personality and Social Psychology, 100, 407-425, 2011.
Honorton, C. Replicability, experimenter influence, and parapsychology: An empirical context for the study of mind. Paper presented at AAAS, Washington, D.C., 1978.
Honorton, C. & Ferrari D.C. (1989) Future telling”: A meta-analysis of forced-choice precognition experiments, 1935–1987, Journal of Parapsychology, v.53, 281-308.
Kennedy, J. E. & Taddonio, J. L. Experimenter effects in parapsychological research. Journal of Parapsychology. 40:1-33,1976.
Krippner, S. Commentary. In Rosenthal, R. and Rubin, D. B. (1978) Interpersonal expectancy effects: The first 345 studies. The Behavioral and Brain Sciences, 3, 398- 399, 1978.
Krippner, S & Murphy, G. Psychical research and human personality. Proceedings of the Society for Psychical Research. 48:1-15, 1949.
Nash, C. B. Comparison of ESP run-score averages of groups liked and disliked by the experimenter. Journal of the American Society for Psychical Research. 62:411- 417, 1968.
Ramsey, M., & Cabibbo, C. Experimenter effects in extrasensory perception. Journal of the American Society for Psychical Research. 69:135-149,1975.
Rhine, J. B., Pratt, J. G., Stuart, C. E., Smith, B. M., & Greenwood, J. G. Extrasensory Perception After Sixty Years. Holt, New York, p 341, 1940. Ritchie SJ, Wiseman R, French CC. Failing the Future: Three Unsuccessful Attempts to Replicate Bem’s ‘Retroactive Facilitation of Recall’ Effect. PLoS ONE 7(3): e33423, 1992.
Rivers, O. B. An exploratory study of the mental health and intelligence of ESP subjects. Journal of Parapsychology. 14:267-277,1950.
Rosenthal, R. The silent language of classrooms and laboratories. In: W. G. Roll, R. L. Morris, & J. D. Morris (eds.), Proceedings of the Parapsychological Association, 1971. Durham, N.C.: Parapsychological Association, 1972. Replicability and experimenter influence: Experimenter effects in behavioral research. Paper presented at AAAS, Washington, D.C., 1978.
Schlitz, M., Wiseman, R., Radin, D., & Watt, C. Of two minds: Skeptic-proponent collaboration within parapsychology. Paper presented at the meeting of the Parapsychological Association, Petaluma, CA, 1995.
Schlitz, M. & Braud W. Distant intentionality and healing: Assessing the evidence. Altern Ther Health Med, 3(6):62-73, 1997.
Subbotsky, E. Sensing the future: The Non-standard observer effect on an ESP task. Lancaster University, UK, 2012.
West, D. J. & Fisk, G. W. A dual experiment with clock cards. The Journal of the Society for Psychical Research. 37:185-189, 1953.
White, R. A. The influence of the experimenter on psi test results. In: B. B. Wolman (ed.). Handbook of Parapsychology. Van Nostrand Reinhold, New York, 1977.
You can also view this post on the Noetic Now Blog.