This article was published in the BC Psychological Association’s journal.
Why We Need a Journal of Non Significant Findings
Sometimes we stumble across an idea that seems to be very logical. For example, when I formulated my dissertation topic, it seemed logical to me that if substance abuse therapists disclosed that they had struggled with substance abuse issues themselves, it may help to build the therapeutic alliance. Many substance abuse treatment facilities only hire people who have had these issues likely due to that premise that it takes ones who have suffered from these issues to understand ones who continue to suffer with those issues (J. Leighterwood, Personal Communication, 1st May 2012).
Once I formulated this research topic, I set out to do the monumental literature review. Although there was some great research that was somewhat related, there was little research that was directly related to the topic. One might think that because there is little research about a given topic, this may mean that no one has formulated such a brilliant research topic (some of us psychologists can get narcissistic).
As a result of formulating what I thought was a brilliant research question that had never been explored before, I spent a couple years pursuing this topic. I set up research conditions in which the variables of therapeutic alliance, therapist self-disclosure, and dropout were controlled. I had one group of men attending substance abuse treatment watch a role-play with a therapist disclosing he had struggled with substance abuse and another group watching the same role-play with the disclosure edited out. I also had told one group that the therapeutic alliance was strong. To another group, I told them that the therapeutic alliance was weak (therefore there were four groups: strong versus weak therapeutic alliance by disclosure versus no disclosure). There was also a question that checked to see if the “manipulation” worked (whether the research participants believed the alliance was strong or weak). The outcome measure was whether or not the research participants would anticipate dropping out.
Although I found a strong and significant relationship between therapeutic alliance and dropout, counter-intuitively, I found that the therapist self disclosure had no relationship with anything (Reeh, 2010). It was not correlated with therapeutic alliance, it was not correlated with dropout, and it did not moderate the relationship between therapeutic alliance and dropout (Reeh, 2010). The numbers were not even close to reaching significance. Therefore, although it made intuitive sense, there was no support for therapist self-disclosure having a significant relationship with anything.
I’d imagine that not finding significant results is a common occurrence. To find out what other researchers have found (or not found) in addition to the formal databases, we can ask our colleagues on list serves, we can talk to researchers who have spent many years exploring a topic, we can read others’ dissertations on the topic etc. However, we have no reliable and comprehensive way to find out what has been done and what has not been explored when researchers do not find significant results (especially when research is conducted not for the purpose of completing a thesis or dissertation).
I began to wonder about how many other people have had the same experience in that they had a great idea, it made intuitive sense, but they found no support for it. If my dissertation research had not resulted in any significant results, there would have been no chance of publishing the results in a journal. Thus, there would be no way to inform others of the research questions and methods that did not result in significant findings so that other researchers could learn from this research and perhaps use different methods or formulate different research questions.
Hence, we need a Journal of Nonsignificant Findings. In this journal we could publish research that did not find anything significant so that future researchers do not remain ignorant of unsupported assumptions and methods that did not work. Thus, researchers in many areas could move farther ahead if they could work from a solid base of knowledge rather than continuing to plod along in a state of semi-ignorance.
After I graduated, a group of ten researchers were working on dissertations on the topic of therapist self disclosure. I can’t help to wonder; is therapist self-disclosure just not that important? Or was it that the methods I used did not result in any significant results? If so, does therapist self-disclosure make a significant difference to someone somewhere? If we had results from about 50 studies, for example, in which no significant results were found regarding therapist self disclosure, maybe then we could conclude that therapist self disclosure is just not that important. For now though, is there any way of finding out if therapist self disclosure is not that important without a Journal of Non Significant Findings?
Reeh, H. E. (2010). Does therapist self-disclosure moderate the relationship between therapeutic
alliance and dropout? (Doctoral dissertation). Available from ProQuest Dissertations and