In this section we will review and update the data we have collected on Feedback Informed Therapy (FIT), and describe how this data clearly demonstrates our effectiveness. We will also discuss the latest research on FIT, including new research and thinking about what makes therapy effective.
Introduction to the Issue of Effectiveness
The Centers for Family Change implemented Feedback Informed Therapy (FIT) in the spring of 2015. Since that time all of our therapists have had our new clients complete a Feedback Rating Scale at the start of every therapy session. At this point we have amassed a solid body of data that clearly demonstrates our effectiveness. Below we explain this data, and provide hard evidence that supports our claim to be highly effective practice. For those who like more detailed information we also include a brief discussion of the Feedback Informed Therapy system that we utilize, the research that supports its, and the ways in which the standards/criteria for determining effectiveness are established.
The Centers for Family Change utilizes the Acorn system of Feedback Informed Therapy (1). This system allows us to evaluate the effectiveness of our work (as a practice, as individual therapists and with specific clients). The Acorn system clearly shows that our therapists are:
- Highly effective
- Helpful to the majority of our patients
- Consistently review and monitor their feedback data (which is also associated with improved effectiveness)
Highly effective: The standard for determining effectiveness of psychotherapy (and other interventions) is known as “effect size.” Acorn defines effect size as “a method of reporting the magnitude of improvement”, i.e,, the amount of improvement. While the Acorn site offers a highly technical discussion of this issue (2) suffice it to say that “effect size” is the gold standard in outcome research. An effect size of .8 (again, see the Acorn site for a detailed discussion of this statistic) is the consensus criteria for effectiveness. Any treatment with an effect size of .8 or greater is considered to be effective, or “large”.
Centers for Family Change therapists have a combined effect size (for all our therapists) of over 1.00 (for the past year, 11/15-11/16). This effect size is considered “highly effective” by Acorn.
Highly Effective Part II: ACE certificates of effectiveness: Acorn awards certificates of effectiveness to those therapists who defined as effective and highly effective (2.d.). Since we began collecting data all of our therapists have been found to be effective, and nearly all of our therapists (95%) have found to be highly effective. Thus, all of our therapists have been awarded Certificates of Effectiveness!
Helpful to the Majority of out Patients: A different way of looking at effectiveness is to look at the number of patients who are found to improve as a result of therapy. Over 75% of our clients report improvement while 59% report significant improvement (from 11/15-11/16).
Consistently Monitor Feedback Data: All of our therapists consistently monitor their “toolkits.” Toolkits refer to the data analysis portal that Acorn provides us. Our toolkits allow us to consider are overall effectiveness, and more importantly track the progress of each of our clients.
Overall, we are extremely proud of our efforts, our ability to helpful to our clients. Moreover, we remain committed to using cutting edge research to improve our efforts to be helpful to all we serve.
More information on Effectiveness: definitions, research and more detail
Defining effectiveness: what do we mean when we say therapy is effective? The general consensus among outcome researchers (those who study whether psychotherapy is helpful) is that the benefits or effectiveness therapy is best determined by the client. Thus, whether clients report reduced levels of distress and less intense and/or frequent symptoms determines whether and to what extent therapy is effective. The Feedback Informed Therapy system that Centers for Family Change utilizes a measure of global level of distress (which includes symptoms of anxiety and depression) as the primary measure of outcome. Therapy is thought to be effective when clients’ show a reduction in their global level of distress.
Seeking client feedback improves effectiveness: Research has consistently found that psychotherapy is effective, helps the majority of people who participate in therapy. Moreover, research has found that most therapists are generally helpful, with a small minority being particularly effective. However, one the most interesting findings in the study of therapeutic effectiveness is that the use of Feedback Informed Therapy, regularly and systematically seeking feedback from clients, improve therapists’ effectiveness, ability to help their patients (3).
The Acorn system of Feedback Informed Therapy: The Centers for Family Change uses the FIT rating scales and data analysis developed by Acorn (also known as the Center for Clinical Informatics (4). The FIT rating scales developed by Acorn, are research based and have sound psychometric properties. Put in more basic terms these scales are reliable and valid, provide an accurate measure of what they assert they measure (reliability and validity are the key determinants of the usefulness of any rating scale or psychological test, i.e., does the test consistently measure what it claims to measure). Thus, the use of psychometrically sound, reliable and valid scales, is critical to any enterprise which claims to measure psychotherapy outcome.
Updates on research on FIT
This section is currently under development!
- The Acorn website link which provides information about the FIT system utilized by Acorn, and a great deal more information: https://psychoutcomes.org
- Definition and discussion of “effect size:
- Studies showing support for the value of utilizing feedback informed therapy
Sapyta J, Riemer M, Bickman L. 2005. Feedback to therapist: theory, research, and practice. J Clin Psychol 61(2):145-53.
Hannan C, Lambert MJ, Harmon C et al. 2005. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol 61(2):155-63.
Lambert MJ, Harmon C, Slade K et al. 2005. Providing feedback to psychotherapists on their patients progress: Clinical results and practice suggestions J Clin Psychol 61(2):165-74.
Harmon C, Hawkins, Lambert MJ et al. 2005. Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. J Clin Psychol 61(2):175-85.
Brown GS, Jones ER. 2005. Implementation of a feedback system in a managed care environment: What are patients teaching us? J Clin Psychol 61(2):187-98.
Miller SD, Duncan BL, Ryan S, et al. 2005. The Partners for Change Outcome Management System. J Clin Psychol 61(2):199-208.
Claiborn CD, Goodyear EK. 2005. Feedback in psychotherapy. J Clin Psychol 61(2):209-21.
Lambert MJ, Whipple JL, Smart DW, et al. 2001. The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychother Res 11(1):49-68.
Lambert MJ, Whipple JL, Vermeersch DA, et al. 2002. Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clin Psychol Psychother 9:91-103.
Whipple JL, Lambert MJ, Vermeersch DA, et al. 2003. Improving the effects of psychotherapy: The use of early identification of treatment failure and problem-solving strategies in routine practice. J Counsel Psychol 50(1):59-68.
Lambert MJ, Whipple JL, Hawkins EJ, et al. 2003. Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clin Psychol Sci Prac 10:288-301.
- See https://psychoutcomes.org/DataCenter/CenterForClinicalInformatics for more information on Acorn, their work, and articles and materials on Feedback Informed Therapy.
- See https://psychoutcomes.org/Questionnaires/ItemInventory for detailed information on the reliability and validity of the FIT scales, and other information on these rating scales.