Article Citation:
Frank M. Gresham,
Evan H. Dart, and
Tai A. Collins (
2017) Generalizability of Multiple Measures of Treatment Integrity: Comparisons Among Direct Observation, Permanent Products, and Self-Report. School Psychology Review: March 2017, Vol. 46, No. 1, pp. 108-121.
General Articles
Generalizability of Multiple Measures of Treatment Integrity: Comparisons Among Direct Observation, Permanent Products, and Self-Report
Frank M. Gresham
Louisiana State University
Evan H. Dart
University of Southern Mississippi
Tai A. Collins
University of Cincinnati
Correspondence concerning this article should be addressed to Frank M. Gresham, PhD, Department of Psychology, Louisiana State University, 119 Audubon Hall, Baton Rouge, LA 70803; e-mail:
Frank M. Gresham, PhD, is a professor in the school psychology program in the Department of Psychology at Louisiana State University. His research interests include data-based decision making within a problem-solving model of consultation, identification and intervention for students with emotional and behavioral disorders, social skills assessment and training, and implementation of response-to-intervention models in schools.
Evan H. Dart, PhD, is an assistant professor in the school psychology program in the Department of Psychology at the University of Southern Mississippi. His research interests include peer-mediated behavioral interventions, the identification of school-based consultation strategies to promote teacher treatment adherence, and the integration of technology into evidence-based intervention strategies.
Tai A. Collins, PhD, is an assistant professor in the school psychology program at the University of Cincinnati. He earned his doctorate in school psychology from Louisiana State University. His research interests include the implementation of peer management and group contingency interventions in schools and developing strategies to reduce the overrepresentation of African American students in special education and their underrepresentation in gifted education programs.
Editor: Amy Reschly
The concept of treatment integrity is an essential component to data-based decision making within a response-to-intervention model. Although treatment integrity is a topic receiving increased attention in the school-based intervention literature, relatively few studies have been conducted regarding the technical adequacy of treatment integrity assessment methods. In light of recent research utilizing generalizability theory (G theory) to assess the dependability of behavioral measurement in schools, the current study used G theory to examine the dependability of direct observation, permanent products, and self-report as measures of treatment integrity when six teachers implemented the Good Behavior Game across three study sites. Results indicated that direct observation yielded the most reliable treatment integrity data, followed by permanent products and self-report. Specifically, when assessment of treatment integrity is conducted twice per week, direct observation should provide a dependable estimate (φ = .821) after only four assessments. Implications, limitations, and future directions are discussed.
Received: September 30, 2015; Accepted: May 15, 2016;
Copyright 2017 by the National Association of School Psychologists
Previous Article
