Observations of family interaction have been essential in the development and evaluation of the treatment program Parent Management Training – the Oregon model (PMTO: Forgatch & Patterson, 2010).
In research evaluating the PMTO, the families are videotaped while the members participate in structured interactions before therapy, at treatment termination and at follow up. Trained coders observe the videotapes and rate the families on the global observation measure Coders Impressions. This measure has been found to predict outcomes of PMTO treatment. It reflects the trained coders’ inferences regarding the parenting skills of the parents, the emotional climate and the level of interaction between the family members. The global observations which are rated in the CI carry similarities to clinical observations. PMTO therapists are thoroughly trained in clinical work with the parenting practices that are targeted in this global observation measure, but PMTO therapists have not been used as observers in PMTO research.
The aim of the studies included in this thesis is to contribute to psychometric evaluation of the global CI measure when used by PMTO therapists without coder training; and also when used by coders trained to reliability.
We studied reliability of trained coders’ global observations of mothers’ parenting to estimate the impact of varying numbers of raters (Paper I).
We studied the reliability of PMTO therapists’ global observations of mothers’ parenting within a G-theory framework (Paper II).
We studied concurrent validation of PMTO therapists’ global observations of mothers’ parenting and used microsocial observations of mother-child interactions by trained coders as a validation criterion (Paper III).
The families in these studies had children with conduct problems, and they were participants in two Norwegian PMTO studies. The two first papers in this thesis are based on different samples of families and observers. In Paper 1 30 families were observed by 6 trained coders. In Paper II 10 families were observed by five PMTO therapists and two trained coders. These data were part of the larger sample included in Paper III, where a total of 52 families were observed by 52 therapists and six trained coders.
Global observations were rated on the CI measure across the three studies. The CI measure was completed by trained coders in Paper I and by both therapists and trained coders in Papers II and III. The two rater types completed the global CI measure after observing videotapes of the families in structured interaction. The main objective of the CI measure is to reflect the observers’ assessments of the quality of the parents’ parenting across the five PMTO parenting practices: discipline, skill encouragement, positive involvement, problem solving and monitoring.
(Paper I) Three and four raters were needed to obtain generalizability coefficients in the .70-.80 range for monitoring and discipline, respectively. One rater was sufficient for a corresponding estimate for positive involvement and for an estimate in .80-.90 range for problem solving. Estimates for skill encouragement were non-acceptable. The paper demonstrated the importance of congruence between the actual assessment design and the estimation model.
(Paper II). Both therapists and trained coders reliably rank-ordered the mothers in the referred families on their parenting skills; and reliably assessed the level of successful parenting. The results indicated that PMTO therapists without coder training provided reliable ratings of parenting constructs relevant to the clinical PMTO program in a manner comparable to that of the trained coders.
(Paper III) The results from the study of concurrent validation of global observations by PMTO therapists supported the validity of the therapists’ observations: Less adequate parenting as observed by the therapists was associated with more negative chains of behavior that was initiated by the mother.
The present studies have laid the groundwork for future clinical research by assessing the reliability and validity of PMTO therapists’ inferences about parenting. Further studies are required to assess the possible benefits of including observations of structured family interactions by therapists in the clinical program.
The overall findings from our studies support the generalizability and validity of PMTO therapists’ and trained coders’ ratings of mothers’ parenting.