Numerous assessment instruments have been designed to measure social communication impairments and restricted, repetitive behaviors associated with autism spectrum disorders (ASD). Use of ASD assessment instruments has extended to a wide range of clinical and research contexts, as awareness has grown about the benefits of early diagnosis and intervention, and about the frequent occurrence of ASD symptoms in children with other psychiatric concerns. However, factors influencing the validity of ASD assessment instruments are not well understood. This can have negative implications, such as misdiagnoses and erroneous estimates of associations between ASD features and causal factors. The present set of studies investigated factors influencing scores on ASD assessment instruments, using multiple instruments, sources of information, and independent samples.
Participants were 407 children aged 2-13 years recruited from clinics in the USA (studies I and III) and 679 toddlers aged 35-47 months recruited from the population-based Norwegian Mother and Child Cohort (MoBa) Study (study II). All children received a comprehensive diagnostic assessment for ASD according to current best-practice standards.
Study I examined how child characteristics influenced the diagnostic validity of three widely used ASD assessment instruments: the Social Responsiveness Scale (SRS), the Autism Diagnostic Interview-Revised (ADI-R), and the Autism Diagnostic Observation Schedule (ADOS). High rates of false positive instrument classifications of ASD were observed for children without ASD who had intellectual disability or elevated emotional/behavioral problems such as mood or disruptive behavior problems. The associations of emotional/behavioral problems with scores on ASD assessment instruments were relatively informant specific. Adjustment for the influence of cognitive abilities and emotional/behavioral problems on the optimal threshold for distinguishing between children with and without ASD resulted in improved diagnostic validity for all of the instruments.
Study II investigated whether parental concern about ASD affected the utility of diagnostic instruments for identifying toddlers with ASD. The findings showed that parental concern about ASD strongly influenced reporting of ASD features on the ADI-R, independent of the child ́s cognitive and language abilities and severity of directly observed ASD features on the ADOS. In the subgroup without a specific parental concern about ASD, nearly half of toddlers with ASD were missed by the ADI-R. However, adjustment of ADI-R cut-offs depending on parental concern greatly improved the tool’s ability to identify toddlers with ASD also in the absence of parental concern about ASD.
Study III evaluated how factors beyond ASD symptoms affected the utility of the Child Behavior Checklist (CBCL) as a screener for ASD. The CBCL is a general behavior rating scale, but specific score profiles have recently been recommended for detection of children in clinical settings who are in need of a full ASD evaluation. These profiles showed poor ability to differentiate between children with ASD and children with other clinical disorders. The proposed cut-offs classified nearly all children with mood or disruptive behavior problems as being at risk for ASD, regardless of whether they actually had ASD or a non-ASD diagnosis. Conversely, the cut-offs missed more than a third of children with ASD who had low levels of mood and disruptive behavior problems. Even after adjusting for emotional/behavioral problems, the CBCL only showed minimally acceptable screening properties for detecting school-aged children with ASD with low levels of mood and disruptive behavior problems.
The findings indicate that scores on widely used ASD assessment measures reflect far more than ASD features. Scores on these instruments may be substantially affected by child-level factors, including cognitive abilities and emotional/behavioral problems, as well as informant characteristics.
Reasonable steps could be taken to enhance measurement precision. Strategies include stratified analyses and statistical adjustment, as well as creation of scores, cut-offs, or even separate instruments that more appropriately account for these child and parent characteristics. Future studies are needed to examine this. Nevertheless, when measures are applied beyond the contexts in which they were designed and validated, or used with populations not necessarily represented in validation studies, it is essential that clinicians and researchers take appropriate precautions to ensure valid interpretation of scores.