Child attachment Checklist scoring

The Relationship Problems Questionnaire (RPQ) was developed to screen symptoms of the inhibited and disinhibited subtype of reactive attachment disorder (RAD). This study further examines the psychometric properties of the RPQ in children with severe emotional and behavioural problems by testing its measurement invariance across informants and its convergent validity. Parents and teachers of 152 children [mean age (M age) = 7.92] from 20 schools for special education filled out the RPQ and the Strengths and Difficulties Questionnaire (SDQ). During a home visit in a subsample of 77 children the Disturbances of Attachment Interview (DAI) was administered to the caregiver and the child was observed using an observational schedule for RAD. Exploratory and confirmatory factor analyses revealed the expected two‐factor structure for both parent and teacher RPQ. Configural and metric invariance, but no scalar invariance, were obtained across informants. Both RPQ‐subscales had acceptable to good internal consistencies and correlated as expected with similar DAI‐subscales. Furthermore, the disinhibited RPQ‐scale related with observations of the child's approach to a stranger. Finally, significant associations were found between the RPQ and the SDQ. Overall, the RPQ has good psychometric qualities as a multi‐informant instrument for RAD‐symptoms in children with severe emotional and behavioural problems. Copyright © 2013 John Wiley & Sons, Ltd.

Keywords: reactive attachment disorder, screening instrument, relationship problems questionnaire, validation

Among childhood psychiatric disorders, reactive attachment disorder (RAD) remains one of the most controversial and least understood. Since the criteria first appeared in psychiatric nosologies in the Diagnostic and Statistical Manual of Mental Disorders, Third Edition (DSM‐III) (American Psychiatric Association, 1980), they have been widely criticized and consensus is still lacking (Zeanah & Gleason, 2010). Although research into RAD expanded, most studies have been conducted among Romanian orphans, and are limited to children younger than six years owing to controversy on the applicability of the criteria in older children (Zilberstein, 2006). Systematic studies of the diagnosis and treatment of RAD are lagging behind. Relatedly, well‐validated instruments for screening or assessing RAD are missing, especially in school‐age children.

The lack of validated instruments is problematic for both clinical practice and research. A more accurate identification of RAD would enable clinicians to refer and/or treat children with RAD‐symptoms and would enable researchers to study the developmental course, possible outcomes, and the efficacy of interventions. To assist clinicians and researchers in the screening of RAD at school‐age, Minnis et al. (2002) developed the Relationship Problems Questionnaire (RPQ) with promising reliability and validity (Minnis et al., 2009; Minnis et al., 2002; Minnis et al., 2007). However, a more extensive psychometric evaluation with multiple informants and methods is appropriate to justify its use in a broader clinical sample.

The current multi‐method study seeks to provide psychometric evidence for the RPQ as a multi‐informant tool for RAD‐symptoms in a clinical group of children with severe emotional and behavioural problems. More specifically, we will investigate whether the constructs measured with the RPQ are similar across informants and whether the RPQ converges with other RAD‐measures. Then, the RPQ may enable clinicians and researchers to efficiently and accurately probe RAD‐symptoms and to identify for which children additional interventions addressing relational problems may be needed to improve child outcome.

Establishing indicators of RAD is complicated as criteria have changed over time. Nevertheless, the description of two types of RAD has been retained since their introduction in the DSM‐III (Zeanah & Gleason, 2010). The inhibited type is characterized by a lack of social approach: these children exhibit emotionally withdrawn and hypervigilant responses, and seem frightened of others. Children with the disinhibited type, on the contrary, are oversociable, fail to show selective attachments and demonstrate clinging and/or indiscriminate friendliness, even in their interactions with unfamiliar persons (Zeanah, 1996; Zilberstein, 2006). The DSM‐IV (American Psychiatric Association, 2000) brings both types together under a single category, whereas the International Classification of Diseases, 10th revision (ICD‐10) (World Health Organization, 1992) describes two separate disorders. Both nosologies further agree that the disorder is present before the age of five, occurs in most contexts and does not result from the presence of a developmental disorder (e.g. autism spectrum disorders, ASD) or cognitive delay, but that it is linked with pathogenic care (Zeanah, 1996). Recently, Zeanah and Gleason (2010) suggested a revision of the DSM‐IV criteria for the DSM‐5. Although they proposed to conceptualize the RAD‐types as two distinct disorders instead of two subtypes of a single disorder, they largely retained the behavioural criteria. Pending the publication of the DSM‐5, we will use the term RAD to cover both types as in the DSM‐IV.

Despite the availability of guidelines for the assessment of RAD (e.g. AACAP, 2005; Chaffin et al., 2006), the need for well‐validated standardardized screening instruments to assist the identification of children that might suffer from RAD still remains. Due to the paucity of research this particularly holds true for older children. To address this need, recently, some efforts have been made to develop instruments specifically probing RAD‐symptoms. One of these instruments is the RPQ.

The RPQ was developed as a screening instrument for RAD to be completed by caregivers of children over five. Items were based on the symptoms of institutionalized children from 18 months till 17 years old in a qualitative case study plus the ICD‐10 and DSM‐IV diagnostic criteria (Minnis et al., 2002). In a study with foster carers of children between 5 and 16 years old, the original version had a sufficient internal consistency (Cronbach's α = 0.70), and the interrater reliability and test–retest reliability were adequate (intraclass correlation coefficient (ICC) values of 0.81 and 0.78, respectively) (Minnis et al., 2002). In another study with an adapted version (Cronbach's α = 0.85) in twins aged 7–9 years factor analysis revealed 10 items that loaded on two factors referring to behaviours reflecting the inhibited and disinhibited form of RAD, respectively, with two of the inhibited items not being part of diagnostic criteria (Minnis et al., 2007). A recent study in which these 10 items were adopted in a teacher version of the RPQ suggested that teachers may be reliable informants of RAD‐symptoms as well (Minnis et al., 2009). As the teacher is an important informant to signal problem behaviour in school‐aged children in addition to parental caregivers, we will test the psychometric qualities of both versions and examine whether both perspectives are comparable.

The main goal of the present study is to examine the psychometric qualities of the RPQ as a multi‐informant instrument for RAD‐symptoms in school‐aged children. Unlike previous studies, evidence was obtained in a broader clinical group of children in elementary schools providing special education for children with emotional and behavioural disorders. Children in these schools show a wide range of behavioural and emotional regulation problems, and family dysfunction. If the RPQ is found to be reliable and valid in this sample, this study would show that the RPQ is a useful instrument to signal inhibited and disinhibited RAD‐symptoms in special education.

To check the psychometric qualities of the RPQ, we first aimed to demonstrate the factorial validity by replicating the two‐dimensional structure of inhibited and disinhibited RAD in a broader clinical sample. Next, we assessed reliabilities of both subscales. Because verifying an instrument's measurement invariance is equally important as verifying its reliability and validity (Vandenberg & Lance, 2000), we subsequently extended previous research by testing three types of measurement invariance across informants, that is configural, metric, and scalar invariance (cf. Chen, 2008). In the case of configural invariance, the same items are associated with the same underlying construct (i.e. inhibited or disinhibited RAD) in parent and teacher RPQ. Metric invariance additionally implies that the factor loadings are equal across informants which means that the strength of the relations between an item and a construct are the same for parents and teachers. In this case, the underlying constructs have an equivalent meaning across informants and predictive relationships of parent and teacher RPQ scores may be compared. If both versions are scalar invariant, parent and teacher scores have the same origin (i.e. intercept) as well and factor means may be meaningfully compared across informants. Thus, in order to use the RPQ as a multi‐informant tool and to compare correlates of both versions configural and metric invariance should be obtained; scalar invariance is needed only when directly comparing RPQ‐scores for both informants. Then, we checked the convergence between the RPQ‐subscales, a caregiver interview (the Disturbances of Attachment Interview, DAI), and a home observation. We expected both RPQ‐subscales, and particularly the parent RPQ, to converge with similar subscales of the interview. Additionally, we anticipated that disinhibited behaviours, which are the most observable, would show the most significant correlations with observed child interactive behaviours, in particular with approach to strangers, demanding behaviour, and less so with emotional/interactional problems. Finally, we examined relations between the RPQ‐subscales and multi‐informant reports of behavioural and emotional problems. Based on recent research regarding RAD‐subtype correlates (Gleason et al., 2011), we expected moderate positive associations of the inhibited and disinhibited subscales with internalizing and externalizing problems, respectively.

Approval for this study was obtained from the institutional ethical committee of the KU Leuven, Belgium. All 38 schools providing special education for children with emotional and behavioural disorders in Flanders (Belgium) were contacted and asked to participate in the study. The principals and school teams of 21 schools agreed to participate. Next, the primary caregivers of all children born between 2001 and 2004 (n = 425) received a letter with information and were asked for their permission for the child's participation. No specific exclusion criteria were used because we aimed to validate the RPQ for all children with emotional and behavioural problems. Nevertheless, we will control for ASD and general intelligence in the analyses.

Caregivers of 178 children (participation rate of 42%) agreed to participate and filled out the RPQ. Almost 90% of the caregivers was a biological parent of the child, but only 30% of the caregivers daily took care of the child together with the other biological parent. Approximately 13%, 25%, and 42% of the caregivers completed primary, lower secondary, and higher secondary school, respectively, and merely 20% obtained a degree of higher education. The children's teachers were asked to fill out a similar questionnaire, if parental permission for it was obtained (94%). Additionally, the school psychologists provided information on child psychiatric diagnoses and general intelligence. We received completed parent and teacher questionnaires for 152 children. These children (87% boys) were between 5.67 years and 10.08 years old (M age = 7.92 years). They had a mean IQ of 85.95 [standard deviation (SD) = 15.42), ranging from 50 to 141. According to their files, the majority of the children (80%) had one or more psychiatric diagnoses of whom about 25% were diagnosed with or suspected to suffer from RAD.

Participants in the home part of the study were selected from the 152 children for whom both parent and teacher questionnaires were available.1 Selection was based on parental permission for a home visit (n = 120) and the RPQ‐scores. The caregivers of the 75% children with the highest scores on parent or teacher RPQ were contacted by telephone to further inform them about the home visits. Seventy‐seven visits (85%) were made. A recruitment flow diagram is available as online supporting information.

At the start of each home visit, caregivers gave informed consent for the second time, in which they also allowed the school psychologists to provide additional background information from the personal files of the children. According to the school psychologists, family risk factors such as tensions in the partner relation, parental psychological problems, problematic parenting history of one or both parents, and large differences in parenting style were (suspected to be) present in about half of the participants. More than three‐quarters of the families frequently dealt with stress or crises. The school psychologists indicated (suspicions of) a history of pathogenic care as evidenced by the presence of physical or emotional maltreatment or neglect or sexual abuse for 48% of the children.

Each home visit was completed by one researcher and lasted about 2.5 hours. In addition to the administration of a caregiver interview, the child's interactive behaviour with the caregiver was observed and reactions toward the (unfamiliar) researcher were used to rate interactive behaviour toward a stranger. Both activities were piloted and evaluated by two researchers in three home visits.

Relationship Problems Questionnaire (RPQ)

Both caregiver and teacher completed the 10 RAD‐items of the RPQ (cf. the bold items in the online appendix to Minnis et al., 2007). Four items intend to measure disinhibited behaviour (e.g. “Gets too physically close to strangers”), while six items aim to probe inhibited behaviour (e.g. “Sometimes looks frozen with fear, without an obvious reason”). Each item has four possible responses (“Not at all like”, “A bit like”, “Like”, “Exactly like”) scored 0, 1, 2 and 3. The original items and responses were translated to Dutch using forward‐backward blind translation (100% agreement).

Disturbances of Attachment Interview (DAI)

The DAI is a semi‐structured interview that was originally created for administration to familiar caregivers of one‐ to five‐year‐old children, and later adapted for use with school‐aged children (Zeanah & Smyke, unpublished manuscript). It gives an indication of the presence of inhibited and disinhibited behaviour, supplemented with information on the occurrence of four secure‐base distortions based on the alternative criteria of Zeanah and Boris (2000). Prior studies have shown interrater reliability (κ = 0.88) and internal consistencies of the inhibited and disinhibited subscale (α values were 0.80 and 0.83, respectively) to be very good (Smyke et al., 2002). Furthermore, Gleason et al. (2011) found convergence between the DAI and a psychiatric diagnostic interview, and between the disinhibited subscale and an observational measurement. They also demonstrated its discriminant validity by relating the inhibited and disinhibited type with depression and attention deficit hyperactivity disorder (ADHD), respectively.

Prior to data collection, the first author was trained during a two‐day session to administer and score the Dutch version of the DAI at the VU Amsterdam, in which the second author was one of the trainers. She was herself trained to identify symptoms of attachment disturbances by Neil Boris and Robert Marvin. For this study, we used the school‐aged version which consists of 11 questions on RAD‐symptoms. The four additional items on secure‐base distortions were not included in the analyses. Each question is scored with 0, 1, or 2 depending on the extent to which the probed attribute is present (no, some, and consistent evidence for an attachment disorder symptom). Principal component analyses (PCAs) with varimax rotation revealed a two‐factor structure with one factor reflecting inhibited (four items) and another representing disinhibited behaviour (seven items). Cronbach's α values were 0.53 and 0.80, respectively. Approximately 30% of the DAIs (n = 22) were double rated with the second rater completely blind to additional information. Interrater reliability was good for both subscales (Spearman Rho correlations of 0.89 and 0.91).

Observational schedule

In addition to the RPQ, Minnis and colleagues also developed a structured observational schedule for RAD (McLaughlin et al., 2010). They formulated key areas for observation based on a literature review which were reviewed by experts. The resulting 20‐item tool had respectable internal consistency (α = 0.75), but only half of the items had good interrater reliability and were specific in discriminating children with RAD from the comparison group (McLaughlin et al., 2010). In this study, these 10 items were used to evaluate the child's observed behaviour. Each item is answered by “yes” (1) or “no” (0), indicating the presence or absence of a behaviour. To reduce the number of variables, three interpretable subscales were created based on PCA with varimax rotation. Four items loaded highly on a factor Approach to Stranger(s) (e.g. “Does the child initiate conversation with the stranger(s) as if previously familiar?”, “Does the child move towards and approach the stranger?”). The other two subscales, Emotional/Interactional Problem Behaviour (e.g. “Does the child display sudden shifts to the extremes of emotion?”, “Does the child refuse or ignore a request from his/her carer?”) and Demanding Behaviour (e.g. “Does the child display a seemingly insatiable demand for attention?”, “Does the child interrupt conversation between the stranger and his/her carer?”), consisted of three items each (α values were 0.69, 0.73, and 0.42, respectively). Higher scores on each subscale indicated more negative child behaviour.

Strengths and Difficulties Questionnaire (SDQ)

Parents and teachers also completed the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997), a 25‐item questionnaire with subscales for emotional problems, conduct problems, hyperactivity, problems with peer relationships, and prosocial behaviour. This screening instrument for behavioural and emotional problems has been well validated (Goodman et al., 2003; Van Leeuwen et al., 2006). In this study, Cronbach's α values for the Dutch version's subscales were generally sufficient, ranging from 0.63 to 0.75 for the parent and from 0.69 to 0.81 for the teacher questionnaire. Consistent with previous studies (e.g. Van Leeuwen et al., 2006), only the parent subscale for peer problems had lower internal consistency (α = 0.52).

To identify the latent variables that cause the manifest variables to covary, we used exploratory factor analysis. We preferred factor analysis over PCA, because during factor extraction the shared, unique and error variance of a variable are separated. Only the shared variance appears in the solution with the underlying factor structure (Costello & Osborne, 2005). As the assumption of multivariate normality was not severely violated, maximum likelihood is the recommended extraction method (Fabrigar et al., 1999). After extraction, the scree plot indicated that two factors should be retained for both the parent and teacher questionnaire. The two‐factor solution explained 36% and 42% of total variance respectively. The items loaded on the Inhibited and Disinhibited subscale as expected. For the Inhibited subscale, factor loadings ranged from 0.31 to 0.61 for the parent version and from 0.37 to 0.81 for the teacher version. For the Disinhibited subscale, factor loadings varied from 0.35 to 0.91 and from 0.42 to 0.93, respectively. Only one item showed a cross‐loading (> 0.30), namely the item “often asks very personal questions” from the parent version of the Disinhibited subscale.

Cronbach's α values for the parent RPQ were 0.63 for the Inhibited subscale, and 0.77 for the Disinhibited subscale. For the teacher version, Cronbach's α values were 0.71 and 0.81 for the Inhibited and Disinhibited subscales, respectively.

Confirmatory factor analysis (CFA) were conducted using Mplus version 6 (Muthén & Muthén, 1998–2010). We tested the a priori factor structure (i.e. two uncorrelated factors Inhibited and Disinhibited RAD) for both the parent and teacher version of the RPQ using the maximum likelihood method. The estimated models were considered acceptable when Comparative Fit Index (CFI) ≥ 0.90 and good when CFI ≥ 0.95 (Bentler, 1992; Hu & Bentler, 1999). In addition, for a good fit, standardized root mean square residual (SRMR) and root mean square error of approximation (RMSEA) values should not exceed 0.08 and 0.06, respectively (Hu & Bentler, 1999). RMSEA values ≤ 0.08 are considered reasonable (Kline, 2005).

Before conducting the CFAs, we created parcels following the assumption that parcels provide a more stable set of manifest variables on which to base structural models (Widaman et al., 2010). We used the well‐established item‐to‐construct balance parceling method (Little et al., 2002) to create the optimal number of three parcels for each factor of RAD resulting in a total of six parcels. These parcels were used as input for CFAs in which we tested for three types of measurement invariance. Tests for configural, metric, and scalar invariance respectively reveal whether the multi‐dimensional structure, factor loadings, and intercepts of the RPQ‐items are invariant for parents and teachers. After constraining the models according to measurement invariance, we used the criteria for small sample sizes (N ≤ 300) of Chen (2007) to evaluate the loss in model fit. Fit indices for these models are presented in Table 1.

Goodness of fit indicators of models for parent and teacher report of RAD (n = 152)

First, CFAs were performed for parent and teacher version separately to test the hypothesized two‐factor model. The model fit was good for the parent questionnaire and nearly perfect for the teacher questionnaire. To test configural invariance, an unconstrained, multigroup CFA with two groups (parents and teachers) was conducted, which still resulted in good fit indices. Thus, the two‐factor model proved to be a good solution for both parents and teachers as informants.

Second, metric invariance was evaluated by comparing the configural invariance model in which factor loadings were freely estimated for each of the groups, with the fit of a model in which factor loadings were constrained to be equal for both groups. As these constraints did not result in a significant loss in model fit (ΔCFI > 0.005, but ΔRMSEA < 0.01 and ΔSRMR < 0.025), we concluded that metric invariance was supported.

Finally, scalar invariance was examined by comparing the metric invariance model in which factor loadings were constrained, with a model in which both factor loadings and intercepts were constrained to be equal for both groups. The scalar invariance model had a less good fit than the metric variance model (ΔCFI > 0.005, ΔRMSEA > 0.01, and ΔSRMR > 0.005). Hence, the assumption of scalar invariance needs to be rejected.

The intercorrelation between parent and teacher questionnaire was significant for the Disinhibited subscales (r = 0.32, p < 0.01). No association was found between the Inhibited subscales or across both subscales.

Convergence between the RPQ and the DAI was examined after controlling for ASD and general intelligence. As expected, for parent report, the Inhibited and Disinhibited subscales of both instruments showed a moderate, positive association (Table 2). With regard to the teacher RPQ, only the Disinhibited subscale related modestly positively with the Disinhibited subscale of the DAI (r = 0.26, p < 0.05).

Correlations between the subscales of the RPQ and the DAI (n = 77) after controlling for ASD and general intelligence

To examine the convergence between the reported RAD‐behaviours with observed child behaviours, the three observational subscales were correlated with the subscales of the RPQ and the DAI after controlling for ASD and general intelligence. Because the first author scored both the observational checklist and the DAI for half of the children, we controlled for coder dependency while correlating these measures as well.

As presented in Table 3, the Disinhibited subscales of the RPQ (parent and teacher) and the DAI showed moderate to strong positive correlations with the observed Approach to Stranger. In addition, we found a moderate, positive association between the Disinhibited subscale of the parent RPQ and DAI, and the child's Demanding Behaviour. This observational subscale also showed a moderate negative correlation with the Inhibited subscale of the teacher RPQ. With regard to the DAI, the Inhibited subscale was also moderately correlated with Emotional/Interactional Problem Behaviour.

Correlations between the observation scales and the subscales of the RPQ versus the DAI (n = 76a) after controlling for ASD and general intelligence, and coder dependency for correlations with the DAI

The different instruments measuring RAD‐symptoms were correlated with the SDQ‐scales after controlling for ASD and general intelligence. Most significant associations were within informants (Table 4). Regarding the RPQ, the Inhibited subscale was not only positively associated with Emotional Problems, but also showed moderate to strong, positive correlations with Conduct and Peer Problems, and a strong, negative association with Prosocial Behaviour. The Inhibited DAI‐scale was modestly to moderately, positively correlated with Emotional Problems and Peer Problems, and modestly, negatively with Prosocial Behaviour. In contrast, the Disinhibited subscales showed less significant associations. For the parent RPQ, this scale was moderately, positively correlated with Peer Problems, and modestly with Hyperactivity and Conduct Problems. The Disinhibited subscale of the teacher RPQ showed only one small positive correlation with Peer Problems, and the Disinhibited DAI‐scale was not associated with the SDQ.

Correlations between the subscales of the SDQ and the RPQ (n = 152) versus the DAI (n = 77) after controlling for ASD and general intelligence

The current study intended to replicate and extend psychometric evidence for the RPQ as a continuous multi‐informant measure for RAD‐symptoms in school‐aged children with severe emotional and behavioural problems. Results indicated that the factorial validity of the RPQ was confirmed, the internal consistencies were acceptable to good, and invariance of the two‐dimensional structure and the factor loadings across informants was demonstrated. Convergence was found between the RPQ and the DAI, and between the disinhibited subscales and observations of child behaviour. Inhibited behaviours were particularly correlated with emotional and behavioural problems assessed with the SDQ. In sum, this study provides evidence for the psychometric qualities of the RPQ.

First, the items of the parent and teacher RPQ loaded on the inhibited and disinhibited subscales as expected. One item from the parent disinhibited scale (“often asks very personal questions”), however, showed a cross‐loading which may be due to ambiguous interpretation. Some parents might have interpreted this item to mean that the child asked them very personal questions rather than asking unfamiliar people personal questions, which would reveal disinhibited behaviour. Therefore, we would suggest making this more explicit, as in the other three disinhibited items.

Second, the RPQ was found to have the same multidimensional structure from parents' and teachers' perspectives (i.e. configural invariance). Moreover, the strength of the relation between a particular behavior (e.g. getting too physically close to strangers) and the underlying construct (e.g. disinhibited) was proven to be the same for parents and teachers (metric invariance). A particular score for the two constructs, however, does not necessarily mean the same for both informants (scalar non‐invariance). This implies that scores of parents and teachers cannot be compared directly and that normative assessment should occur for both informants separately. The different meaning of scores may be due to parents' and teachers' differing frames of reference (i.e. home and school context), each having its advantages and disadvantages. Generally, the inclusion of both informants in clinical assessment is useful to obtain information on the occurrence of RAD‐behaviours in different contexts. For this purpose, the RPQ seems valuable as it measures similar constructs and the items represent the constructs equally well for parents and teachers. For research purposes, factorial invariance is sufficient and these results allow us to compare correlates of parents' and teachers' perceptions of inhibited and disinhibited child behaviours.

Third, the current study was the first to compare the RPQ with the DAI. Although correlations were quite modest probably because of differences in method and scoring (e.g. caregiver self‐report likert‐type questionnaire versus expert coding of presence of maladaptive behaviour from interview), the expectation of convergence between the parent RPQ‐subscales and the caregiver interview (DAI) was confirmed. More specifically, the disinhibited subscales of both instruments were interrelated and a similar convergence was found between the inhibited subscales, which confirms the specificity (of relations) of both subtypes. Convergence between teacher RPQ and parent DAI was lower, although both informants agreed to some extent on the presence of disinhibited behaviours. Similarly, the disinhibited subscales of RPQ and DAI, as compared to the inhibited, related most clearly to the observation of child interactive behaviour at home. Children with higher scores on the disinhibited RPQ or DAI showed more behaviours such as initiating conversation with the unfamiliar researcher as if familiar or interrupting conversation between the researcher and the children's carer. Probably, the presence of disinhibited behaviours is more easily observable by different informants, leading to stronger convergence. Inhibited RAD, in contrast, refers to the absence of social behaviours, which is more difficult to observe. Nevertheless, the RPQ‐ and DAI‐subscales overall show similar relations with the observational subscales which provides additional evidence for the validity of the RPQ.

Finally, the inhibited subscales of RPQ and DAI, as compared to the disinhibited, are more significantly related to behavioural and emotional problems as assessed with the SDQ. In addition to the expected association with internalizing problems, the inhibited subscales were also related to externalizing problem behaviours, which might indicate that inhibited behaviours have an impact on children's general functioning. Other possible explanations, such as comorbidity or symptom overlap which may be more pronounced in older children, come to mind as well. However, this finding may also reveal a measurement issue since the inhibited RPQ items seem to be less specific than the disinhibited items, which may have caused stronger correlations with the SDQ. Therefore, we suggest to further investigate the specificity of the inhibited subscale in addition to the expression of (inhibited) RAD in school‐aged children. Fewer associations were found with the disinhibited subscales. The disinhibited RPQ‐scales were most consistently related to peer relationship problems which confirms a similar finding in 11‐year old children (Rutter et al., 2007) and may indicate that children showing disinhibited behaviours are more at risk for relational problems specifically. Furthermore, disinhibited behaviours were to some extent related to hyperactivity according to the parent which is in line with recent research on disinhibited RAD and ADHD‐symptoms (Gleason et al., 2011). Despite the significance of associations between RPQ and SDQ, especially for the inhibited subscale, these were all small to moderate and thus, supported the discriminant validity of the RPQ.

Even though this study contributes significantly to the field, some limitations should be noted. A first limitation pertains to the study's sample size. While our sample covered a fairly high proportion of children from special education for behavioural and emotional disorders in Flanders, its size was still too limited to permit separate analyses according to child characteristics (e.g. gender, psychiatric diagnosis). The second limitation is related to the absence of a more traditional evaluation of screening instruments in this study. Although we received information on child diagnoses of RAD, we were not able to test the questionnaire's performance in identifying RAD‐diagnoses, such as its predictive values, for two reasons. First, we lacked control on the standardization and validity of psychiatric diagnoses, and second, a validated cutoff score indicating RAD is missing for the RPQ. Hence, examination in larger and controlled clinical samples is needed to provide such a cutoff for evaluation of the RPQs performance and to further improve its usefulness in screening. The RPQ should, however, never be used as a diagnostic instrument on its own, because it does not establish all RAD‐criteria. Nevertheless, it may provide a contribution to diagnosis in school‐aged children within a broader assessment including an anamnesis on the presence of RAD‐symptoms before age five and of pathogenic care, and with attention to differential diagnosis. A final limitation of this study concerns the observational checklist used to examine the convergent validity of the RPQ. This checklist was developed to probe RAD‐behaviours, but research on its usefulness is limited. Therefore, we recommend to further validate the RPQ with a well‐studied observational tool for RAD to replicate its convergence with observations. Nevertheless, the use of multiple methods and informants to demonstrate the convergent validity of the RPQ is a particular strength of the current study. One of the study's implications is that it provides support for the validity of a multi‐informant instrument that enables researchers to study the developmental course and possible outcomes of RAD‐symptoms. This study also clears the way for further examination of the predictive value of the RPQ in a diagnostic protocol. Regarding clinical practice, we recommend to use both parent and teacher RPQ as supplementary information to decide whether interventions addressing relational problems might be needed and in what context.

This multi‐method study provides evidence for the psychometric qualities of the RPQ as a multi‐informant instrument for RAD‐symptoms in special education for children with emotional and behavioural disorders. In particular, previous research was extended by verifying the invariance of the two‐dimensional structure and factor loadings across informants, and by confirming the convergent and discriminant validity of the RPQ.

The authors have no competing interests.

This study was supported by a grant of the Fund for Scientific Research – Flanders (G.0555.09 N) to Karine Verschueren. The authors are grateful to the schools and participants in this study.

1The children (n = 11) from the participating school that acted as a pilot school for our measures were not included in the selection for the home part of the study.

• AACAP . (2005) Practice parameter for the assessment and treatment of children and adolescents with reactive attachment disorder of infancy and early childhood. Journal of the American Academy of Child and Adolescent Psychiatry, 44(11), 1206–1219. [PubMed] [Google Scholar]
• American Psychiatric Association . (1980) Diagnostic and Statistical Manual of Mental Disorders, 3rd edition, Washington, DC: American Psychiatric Association. [Google Scholar]
• American Psychiatric Association . (2000) Diagnostic and Statistical Manual of Mental Disorders, 4th revised edition, Washington, DC: American Psychiatric Association. [Google Scholar]
• Bentler P.M. (1992) On the fit of models to covariances and methodology to the bulletin. Psychological Bulletin, 112(3), 400–404. [PubMed] [Google Scholar]
• Chaffin M., Hanson R., Saunders B.E., Nichols T., Barnett D., Zeanah C.H., Berliner, L. , Egeland, B. , Newman, E. , Lyon, T. , LeTourneau, E. , Miller‐Perrin, C. (2006) Report of the APSAC task force on attachment therapy, reactive attachment disorder, and attachment problems. Child Maltreatment, 11(1), 76–89. DOI: 10.1177/1077559505283699 [PubMed] [CrossRef] [Google Scholar]
• Chen F.F. (2007) Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. [Google Scholar]
• Chen F.F. (2008) What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross‐cultural research. Journal of Personality and Social Psychology, 95(5), 1005–1018. DOI: 10.1037/a0013193 [PubMed] [CrossRef] [Google Scholar]
• Costello A.B., Osborne J.W. (2005) Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1–9. http://pareonline.net/getvn.asp?v=10&n=7 [Google Scholar]
• Fabrigar L.R., Wegener D.T., MacCallum R.C., Strahan E.J. (1999) Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. DOI: 10.1037/1082-989x.4.3.272 [CrossRef] [Google Scholar]
• Gleason M.M., Fox N.A., Drury S., Smyke A., Egger H.L., Nelson C.A., Gregas, M.C. , Zeanah, C.H. (2011) Validity of evidence‐derived criteria for reactive attachment disorder: indiscriminately social/disinhibited and emotionally withdrawn/inhibited types. Journal of the American Academy of Child and Adolescent Psychiatry, 50(3), 216–231. DOI: 10.1016/j.jaac.2010.12.012 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
• Goodman R. (1997) The Strengths and Difficulties Questionnaire: a research note. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 38(5), 581–586. [PubMed] [Google Scholar]
• Goodman R., Ford T., Simmons H., Gatward R., Meltzer H. (2003) Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample (reprinted from The British Journal of Psychiatry, 177, 534–539, 2000). International Review of Psychiatry, 15(1–2), 166–172. DOI: 10.1080/0954026021000046128 [PubMed] [CrossRef] [Google Scholar]
• Hu L.T., Bentler P.M. (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling–a Multidisciplinary Journal, 6(1), 1–55. DOI: 10.1080/10705519909540118 [CrossRef] [Google Scholar]
• Kline R.B. (2005) Principles and Practice of Structural Equation Modeling, 2nd edition, New York: The Guilford Press. [Google Scholar]
• Little T.D., Cunningham W.A., Shahar G., Widaman K.F. (2002) To parcel or not to parcel: exploring the question, weighing the merits. Structural Equation Modeling, 9(2), 151–173. DOI: 10.1207/s15328007sem0902_1 [CrossRef] [Google Scholar]
• McLaughlin A., Espie C., Minnis H. (2010) Development of a brief waiting room observation for behaviours typical of reactive attachment disorder. Child and Adolescent Mental Health, 15(2), 73–79. [Google Scholar]
• Minnis H., Green J., O'Connor T., Liew A., Glaser D., Taylor E., Follan, M. , Young, D. , Barnes, J. , Gillberg, C. , Pelosi, A. , Arthur, J. , Burston, A. , Connolly, B. , Sadiq, F.A. (2009) An exploratory study of the association between reactive attachment disorder and attachment narratives in early school‐age children. Journal of Child Psychology and Psychiatry, 50(8), 931–942. DOI: 10.1111/j.1469-7610.2009.02075.x [PubMed] [CrossRef] [Google Scholar]
• Minnis H., Rabe‐Hesketh S., Wolkind S. (2002) Development of a brief, clinically relevant, scale for measuring attachment disorders. International Journal of Methods in Psychiatric Research, 11(2), 90–98. DOI: 10.1002/mpr.127 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
• Minnis H., Reekie J., Young D., O'Connor T., Ronald A., Grayand A., Plomin, R. (2007) Genetic, environmental and gender influences on attachment disorder behaviours. British Journal of Psychiatry, 190(6), 490–495. DOI: 10.1192/bjp.bp.105.019745 [PubMed] [CrossRef] [Google Scholar]
• Muthén L.K., Muthén B.O. (1998–2010) Mplus User's Guide. Sixth Edition, Los Angeles, CA: Muthén & Muthén. [Google Scholar]
• Rutter M., Colvert E., Kreppner J., Beckett C., Castle J., Groothues C., Hawkins, A. , O'Connor, T.G. , Stevens, S.E. , Sonuga‐Barke, E.J.S. (2007) Early adolescent outcomes for institutionally‐deprived and non‐deprived adoptees. I: Disinhibited attachment. Journal of Child Psychology and Psychiatry, 48(1), 17–30. DOI: 10.1111/j.1469-7610.2006.01688.x [PubMed] [CrossRef] [Google Scholar]
• Smyke A.T., Dumitrescu A., Zeanah C.H. (2002) Attachment disturbances in young children. I: The continuum of caretaking casualty. Journal of the American Academy of Child and Adolescent Psychiatry, 41(8), 972–982. [PubMed] [Google Scholar]
• Van Leeuwen K., Meerschaert T., Bosmans G., De Medts L., Braet C. (2006) The Strengths and Difficulties Questionnaire in a community sample of young children in Flanders. European Journal of Psychological Assessment, 22(3), 189–197. [Google Scholar]
• Vandenberg R.J., Lance C.E. (2000) A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. [Google Scholar]
• Widaman K.F., Ferrer E., Conger R.D. (2010) Factorial invariance within longitudinal structural equation models: measuring the same construct across time. Child Development Perspectives, 4(1), 10–18. [PMC free article] [PubMed] [Google Scholar]
• World Health Organization . (1992) ICD‐10 Classifications of Mental and Behavioural Disorder: Clinical Descriptions and Diagnostic Guidelines, Geneva: World Health Organization. [Google Scholar]
• Zeanah C.H. (1996) Beyond insecurity: a reconceptualization of attachment disorders of infancy. Journal of Consulting & Clinical Psychology, 64(1), 42–52. [PubMed] [Google Scholar]
• Zeanah C.H., Boris N.W. (2000) Disturbances and disorders of attachment in early childhood In Handbook of Infant Mental Health, 2nd edition, Zeanah C.H. (ed.) pp. 353–368, New York: Guilford Press. [Google Scholar]
• Zeanah C.H., Gleason M.M. (2010) Reactive attachment disorder: a review for DSM‐V. http://www.dsm5.org/ProposedRevisions/Pages/proposedrevision.aspx?rid=120# [25 May 2011]
• Zilberstein K. (2006) Clarifying core characteristics of attachment disorders: a review of current research and theory. American Journal of Orthopsychiatry, 76(1), 55–64. [PubMed] [Google Scholar]