False Positives, False Negatives, and Standardized Testing for Reactive Attachment Disorder

When tests are developed for medical or psychological disorders, the idea is usually this: There is a disorder that can be identified. However, it may be difficult and complicated to identify it in the usual ways, or it may be desirable to identify it when only a few vague symptoms have appeared, in order to treat it early and keep it from getting worse. A medical or psychological test is a way of measuring a sample of behavior or biological functioning that will help predict whether a disorder is developing, or that can be an effective substitute for difficult, intrusive , and time-consuming examination of other kinds. Effective tests are very useful in ruling out problems that the symptoms might suggest but that are not really present, and thus allowing appropriate treatment to be chosen without waste of time.

But the tricky part about tests is that even the good ones are not always right. They may result in false positives and indicate the presence of a problem when there really is none. They may also result in false negatives and show that there is no disorder—but later events demonstrate that the disorder was actually there. Even the best tests show some false positives and some false negatives. The practical goal is not to get rid of all of these, but to be able to state how often they occur, and to interpret results in the light of that information.

Test development is a complicated and tedious matter in which small errors may ruin the value of extensive efforts. For example, an imprecise or erroneous definition of a disorder can result in an ineffective test. Test developers must work hard to exclude sources of bias, especially if diagnosis of the disorder has more than a small reliance on subjective opinions. Persons who perform or even know the diagnosis given to a participant must not also be the ones who perform or score the test, for fear that their beliefs will inadvertently influence the ways they administer or interpret the test. If a diagnosis depends on the examiner’s opinion rather than an objective measurement, it’s important to have several examiners make independent assessments, and to ascertain the extent to which they agree. If a test has not been developed following these and other guidelines, it cannot be trusted. Tests should be regarded with suspicion if they have not been published in peer-reviewed journals, whose expert reviewers will have done some of the work of assessing the test’s credentials. Transparency of reporting is the key to test selection; tests that are published privately by their developers, and whose background is not available to the reader, should be approached warily.

How does all this relate to the issue of testing for Reactive Attachment Disorder? There are presently no thoroughly-validated tests for this disorder--  and one reason is that the condition remains only incompletely defined, and has somewhat different descriptions in DSM (the standard U.S. listing of mental disorders) and ICD (the European manual).

Nonetheless, some American practitioners, especially those who advocate the use of “holding therapy” for children diagnosed with Reactive Attachment Disorder, choose as a diagnostic test the Randolph Attachment Disorder Questionnaire (RADQ), an instrument self-published by Elizabeth Randolph. The validating information claimed for the RADQ by Randolph has never been published in any peer-reviewed journal, and the one related article in a peer-reviewed journal (by Cappelletty et al) concluded that scores on the RADQ did not correlate with any validated test for childhood emotional disturbance. Beyond that, however, it is notable that Randolph herself stated plainly in her self-published work that the RADQ was not intended as a test for Reactive Attachment Disorder, but instead was an assessment of a different, suppositious disorder never described in any peer-reviewed publication. Whatever Randolph intended to test, in any case, she failed to guard against bias in her results by herself doing both the job of subjective diagnosis and that of performing and scoring the test. Although her publication includes a report of an analysis of variance on the test results, it does not state how many false positives or false negatives occurred. This raises the question whether there were no such false results, simply because both the original diagnosis and the test result were in each case formulated by Randolph, who agreed with herself strongly in her assessment of each child; this of course is a far cry from having test scores that are validated by their agreement with an independent diagnosis.

The RADQ should be excluded as a possible diagnostic tool for Reactive Attachment Disorder  first on the showing of its own developer, who did not intend it to do that job, and second on the basis of its complete lack of conformity to normal guidelines for test development. That neither false positives nor false negatives have been reported is a statement not of the effectiveness of the test, but of a failure to consider a basic testing issue.

What then? Are there any standardized tests for Reactive Attachment Disorder? Helen Minnis, a Scottish psychiatrist, has been working for a number of years to try to develop such an assessment, but although she has created evaluative methods, she has no standardized brief test. In a 2009 paper with a group of colleagues (An exploratory study of the association between reactive attachment disorder and attachment narratives in early school-age children. Journal of Child Psychology and Psychiatry, 50(8), 931-942), Minnis described the complications and difficulties of this work, beginning with the lack of clarity in descriptions of the disorder: “Although the concept of RAD is encapsulated in psychiatric classification systems… the research base is scant, particularly in relation to school-age children… In this paper, we use the term RAD as in DSM to cover both the ‘inhibited’ and the ‘disinhibited’ phenotypes… The DSM and ICD systems both define RAD as being associated with early maltreatment and characterized by disinhibited behavior (indiscriminate sociability) or inhibited (withdrawn, hypervigilant) behaviors.” Minnis goes on to point out that there is not much consensus about the effect of changes with age on RAD, and that one system includes attention-getting and aggression toward self and others among the symptoms. Minnis pointed out that research has indicated that children may show symptoms of RAD and also be evaluated as securely attached to caregivers. (Does this suggest that in fact Reactive Attachment Disorder has nothing to do with attachment? See below—J.M.)

Children in the Minnis study were referred because of symptoms noticed by social workers and mental health teams, but not because of evidence of pathogenic care. Thirty-three children  diagnosed with RAD on the basis of interviews and observations were compared to 37 children matched on age and sex but not diagnosed with RAD.  Working with an extensive protocol rather than a brief test like the RADQ, Minnis looked for shared characteristics of children diagnosed with RAD. Minnis and her colleagues concluded that their findings “reinforce the conclusions from other literature that RAD is a phenomenon different in kind from attachment specific behaviors…. RAD can perhaps be seen as one of the pervasive disorders of social impairment…”.  

Minnis’s work suggests that a brief test diagnosing RAD is not going to be possible in the near future. Beyond that, Minnis and her colleagues point out the possibility that RAD is not about attachment in any ordinary sense of the term. I would note that this finding implies that efforts to destroy or create attachments are essentially irrelevant to treatment of  RAD--  in contradiction to the belief systems of persons who currently use the RADQ.