Concerned About Unconventional Mental Health Interventions?

Concerned About Unconventional Mental Health Interventions?
Alternative Psychotherapies: Evaluating Unconventional Mental Health Treatments

Saturday, January 19, 2013

False Positives, False Negatives, and Standardized Testing for Reactive Attachment Disorder


When tests are developed for medical or psychological disorders, the idea is usually this: There is a disorder that can be identified. However, it may be difficult and complicated to identify it in the usual ways, or it may be desirable to identify it when only a few vague symptoms have appeared, in order to treat it early and keep it from getting worse. A medical or psychological test is a way of measuring a sample of behavior or biological functioning that will help predict whether a disorder is developing, or that can be an effective substitute for difficult, intrusive , and time-consuming examination of other kinds. Effective tests are very useful in ruling out problems that the symptoms might suggest but that are not really present, and thus allowing appropriate treatment to be chosen without waste of time.

But the tricky part about tests is that even the good ones are not always right. They may result in false positives and indicate the presence of a problem when there really is none. They may also result in false negatives and show that there is no disorder—but later events demonstrate that the disorder was actually there. Even the best tests show some false positives and some false negatives. The practical goal is not to get rid of all of these, but to be able to state how often they occur, and to interpret results in the light of that information.

Test development is a complicated and tedious matter in which small errors may ruin the value of extensive efforts. For example, an imprecise or erroneous definition of a disorder can result in an ineffective test. Test developers must work hard to exclude sources of bias, especially if diagnosis of the disorder has more than a small reliance on subjective opinions. Persons who perform or even know the diagnosis given to a participant must not also be the ones who perform or score the test, for fear that their beliefs will inadvertently influence the ways they administer or interpret the test. If a diagnosis depends on the examiner’s opinion rather than an objective measurement, it’s important to have several examiners make independent assessments, and to ascertain the extent to which they agree. If a test has not been developed following these and other guidelines, it cannot be trusted. Tests should be regarded with suspicion if they have not been published in peer-reviewed journals, whose expert reviewers will have done some of the work of assessing the test’s credentials. Transparency of reporting is the key to test selection; tests that are published privately by their developers, and whose background is not available to the reader, should be approached warily.

How does all this relate to the issue of testing for Reactive Attachment Disorder? There are presently no thoroughly-validated tests for this disorder--  and one reason is that the condition remains only incompletely defined, and has somewhat different descriptions in DSM (the standard U.S. listing of mental disorders) and ICD (the European manual).

Nonetheless, some American practitioners, especially those who advocate the use of “holding therapy” for children diagnosed with Reactive Attachment Disorder, choose as a diagnostic test the Randolph Attachment Disorder Questionnaire (RADQ), an instrument self-published by Elizabeth Randolph. The validating information claimed for the RADQ by Randolph has never been published in any peer-reviewed journal, and the one related article in a peer-reviewed journal (by Cappelletty et al) concluded that scores on the RADQ did not correlate with any validated test for childhood emotional disturbance. Beyond that, however, it is notable that Randolph herself stated plainly in her self-published work that the RADQ was not intended as a test for Reactive Attachment Disorder, but instead was an assessment of a different, suppositious disorder never described in any peer-reviewed publication. Whatever Randolph intended to test, in any case, she failed to guard against bias in her results by herself doing both the job of subjective diagnosis and that of performing and scoring the test. Although her publication includes a report of an analysis of variance on the test results, it does not state how many false positives or false negatives occurred. This raises the question whether there were no such false results, simply because both the original diagnosis and the test result were in each case formulated by Randolph, who agreed with herself strongly in her assessment of each child; this of course is a far cry from having test scores that are validated by their agreement with an independent diagnosis.

The RADQ should be excluded as a possible diagnostic tool for Reactive Attachment Disorder  first on the showing of its own developer, who did not intend it to do that job, and second on the basis of its complete lack of conformity to normal guidelines for test development. That neither false positives nor false negatives have been reported is a statement not of the effectiveness of the test, but of a failure to consider a basic testing issue.

What then? Are there any standardized tests for Reactive Attachment Disorder? Helen Minnis, a Scottish psychiatrist, has been working for a number of years to try to develop such an assessment, but although she has created evaluative methods, she has no standardized brief test. In a 2009 paper with a group of colleagues (An exploratory study of the association between reactive attachment disorder and attachment narratives in early school-age children. Journal of Child Psychology and Psychiatry, 50(8), 931-942), Minnis described the complications and difficulties of this work, beginning with the lack of clarity in descriptions of the disorder: “Although the concept of RAD is encapsulated in psychiatric classification systems… the research base is scant, particularly in relation to school-age children… In this paper, we use the term RAD as in DSM to cover both the ‘inhibited’ and the ‘disinhibited’ phenotypes… The DSM and ICD systems both define RAD as being associated with early maltreatment and characterized by disinhibited behavior (indiscriminate sociability) or inhibited (withdrawn, hypervigilant) behaviors.” Minnis goes on to point out that there is not much consensus about the effect of changes with age on RAD, and that one system includes attention-getting and aggression toward self and others among the symptoms. Minnis pointed out that research has indicated that children may show symptoms of RAD and also be evaluated as securely attached to caregivers. (Does this suggest that in fact Reactive Attachment Disorder has nothing to do with attachment? See below—J.M.)

Children in the Minnis study were referred because of symptoms noticed by social workers and mental health teams, but not because of evidence of pathogenic care. Thirty-three children  diagnosed with RAD on the basis of interviews and observations were compared to 37 children matched on age and sex but not diagnosed with RAD.  Working with an extensive protocol rather than a brief test like the RADQ, Minnis looked for shared characteristics of children diagnosed with RAD. Minnis and her colleagues concluded that their findings “reinforce the conclusions from other literature that RAD is a phenomenon different in kind from attachment specific behaviors…. RAD can perhaps be seen as one of the pervasive disorders of social impairment…”.  

Minnis’s work suggests that a brief test diagnosing RAD is not going to be possible in the near future. Beyond that, Minnis and her colleagues point out the possibility that RAD is not about attachment in any ordinary sense of the term. I would note that this finding implies that efforts to destroy or create attachments are essentially irrelevant to treatment of  RAD--  in contradiction to the belief systems of persons who currently use the RADQ.


7 comments:

  1. Been out of touch for a while because all hell broke loose in our family. I don't care what diagnosis anyone applies to my daughter's behaviors, but if she's beating us up and getting arrested (multiple times) for domestic violence, I need some practical, research-backed help, and quickly: for the violence. Period. The closest I've found locally is a DBT program for adolescents/families, which we will do once she's been sprung from lockup/psych hospital/acute residential treatment (ART)/etc. Further afield, I've been prowling around online and discovered the StepUp Curriculum for teens who violent towards their parents. Don't know anything about it except that it draws from programs aimed at adult batterers---and there's no mention of attachment, as far as I can tell, and lots of focus on skills building, which seems appropriate developmentally for teens. Any thoughts?

    http://www.kingcounty.gov/courts/step-up/Curriculum.aspx

    ReplyDelete
  2. Can we look to research on the use of DBT in adults who commit domestic/familial violence and extrapolate to teens in creating treatment plans, or is that just developmentally inappropriate?

    http://www.borderlinepersonalitydisorder.com/documents/DBTforDomesticViolence2000.pdf

    ReplyDelete
  3. I'm so sorry to hear that your family has been having this serious problem.I know you must be especially concerned about what it all means for your daughter's future.

    StepUp appears to be harmless, anyway, although there doesn't seem to be any systematic evidence that it's effective. But under the circumstances you may not be able to find much that's suitable and evidence-based too. I think
    extrapolating from DBT is probably the best thing you can do right now for her specific treatment. Doing the StepUp in addition has the advantage of guiding what you do and providing you with some coaching which you can use or ignore as seems best to you.

    What a difficult time for all of you! Please accept my sympathy and good wishes-- Jean

    ReplyDelete
  4. Thanks, Jean. I appreciate your professional expertise as well as your kind thoughts.--Marianne

    ReplyDelete
    Replies
    1. You're welcome,if there was the slightest help there--

      Delete
  5. Specifically, now, and over time, as I've been reading your blog and we've been conversing, you've been enormously helpful. Yesterday, during the treatment plan meeting at the ART (acute residential treatment) center, when the experienced DCF social worker, who is usually quite on target, started talking about all the "attachment" work that my daughter needs to overcome her physical aggression, I refocused us all, and and asked about what ART has to offer in the way of psycho-education, therapeutic approaches, and medication support for physical aggression. Out of the corner of my eye, I could see the DCF worker pouting, and later she rephrased what she wanted to see happen as "family work, family work, FAMILY WORK," but the implication was clearly attachment work. My daughter has once again received a diagnosis of RAD, by yet another psychiatrist, who knows nothing and cares less than that about her early history. I ignore that too. Because what we *all* know about her early history (and can find details in police blotters about arrests, convictions, and sentencing) is the tremendous amount of domestic violence my daughter was a "witness" to (though, I believe clinicians are moving away from the benign "witness" to talking about "participant," since that conveys the impact of domestic violence on a child, whether the child is physically injured or not). Can we deal with the facts, I want to scream? But I'm diplomatic and judicious, and redirect their unverifiable interpretations to what is plainly obvious: kids learn what they see modeled for them, over and over and over again. And unlearning is hard, focused work. Reducing anxiety and depression and the impulsivity of ADHD can be markedly helped by the correct medications. And researchers who have worked with programs aimed at batterers and battering behavior know a little bit about that unlearning strategy. Let's follow the smoke and fire like a scientist, and not like a fantasy writer. Argh!!!!!!!!!!!!!!!!!!

    ReplyDelete
    Replies
    1. All I can say, is thank goodness your daughter has you to try to bring an element of rationality into this. The problem with the family/attachment work is the assumption that one size fits all and a single-factor theory explains everything. This is a fad among child and family workers just like the fads that blow through the educational system from time to time,promising that an "innovation" will fix everything--with t-shirts for everyone, no less!

      Maybe it's true that if we could "fix" some simple internal flaw through depth work, all the behavioral problems would be ironed out. So far,however, nobody seems to know how to identify the flaw or mend it-- so why not deal with the symptoms and alleviate them in ways that are known to be effective?

      Obedience and affection shouldn't be equated with attachment, and even if they could, re-enactment of scenes from early life could not create attachment... but you know all about that.

      Do you remember Nicole Hollander's "Sylvia", with her fast psychotherapy? Her client complains, "But you're just treating the symptoms,not the basic problem" and Sylvia replies, "Words to live by!"

      Delete