When Peer Review Fails: With Special Relevance to RAD

The idea of peer-reviewed articles in professional journals is, or at least was, a helpful way of keeping nonsense and excessive speculation out of respected print publications. Authors submit articles to journals, and editors send them on to professionals who are knowledgeable in relevant fields. Those people, peers of the article authors in that they have similar training, review the submitted material and convey their conclusions to the editors. These conclusions may be to reject the article entirely or to accept it just as it stands, but most often peer reviewers suggest acceptance with some major or minor revisions. Good reviewers can assess the background of a submitted article, can understand whether information cited from other work is correct, can evaluate research methods that were used, and can advise on improving organization and writing.

This is a lot of work and may or may not be done as well as we’d like. Not all peer reviews catch problems—and when big problems remain, journal editors may sometimes have to retract an article that contains major errors or that is just a repetition of something the author has published elsewhere.

In addition to the occasionally failures of reviews, nowadays we have the problem of the ever-expanding number of journals. Some of these are “predatory” journals whose real purpose is to collect fees from authors (conventional professional journals do not charge except sometimes for publication of  illustrations). Some are “open access” journals that charge fees and provide quick publication for articles that might not be accepted otherwise, or whose publication might be delayed longer than is comfortable for an author who is afraid of perishing without a publication. These journals refer to themselves as “peer reviewed”, and no doubt someone does “peer” at the submissions, but little effective reviewing happens.

When predatory or open access journals make mistakes about accepting articles, those mistakes are rarely corrected. Journals that take seriously the job of reviewing are likely to retract wrong articles or publish corrections—or, they very quickly publish additional articles critiquing the problem material. For example, the journal Child Development, which is about as serious and respectable as a publication can get, recently published a piece claiming that exposure to wireless technologies has been shown to affect child development (Sage, C., & Burgio,E. [2018]. Electromagnetic fields, pulsed radiofrequency radiation, and epigenetics: How wireless technologies may affect childhood development. Child Development, 89, 129-136). Interestingly, when this article was published on line, it did not include a conflict of interest statement, which was added for the print version, as one of the authors is the owner of an environmental consulting firm and a consultant on relevant policy and decisions.

When the Sage and Burgio article appeared on line, it was seen by others than the original reviewers, and these people were able to recognize that the authors had “cherry-picked” and misrepresented material. Two authors submitted a commentary that the journal published in the same print edition as the Sage and Burgio article—the commentary stating that “these claims are devoid of merit, and… should not have been given a scientific veneer of legitimacy” (Grimes, D., & Bishop, D. [2018]. Distinguishing polemic from commentary in science: Some guidelines illustrated with the case of Sage and Burgio [2017].) Grimes and Bishop stated that it was their view “that this piece has potential to cause serious harm, and should never have been published.” They followed this statement with a suggested series of questions to be used by readers and reviewers in ascertaining whether an article deserves publication or should be regarded as credible. Among these questions, all of which deserve real attention, were four of particular interest: 1. Is there a plausible mechanism for a claimed effect? 2. Does evidence come from [genuinely] peer-reviewed sources? 3. Are all relevant studies considered? 4. Are results of specific studies misrepresented?  Applying their list of questions, Grimes and Bishop were able to show that the Sage and Burgio paper was completely unacceptable. However, because the paper was not actually retracted, Sage and Burgio may (unfortunately) continue to argue that their work was validated by its acceptance by a prestigious peer-reviewed journal.

In the title of this post I said we would get to Reactive Attachment Disorder, and I am about to do that, with a description of an article that probably should not have been accepted by a journal and of another article that critiqued the first one. The problematic article was an effort to argue that difficult and oppositional behavior, conduct disorder, and callous-unemotional traits are closely associated with RAD (Mayes, S., Calhoun, S., Waschenbusch, D., Breaux, R., & Baweja, R. [20i7]. Reactive attachment/disinhibited social engagement disorders: Callous-unemotional traits and comorbid disorders. Research in Developmental Disabilities, 63, 28-37).

A little history is needed to understand the background of the Mayes et al article. Definitions of RAD have changed over the years, beginning in 1980 with the idea that the disorder involved infant failure to thrive and gradually changing to comprise unusual behaviors of young children toward adult caregivers. Until recently, RAD was considered to exist in two forms, an inhibited form in which young children did not seek care or comfort from adults, and a disinhibited form in which young children were as likely to approach strange adults as to approach familiar people for care and social interaction. The 2013 version of the Diagnostic and Statistical Manual of Mental Disorders (DSM) separated the inhibited version, which kept the term RAD, from the disinhibited version, which now became Disinhibited Social Engagement Disorder (DSED).

Meanwhile, however, there had been as far back as the 1990s an effort to create an alternative view of a disorder attributed to early separations, abuse, and neglect. This putative disorder was often referred to as Reactive Attachment Disorder, but instead of involving unusual social behavior toward caregivers, it was said to include disturbing aggressive behaviors like cruelty, theft, lying, and “fascination with blood and gore”. Some alternative practitioners claimed that they had discovered a new form of the disorder, not Reactive Attachment Disorder at all, but something they called simply Attachment Disorder. This notional diagnosis was promulgated by practitioners of Holding Therrapy/Attachment Therapy, by a number of adoption groups, and by support groups for adoptive parents. The idea of violent Attachment Disorder behavior was taught by made-for-TV movies like “Child of Rage” and by various related websites. For a number of years, newspaper articles about child abuse have often proposed that abused children showed disturbing behaviors said to be characteristic of RAD or AD and in some sense had forced adults to mistreat them. These beliefs continue to be promulgated, as can be seen at https://blogs.psychcentral.com/childhood-behavioral/2018/05/why-parents-of-r-a-d-children-always-look-like-aholes/, a blog where commenters claim that adoptive parents of children  “diagnosed with RAD” and behaving aggressively must treat those children as coldly and mechanically as possible. In reply to criticism of this position, comments are focused on the idea that anyone who does not think RAD includes callous-unemotional or aggressive behavior has simply never lived with such a child—a rather inconsequential response, as no one has said that their children do not display the claimed behaviors, just that these are not evidence of RAD.

Conventional, evidence-based work on childhood mental illness has rejected the idea that externalizing behavior, conduct disorders, or callous-unemotional characteristics such as lack of empathy are related to attachment experiences or should be treated by methods that stress attachment. The article by Mayes et al argues against that conventional view and presents a small amount of data that are used to conclude that children diagnosed with RAD are also likely to be violent, cruel, callous, and unemotional. The study examined 20 children, ages 4-17, who had been diagnosed with RAD or DSED or both, and compared them with much larger groups of children diagnosed as autistic or as having ADHD. The authors suggested that the DSM discussion of RAD should include references to conduct disorders and callous-unemotional traits as associated with RAD.

A trenchant response by a competent critic of misused attachment concepts followed the Mayes et al paper (Allen, B. [2018]. Misperceptions of reactive attachment disorders persist: Poor methods and unsupported conclusions. Research in Developmental Disabilities, 77, 24-29). In keeping with our concern with failed peer review, Allen offered guidelines for reviewing articles that stress attachment issues. He used the Mayes paper as an example of a publication about which reviewers had not asked the right questions.

Here are the questions Allen suggested as review guidelines:

1.      How well is attachment research reviewed? This is an area where, Allen suggests, Mayes et al seem to have “cherry-picked”, stressing early writings of John Bowlby and omitting some of the contradictory conclusions of his later work.

I would like to add that Mayes et al used a 2003 paper by Hall and Geher to support the claim of aggressive externalizing behavior in children with RAD; the Hall and Geher paper in turn was dependent on what may be the ur-article on such behavior in children diagnosed with RAD, an obscure publication, probably not peer-reviewed in any real sense, that put forth the idea that RAD children can vomit or defecate at will and do so simply to antagonize their caregivers (Reber, K. [1996]. Children at risk for attachment disorder: Assessment, diagnosis, and treatment. Progress: Family Systems Research and Therapy, 5, 83-98 ; this publication of a freestanding California institute used to be available on the Internet but I have not been able to find it recently). (Reber’s claims may have been behind the Internet arguments that Candace Newmaker died intentionally in order to cause trouble for the therapists who asphyxiated her.) The influence of Reber on Hall and Geher and subsequently on Mayes et al demonstrates that reviewers need to be aware not only of the background publications being cited in an article, but also of the nature of the material cited in those publications.

2.      Are attachment constructs clearly defined? Unfortunately, attachment concepts seem to have achieved the position of obscenity, in that most people can’t define them but believe they know them when they see them. Nonetheless, as Allen suggests, it is crucial to know whether authors are concerned with attachment behavior, attachment representations, or attachment theory. The choice of constructs is directly related to understanding of how a diagnosis was made—a point left obscure in the Mayes et al paper.

3.      Are assessment measures and methods clearly specified? All too often, discussions of RAD have involved “validation” of assessment methods against the intuition or “practice wisdom” of one or two clinicians. Elizabeth Randolph’s Randolph Attachment Disorder Questionnaire (RADQ) is described by its author as substantiated in part by her ability to diagnose an attachment disorder when a child cannot crawl backward on command.

Unspecified and vague assessment measures in the Mayes et al paper are especially important because of the age range of children examined—from 4 to 17 years of age, with a mean age about 10. Although toddlers can be expected to reveal the quality of their attachment to caregivers by their behavior with strangers or when separated from familiar people, this can hardly be expected with school-age children or teenagers, whose cognitive and emotional development and daily experiences allow them to handle separation or contact with strangers as normal events that affect their behavior very little. Assessments of 4-year-olds, 10-year-olds, and 17-year-olds can hardly take the same forms, yet Mayes et al do not state differences in evaluation of children of different ages, and ignore the many discussions in the literature about the maximum age at which RAD could be a meaningful diagnosis.

4.      Do the authors use tautological reasoning? When clinicians believe that RAD involves aggressive or callous-unemotional traits, they may use such traits as a reason to diagnose RAD, even though RAD by definition does not include these traits. Allen queries whether this illogical approach is responsible for the conclusions drawn by Mayes et al.

Historically, proponents of the idea that externalizing problems were part of attachment disorders have also argued  that children with problematic attachment histories (separation, abuse, neglect) will of necessity have RAD and [therefore] externalizing problems, such a history being in the past a factor in the RAD diagnosis; this claim was even applied when children showed no behavioral disturbance, with convinced clinicians telling parents that the child’s normal behavior was simply an indication of the degree of disturbance, intended to lull the parent’s concerns until the child found a way to attack effectively. 

5.      Do the authors make remarkable conclusions, which may not be supported by their data? Allen suggests that mistaken conclusions are drawn by Mayes et al, for example that it is incorrect to claim that RAD is associated with internalizing problems and DSED with externalizing problems (e.g., cruelty, lack of empathy), even though this is what other  unemotional traits, the Mayes et al conclusions fail to recognize the limitations of their study  as stated above and discussed in the Allen paper.

It is clear that if reviewers had used the questions supplied by Allen in their reviews of the Mayes et al paper, the paper would never have appeared except in a considerably revised form.

Revisions would also have been required for publication if the guidelines proposed by Grimes and Bishop (2018), some of which were discussed earlier in this post, had been applied. The plausible mechanism demanded by Grimes and Bishop is certainly missing in the Mayes et al article—by what mechanism would a failure of attachment lead to cruel and callous-unemotional behavior posited by Mayes, or the disobedience, ingratitude, and lack of remorse claimed by Randolph and others? Attachment theory and the idea of attachment representations, with their stress on cognitive and social development, do not provide any explanation of the proposed mechanism.

A question about causal claims put by Grimes and Bishop would also require Mayes et al to revise their article and supply a much elaborated discussion of limitations. This nonrandomized study design does not allow for the stated conclusion that externalizing behaviors are caused by problematic attachment histories.

There is little question that guidance is needed by peer reviewers dealing with submitted papers involving the complexities of attachment concepts and the conflation of established views of attachment with alternative attachment theories. Given the approaches proposed by Allen and by Grimes and Bishop, we can hope, along with Allen, that “through this process of enhanced scrutiny of papers examining RAD/DSED that misperceptions and subsequent substandard clinical services can be significantly improved.”