change the world badge

change the world badge

feedspot

Child Psychology Blogs

Concerned About Unconventional Mental Health Interventions?

Concerned About Unconventional Mental Health Interventions?
Alternative Psychotherapies: Evaluating Unconventional Mental Health Treatments

Thursday, September 9, 2021

Not on the Levels: Harman Says She Evaluated Turning Points

 

Jennifer Harman, a proponent of the parental alienation belief system, has published an article that reports her attempt to evaluate Linda Gottlieb’s intensive parental alienation treatment, Turning Points for Families (TPFF). For those readers who are interested, the citation is

Harman, J., Saunders, L., & Afifi, T. (2021). Evaluation of the Turning Points for Families (TPFF) program for severely alienated children. Journal of Family Therapy. DOI: 10.1111/1467-6427.

In this post, I will discuss a number of issues of research design and analysis that argue against any conclusion that TPFF is a demonstrably effective treatment for parental alienation (PA). It’s notable that this conclusion, though not exactly reached, was possibly assumed in Harman’s plan, as she notes early in the paper that Gottlieb has provided “evidence-based treatment since 1994. (It is not clear what this means, as Gottlieb has stated that she did not formulate TPFF until 2015.)

My title for this post, “Not on the Levels”, is not a calumny against Harman’s integrity or her appropriate intentions in carrying out this piece of outcome research. I think it is most parsimonious to assume that Harman intended to do this work correctly but made some errors that closely resemble those of other PA proponents like William Bernet and Amy Baker. These errors can be described in terms of “levels” of various kinds – particularly levels of evidence and levels of measurement.

I.                    Levels of evidence

Harman et al. state in their abstract and essential points that they have shown that TPFF is an effective treatment. They have not concerned themselves, or reminded their readers, about the idea that not all outcome studies are equally meaningful. How much credence can be given to an outcome study depends on the way data were collected and the comparisons that were made (if any). (More about this concept is available at www.effectivechildtherapy.org and at www.cebc4cw.org, websites that evaluate outcome research on therapies for children and adolescents.) Reports of the effectiveness of a treatment can be said to be evidence-based when the outcome research has involved randomized controlled trials or careful clinical controlled trials; in both these cases, the word “control” signifies that treated individuals’ outcomes were compared to outcomes for similar individuals who received a different treatment.

Harman’s study did not involve any comparison group. It was essentially a study of events in a single group over a period of time, a level of evidence sometimes described (a bit deceptively) as “promising”. (“Of interest” might be a better description for treatments with these uncontrolled outcome studies.) Just as Harman could not correctly claim that Gottlieb has been doing evidence-based treatment for a long time, she and her colleagues cannot correctly state that this attempt at evaluation has shown the effectiveness of TPFF. In order to reach that conclusion, they would have needed to do similar observations on a matched group of children and parents who did not receive TPFF, and to show differences between the treated and untreated groups.

II.                 Levels of measurement

In 1946, the Harvard psychologist S. Smith Stevens (known to students at the time as “Shitty Smitty” for his rather unlovable personality) published a seminal paper in the journal Science, on the idea of levels of measurement and the kinds of statistical analysis that could be used for each one. The idea here is that we can and do measure events in different ways. If I look at a painting and say that it contains greens, blues, and pinks, I am measuring in a very simple way. If I say that this painting is greener than that one, I am measuring in a somewhat more complex way. And if I say I have counted 92 green one-inch squares in this painting and 95 green one-inch squares in that one, I am at the most complex level of measurement of all. If I can count how many green squares there are in different paintings, I can say, for example, that one painting has twice as many green squares as another one, or I can state the mean number of green squares in a group of paintings. I can’t do that kind of arithmetic if I am simply saying that one is greener than another, and if I can’t do that kind of arithmetic, I can’t use the various important parametric statistics like means, standard deviations, Student’s t, or ANOVA. Instead, I have to use some less refined statistics from a group called nonparametric statistics.

Harman’s statistical analysis falls by the wayside because she has ignored levels of measurement, just as Bernet and Baker have done elsewhere. She has performed what someone has called “quantitative alchemy” by trying to force measurements of what is “greener” (more desirable) into statistical procedures that are meant for measures that can be added, subtracted, multiplied and divided. This does not lead to meaningful conclusions.

Harman, Bernet, Baker, and other PA proponents have an unfortunate affection for Likert scales. These are the ubiquitous ways of measuring that ask people to rate an event or statement as “don’t agree at all”, “agree somewhat”, “totally agree”, or some version of that procedure. That is what Harman did in her study: observers of videorecordings rated events they saw on a Likert scale. This scale does not permit any of the arithmetical operations needed for parametric statistics, because it is not reasonable to claim (for instance) that a rating of 5 is 5 times a rating of one. Nevertheless, Harman and her colleagues (including the statistician who sees to have been given authorship for that reason) have gone merrily along and presented means and other parametric statistics as if they are appropriate.

Just for the sake of argument, however, let’s for a moment accept the use of the parametric statistics and look at how the rated behaviors changed over time. I would have assumed that the goal of TPFF was to improve the parent-child relationship quality. Harman et al say “we did not anticipate large changes in relationship quality in a short, 4-day intervention”, and they were apparently right not to anticipate such, as the t for this measure was -0.66, p=0.51. (I would ask, though, if you don’t anticipate a change in 4 days, why design a 4-day intervention?)

As a final point on measurement in this study, it was unclear to me why Harman did not just use a count of events, which could have been analyzed parametrically, and did not present any information about whether the observed behaviors were performed by the child or by the parent. Are any reported changes to be read as changes in the child’s behavior, the parent’s behavior, or both? Perhaps the moral of this story is that there should be a license required to use the SPSS package.

III.              Levels of severity and their implications for research design

Harman et al title their report as involving “severely alienated children”. Yet here is what they say about the children: “The majority of the children who [had] refused to have ay relationship with the alienated parent traveled significant distances across the country and spent several hours or days with them prior to the intervention…a large number of alienated children had transferred custody to the alienated parent for between a week and as long as 44 days after the order had been entered.” In other words, although these children had been said (by someone) to be severely resistant to contact with one parent, they now were not resisting at all even before the TPFF program began.

If it is true that the children were not resistant when they arrived at TPFF,  what confounding variables must we consider as possibly causing this change from the previously reported resistance? Is the court order alone the cause of any changes suggested as resulting from TPFF? Is TPFF to be praised as not having caused the children to become resistant again? Why is it appropriate to use an expensive and disruptive program if the children are already doing what was wanted, and why, in particular, should the preferred parent be ordered to participate in treatment with a TPFF-approved therapist? This would appear to be a good business model but a weak treatment model.

Beyond the issue of levels of severity, of course, there is the much larger issue of how PA is identified to begin with. The only consistency in the various treatment program outcome studies is that a court has decided there is PA present. On what basis that has been done remains unclear. Harman, unfortunately, states that the Parental Acceptance-Rejection Questionnaire, discussed by Bernet and colleagues as a way to discuss “splitting”, is a PA assessment tool! I hope her PA colleagues will talk to her about this point.

IV.              Levels of harm

The possibility that PA treatments can be harmful to children and adolescents, which I as well as other authors have considered, is of much concern to PA proponents. Harman et al, like other PA advocates, remark that my published discussion of this point is based on a small number of cases in which the children claiming harms were still alienated. There has been a gradual increase in the number of anecdotal claims of this kind, with at least one PTSD diagnosis, but I would be the first to acknowledge that they have never been investigated systematically, nor have the reports brought PA proponents to look for adverse effects of their programs. There are a number of lawsuits in the making as some affected children approach their 18th birthdays, and those will bring out some interesting information.

Anyone treating children and adolescents is bound to have concerns about very serious harms, self-injury, depression, suicidality or completed suicide, or running away and being trafficked or otherwise injured. Harman et al stated that no harms were caused by TPFF, as no children ran away or self-harmed during the intervention or some subsequent period. These measures are important but minimal. It may be that following the expected lawsuits we will see some evidence of harms that proponents of PA interventions will have to consider in the future.

 

Conclusion

On a number of different levels, the outcome research on TPFF reported by Harman et al fails to support the conclusions that TPFF is a safe and effective treatment for children in cases of parental alienation. In addition, the serious flaws of design and analysis in this work suggest that the critiques of Joan Meier’s work, recently presented by PA advocates, may be open to serious question.  

No comments:

Post a Comment