Jennifer Harman, a proponent of the parental
alienation belief system, has published an article that reports her attempt to
evaluate Linda Gottlieb’s intensive parental alienation treatment, Turning
Points for Families (TPFF). For those readers who are interested, the citation
is
Harman, J., Saunders, L., & Afifi, T. (2021).
Evaluation of the Turning Points for Families (TPFF) program for severely
alienated children. Journal of Family Therapy. DOI: 10.1111/1467-6427.
In this post, I will discuss a number of issues of
research design and analysis that argue against any conclusion that TPFF is a
demonstrably effective treatment for parental alienation (PA). It’s notable
that this conclusion, though not exactly reached, was possibly assumed in Harman’s
plan, as she notes early in the paper that Gottlieb has provided
“evidence-based treatment since 1994. (It is not clear what this means, as
Gottlieb has stated that she did not formulate TPFF until 2015.)
My title for this post, “Not on the Levels”, is not a
calumny against Harman’s integrity or her appropriate intentions in carrying
out this piece of outcome research. I think it is most parsimonious to assume
that Harman intended to do this work correctly but made some errors that
closely resemble those of other PA proponents like William Bernet and Amy
Baker. These errors can be described in terms of “levels” of various kinds –
particularly levels of evidence and levels of measurement.
I.
Levels of evidence
Harman et al. state in
their abstract and essential points that they have shown that TPFF is an
effective treatment. They have not concerned themselves, or reminded their
readers, about the idea that not all outcome studies are equally meaningful.
How much credence can be given to an outcome study depends on the way data were
collected and the comparisons that were made (if any). (More about this concept
is available at www.effectivechildtherapy.org
and at www.cebc4cw.org,
websites that evaluate outcome research on therapies for children and
adolescents.) Reports of the effectiveness of a treatment can be said to be
evidence-based when the outcome research has involved randomized controlled
trials or careful clinical controlled trials; in both these cases, the word
“control” signifies that treated individuals’ outcomes were compared to
outcomes for similar individuals who received a different treatment.
Harman’s study did not
involve any comparison group. It was essentially a study of events in a single
group over a period of time, a level of evidence sometimes described (a bit
deceptively) as “promising”. (“Of interest” might be a better description for
treatments with these uncontrolled outcome studies.) Just as Harman could not
correctly claim that Gottlieb has been doing evidence-based treatment for a
long time, she and her colleagues cannot correctly state that this attempt at
evaluation has shown the effectiveness of TPFF. In order to reach that conclusion,
they would have needed to do similar observations on a matched group of children
and parents who did not receive TPFF, and to show differences between the
treated and untreated groups.
II.
Levels of measurement
In 1946, the Harvard
psychologist S. Smith Stevens (known to students at the time as “Shitty Smitty”
for his rather unlovable personality) published a seminal paper in the journal
Science, on the idea of levels of measurement and the kinds of statistical
analysis that could be used for each one. The idea here is that we can and do
measure events in different ways. If I look at a painting and say that it
contains greens, blues, and pinks, I am measuring in a very simple way. If I
say that this painting is greener than that one, I am measuring in a somewhat
more complex way. And if I say I have counted 92 green one-inch squares in this
painting and 95 green one-inch squares in that one, I am at the most complex
level of measurement of all. If I can count how many green squares there are in
different paintings, I can say, for example, that one painting has twice as
many green squares as another one, or I can state the mean number of green
squares in a group of paintings. I can’t do that kind of arithmetic if I am
simply saying that one is greener than another, and if I can’t do that kind of
arithmetic, I can’t use the various important parametric statistics like means,
standard deviations, Student’s t, or ANOVA. Instead, I have to use some less refined
statistics from a group called nonparametric statistics.
Harman’s statistical
analysis falls by the wayside because she has ignored levels of measurement,
just as Bernet and Baker have done elsewhere. She has performed what someone has
called “quantitative alchemy” by trying to force measurements of what is
“greener” (more desirable) into statistical procedures that are meant for
measures that can be added, subtracted, multiplied and divided. This does not
lead to meaningful conclusions.
Harman, Bernet, Baker,
and other PA proponents have an unfortunate affection for Likert scales. These
are the ubiquitous ways of measuring that ask people to rate an event or
statement as “don’t agree at all”, “agree somewhat”, “totally agree”, or some
version of that procedure. That is what Harman did in her study: observers of
videorecordings rated events they saw on a Likert scale. This scale does not
permit any of the arithmetical operations needed for parametric statistics,
because it is not reasonable to claim (for instance) that a rating of 5 is 5
times a rating of one. Nevertheless, Harman and her colleagues (including the
statistician who sees to have been given authorship for that reason) have gone
merrily along and presented means and other parametric statistics as if they
are appropriate.
Just for the sake of
argument, however, let’s for a moment accept the use of the parametric
statistics and look at how the rated behaviors changed over time. I would have
assumed that the goal of TPFF was to improve the parent-child relationship
quality. Harman et al say “we did not anticipate large changes in relationship
quality in a short, 4-day intervention”, and they were apparently right not to
anticipate such, as the t for this measure was -0.66, p=0.51. (I would ask,
though, if you don’t anticipate a change in 4 days, why design a 4-day
intervention?)
As a final point on
measurement in this study, it was unclear to me why Harman did not just use a
count of events, which could have been analyzed parametrically, and did not
present any information about whether the observed behaviors were performed by
the child or by the parent. Are any reported changes to be read as changes in
the child’s behavior, the parent’s behavior, or both? Perhaps the moral of this
story is that there should be a license required to use the SPSS package.
III.
Levels of severity and their implications
for research design
Harman et al title their
report as involving “severely alienated children”. Yet here is what they say
about the children: “The majority of the children who [had] refused to have ay
relationship with the alienated parent traveled significant distances across
the country and spent several hours or days with them prior to the
intervention…a large number of alienated children had transferred custody
to the alienated parent for between a week and as long as 44 days after the
order had been entered.” In other words, although these children had been said
(by someone) to be severely resistant to contact with one parent, they now were
not resisting at all even before the TPFF program began.
If it is true that the
children were not resistant when they arrived at TPFF, what confounding variables must we consider as
possibly causing this change from the previously reported resistance? Is the
court order alone the cause of any changes suggested as resulting from TPFF? Is
TPFF to be praised as not having caused the children to become resistant again?
Why is it appropriate to use an expensive and disruptive program if the
children are already doing what was wanted, and why, in particular, should the
preferred parent be ordered to participate in treatment with a TPFF-approved
therapist? This would appear to be a good business model but a weak treatment
model.
Beyond the issue of
levels of severity, of course, there is the much larger issue of how PA is
identified to begin with. The only consistency in the various treatment program
outcome studies is that a court has decided there is PA present. On what basis
that has been done remains unclear. Harman, unfortunately, states that the
Parental Acceptance-Rejection Questionnaire, discussed by Bernet and colleagues
as a way to discuss “splitting”, is a PA assessment tool! I hope her PA colleagues
will talk to her about this point.
IV.
Levels of harm
The possibility that PA
treatments can be harmful to children and adolescents, which I as well as other
authors have considered, is of much concern to PA proponents. Harman et al,
like other PA advocates, remark that my published discussion of this point is
based on a small number of cases in which the children claiming harms were
still alienated. There has been a gradual increase in the number of anecdotal
claims of this kind, with at least one PTSD diagnosis, but I would be the first
to acknowledge that they have never been investigated systematically, nor have
the reports brought PA proponents to look for adverse effects of their
programs. There are a number of lawsuits in the making as some affected children
approach their 18th birthdays, and those will bring out some
interesting information.
Anyone treating children
and adolescents is bound to have concerns about very serious harms,
self-injury, depression, suicidality or completed suicide, or running away and
being trafficked or otherwise injured. Harman et al stated that no harms were
caused by TPFF, as no children ran away or self-harmed during the intervention or
some subsequent period. These measures are important but minimal. It may be
that following the expected lawsuits we will see some evidence of harms that
proponents of PA interventions will have to consider in the future.
Conclusion
On a number of different
levels, the outcome research on TPFF reported by Harman et al fails to support
the conclusions that TPFF is a safe and effective treatment for children in
cases of parental alienation. In addition, the serious flaws of design and
analysis in this work suggest that the critiques of Joan Meier’s work, recently
presented by PA advocates, may be open to serious question.
No comments:
Post a Comment