Empirically Based Decisions: A Comment

Correspondence concerning this article should be addressed to Larry E. Beutler, Counseling/Clinical/School Psychology Program, Department of Education, University of California, Santa Barbara, California 93106.
E-mail: beutler@education.ucsb.edu

Norcross (2000), Woody (2000), and Garb (2000) raise some important issues that need to be considered in any effort to develop a science based practice. Science progresses best through dialogue among those with divergent views. I find much in the comments of these three reviewers with which I can agree. Some of their comments, however, provide an opportunity to think through some of the apparent difficulties with the approach that I’ve proposed. Because of the limited space available, I will focus on the points of greatest apparent disagreement. I will save the comments on Woody’s response for last because her paper has raised the most issues and deserves the largest amount of attention.

Norcross (2000) has correctly observed that the idea of a principle-driven, integrative psychotherapy is not new and certainly is not my creation. His own efforts in this area are commendable and one must also acknowledge the work of Goldstein and Padawar (1982), Prochaska (1984), and many others. However, with respect to my particular rendition of this theme, Norcross laments that I give too little attention to the therapeutic relationship (I had noted that 2 of the 10 psychotherapy principles applied directly to facilitating the therapeutic alliance). Alternatively, 7 of the principles define how intervention procedures can be selected to fit the needs and characteristics of particular patients. What was not but should be said is that most of these latter principles have implications for the therapeutic relationship as well. Only 2 of the principles that we defined empirically were general enough to exert an effect that was independent of the procedures used and the patient-mediating qualities. These 2 principles emphasize the value of keeping patients informed of the nature of process and progress and the value of providing a safe and respectful environment. We identified these as "basic principles" because they applied to all treatments.

The differential treatment principles, in contrast, were defined as "optimal principles," and they demonstrated that a treatment procedure that is compatible with this particular patient’s personality and presentation enhances the quality of the relationship and, through that process, enhances treatment outcome (Beutler et al., 1999; Beutler, Clarkin, & Bongar, 2000). In fact, it is precisely because of their ability to enhance the quality of the treatment relationship that it is so important to selectively employ treatments that are based on empirically-informed principles. If anything is remarkable in these observations, it is that not all of the optimizing principles we found exerted their effects through enhancements to the treatment relationship. Surprising to us, some procedures and some aspects of patient–treatment "fit" proved to exert a direct effect on improvement, quite independent of the quality of the treatment alliance.

Our research points to how limited the value is of maintaining the traditional distinction between so-called "common" factors, which are largely comprised of relationship factors, and "specific" factors, which are assumed to be independent of the relationship. Such definitions do not do justice either to the complexity of a "helping relationship" or to its centrality in the process of change. The relationship arises from a complex interaction between the procedures used by the therapist and the person with whom he or she is counseling. The relationship must be understood as a factor of influence that is only partially created by a consistently respectful and predictable environment. It also arises when a skillful therapist applies the right procedure to the right patient at the right time. I believe that we must make specific the factors that we have been trained to think of as "nonspecific"by bringing all of those things over which the therapist has some control, including the development of the relationship, under the tent of active, planful, differential, and specific interventions. The horse-training example, with which Norcross (2000) was amused, is an example of how the relationship is both central to and a direct product of what one does as a therapist and the creative ability of the therapist to fit these things to what the patient is and presents. As Woody (2000) observes, the musician follows rules too, but it is the creative application of these rules (interpretation) that produce grand music.

Garb (2000) illustrates, much better than I, the importance of my contention that clinical psychologists are poor observers of themselves. They are not, by nature, self-correcting, and the nature of clinical practice does not ordinarily allow either the feedback or the opportunity for disproof that is so necessary to correct one’s perspective and approach. This observation emphasizes the importance of treatments that are derived from scientific research because only controlled research is specifically designed to provide the needed feedback and disproof. I hope that Woody (2000) is incorrect in suggesting that pointing this out will further polarize practitioners and scientists. Simply, I had hoped to convey the viewpoint that the environments in which practice is conducted and the environments in which research is conducted each have their limitations. I believe that the structure in which most research is done is not conducive to maximizing patient change (I’ve already said that I believe it to be too inflexible), but neither is the clinical environment conducive to detecting treatments that work. Both are necessary to ultimately reach our mutual goals of helping people change.

There is only one brief point made by Garb (2000) with which I take exception. I do not reject the empirically supported treatments (EST) movement. I have been, and still am, part of it. I simply want to modify it (make it more relevant), and I want to supplement it by directing the priority away from expanding the lists of interventions and redirect it to discovering methods that cut across theoretical boundaries. I believe that there is ample evidence that some patients who do not respond to one intervention will respond to another. Thus, I think that we would be well advised to replace the proscriptions that come with each theory of psychotherapy with differential decision rules, all of which are based on empirically derived principles of what works with whom rather than on touting the uniform superiority of one narrow theoretical model over another.

Woody (2000) was the most provocative of the three reviewers, and her comments could evoke an extensive discussion that would clearly go beyond the space limitations of this brief comment. I will respond quickly to several issues that she raised, including the one that served as the basis of the title she selected:

1. Woody (2000) objects to my suggestion that contemporary lists of ESTs are unreasonable and unfeasible. She suggests that such lists are not as long as they appear because there is some reciprocity among manuals. She assumes that when one learns one cognitive therapy manual, that person can then practice the essentials of all or most other cognitive therapy manuals. If this is true, why do such lists include many different (ostensibly distinct) manuals of CT, each for a different disorder or symptom? Why do we not just list one manual or seek to find the common principles that unite the different manuals? As the assumptions she makes unfold, it appears that I have a closet convert in Woody. We apparently both believe that the principles of effective treatment transcend particular manuals and particular disorders. So, why then should we keep up the facade that there are distinctive forms of CT, psychodynamic therapy, or other therapies that have to be separately defined for each specific disorder? Why should the EST work groups devote so much time to ensuring that the various treatment manuals are differentiated and the diagnosis of the patient population to whom they are applied are defined? Currently, only those treatments that use the same manual (or a very clear modification) on two or more occasions, and on a very specific disorder, are tendered approval.

Why? Because the work groups do not share Woody’s (2000) and my assumption of common principles. They operate on the alternative assumption that there is minimal generalization across manuals and across disorders. The question of whether one can use a CT manual developed for eating disorders on a patient with depression or major anxiety disorders, as raised by Woody, is an empirical one and one that I would love to see addressed, but it has not been. And, because there are different manuals defined for the same disorder, is there not the possibility that some of the procedures that are proscribed by CT but prescribed in other EST manuals are also effective? If so, then why not construct a manual, as I have proposed (Beutler & Harwood, in press) that defines the conditions under which procedures that are proscribed by one approach and prescribed by another, may work?

But let us hope that Woody (2000) and I are right in believing that there are common principles that can be extracted from one manual and applied to another disorder with minimal disruption. Because if we are not, then my original assertion is defensible—The tactic of defining long lists of manuals to fit the 400 different DSM disorders is infeasible and unreasonable. Note, for example, that the National Institute of Mental Health Treatment of Depression Collaborative Research Program, the program that began the manual revolution, required experienced therapists to subject themselves to a training program that lasted for 2 years. Fewer than 50% of the entering therapists ultimately were considered to have achieved the skill required, and much effort was required to keep therapists from drifting away from the manuals. Nowadays there are literally hundreds of manuals and several work groups that are devoted to the task of adding names to the list of treatments that are empirically supported.

Woody (2000) suggests that some therapists will always be generalists and others will specialize, and that this will save the hapless clinician from having to learn this always-growing list of manualized treatments. Let us define, momentarily, a generalist as one who has learned to treat 20 or more diagnostic groups. Assuming that there is a 50% failure rate, this therapist must be trained on 40 different manuals, each of which may take from 1 to 2 years. I hope medical costs don’t go up to support such an intensive training program that is designed to prepare a cadre of octogenarian clinicians.

As for the specialists, I wonder how reasonable it is, in a highly competitive and shrinking practice environment, to expect that a therapist who has learned interpersonal psychotherapy (IPT) for depression, after 2 years of intensive work, will not also try to apply it to drug abuse, marital problems, and many other conditions for which she or he has not been trained and for which IPT manuals are either not available or are not empirically supported. And what about those who are trained but do not achieve the research criteria of competence, compliance, or skill?

Any of us who have conducted experimental trials research have also had to contend with the fact that even though therapists are closely monitored (another expensive proposition), there is a lot of drift—Therapists drift away from the manual. Unlike Woody, most therapists are not content with the procedures that arise from a single theory (Norcross & Prochaska, 1988). What do we do about them, and who pays for keeping them on track?

2. On another point, Woody (2000) is quite right. There may be no possibility that the field will ever reach a consensus about what constitutes evidence that is sufficient to justify a treatment being incorporated into practice. I would not expect consensus. I would be satisfied if only we would enter into a broad dialogue about the following important question: What evidence of a treatment’s effects would you find sufficiently persuasive to recommend it as a basis for practice, regardless of how inconvenient it may be to change one’s prior practices and how unconventional the treatment might be? This is not a simple question; it demands that we not only consider the evidence for a treatment, but the cost of giving up old practices and beliefs as well. It is a challenge to both scientists and practitioners. But this dialogue would be quite different than the one with which we seem to be currently preoccupied. This latter one assumes that a list of recommended treatments can exist independently of a general understanding of what constitutes the varying standards of evidence by which practitioners respond.

The current Division 12 Committee on Empirically Supported Treatments, of which both Woody and I are members, provides a good example of what such a dialogue would look like. This committee is comprised of very dedicated and truly exceptional individuals. Over the past (nearly) 2 years, we have been engaged in exactly the type of dialogue that I have called for—a discussion of what constitutes an "acceptable" standard of evidence. Many divergent viewpoints have been represented, and a working document has been produced that establishes a set of criteria by which we will soon begin anew to evaluate various treatments. On the surface, these criteria are not much different than those used by the original task force. They are not criteria with which I am very happy because they retain an emphasis on brand name therapies applied to diagnostically defined conditions. They do not explicitly acknowledge the presence of cross-cutting principles, the role of relationship factors, or interactions among treatment procedures and (nondiagnostic) patient qualities. However, I will contribute to and support the work of this committee because I believe that we all have learned a good deal through the process that goes far beyond the product produced.

Unfortunately, this process was confined to a relatively small group of academics. I believe that a dialogue of this sort should be much more broad ranging and should involve the practice community at least as much as it involves the scientific one. And while I acknowledge that it is unlikely that we will reach a consensus, I think we very well may be able to define some boundaries of agreement and gain some increased understanding of the issues with which each interest group contends. My own experience as a member of this committee has been to gain a renewed respect for the problems of evaluating research that might address the question around which I would like us to organize our work: Which specific classes of intervention work with what specific patients under what circumstances, and which ones don’t?

3. Woody's (2000) main point is that my principle-based treatment model throws out the "baby with the bathwater" by discarding ESTs while "failing to provide any concrete standards for treatment selection" (¶ 12). She laments that my principles might allow one to practice such wide-ranging therapies as thought-field therapy (TFT) and recovered memory therapy (RMT), both of which have come under scrutiny of scientists for their unsupported (and maybe unsupportable) claims.

I’ve already noted that I consider my approach to be a supplement to, and not necessarily a complete replacement for, the conventional EST approach. Beyond the need to clarify that point, Woody's (2000) apparent assumption is that these brand name labels (TFT, RMT, CT, IPT, etc.) mean something specific and that these specific things can be understood separately from the effects of their context and delivery, the therapist, or the patient. In contrast, my assumption is that within any brand name therapy, there are many procedures that are both used differently by different therapists and are obscured by the label. The labels perpetuate a myth about therapy uniformity. Some of these procedures are useful and some are not, but I believe that the effects of all must be understood in terms of who delivers them and who receives them. I believe that little is conveyed by the brand name beyond producing stereotypic reactions. If the labels really conveyed something specific, we would not have to have 30 different CT manuals.

If we really were to base our practice on what our science has told us so far, we would know that the unique techniques derived and used by any brand name therapy (dare I say it—probably even thought-field therapy, although the research has not been conducted) produce few unique effects. As Norcross (2000) observes, an empathic and safe relationship is the most truly empirical of the therapies. To the degree that different brand name therapies are effective or ineffective may well be a function of how they balance the use of procedures that are relationship enhancing with those that are not for a particular patient. And on the basis of our research, the factors that develop and heal relationships are only weakly attributable to the therapist’s use of principles that can be applied without knowing something about the particular patient. I believe that healing is largely attributable to how effectively the procedures used enhance the patient’s trust and faith, and how well they fit the person for whom they are applied. Regardless of to what the therapist or patient attributes the change (to tapping oneself or to recovering early memories), the majority of the change is, more objectively, a quality of (a) the effort the patient is motivated to make and (b) the things the therapist does to "join" or collaborate with the patient in that effort. While some procedures have unique power, their use can also be captured within a principle-oriented rather than a technique-oriented intervention.

A principle-driven therapy suggests that selecting interventions that both enhance the therapeutic bond and that fit certain requirements and needs of the patient may be more important in this process than the identity of most of the specific procedures used. Thus, even memory recovery procedures and tapping oneself may have a specific benefit when used with certain, receptive people. Good things may happen in spite of using techniques that are, if one could look at them in an isolated fashion, quite ineffective. As Garb (2000) notes, the therapist’s attributions of what causes what may be both irrelevant and wrong. Although I don’t imply, as Woody (2000) suggests, that therapists must learn a bunch of new techniques, I do think that effective feedback on the value of their interventions and corrections of their attributions may lead them to want to expand their armamentarium to compensate for areas of weakness.

4. That takes us to a concluding argument by Woody (2000) with which I can heartily agree. Specifically, she asserts that much of success in implementing any treatment depends on being able to reliably maintain a treatment when indicated and changing one that is not indicated. A principle-driven therapy, like any other, relies on obtaining reliable feedback of what is working and what is not. Thus, we have developed procedures, based on the principles we have defined (e.g., Beutler, in press; Beutler et al., 2000; Fisher, Beutler, & Williams, 1999), that not only allow but insist on feedback. Feedback of three types is required to implement a reliable principle-driven therapy, all of which are consistent with Woody's emphasis on good measurement: (a) feedback on patient progress, (b) feedback about change in any of the patient variables that indicate the use of different classes of intervention, and (c) feedback to the therapist about their compliance with the treatment recommendations. The first two types of feedback can be obtained by specific procedures that are built into our own work with a prescriptive, principle-driven therapy (Beutler, in press). The third is more complicated because it means that someone external to the therapist and patient must be able to view the treatment and note the therapist’s degree of compliance with the principles. Although we have procedures for doing this as part of our research-derived methods (Beutler & Harwood, in press), the costs of doing so in settings supported by health care management agencies, whose major interests continue to be with costs rather than with benefits, remains an obstacle. Perhaps Woody is right; we should expect more of managed health care companies and more of practicing clinicians to ensure that the treatments that work are actually applied.

References

Garb, H. N. (2000). On empirically based decision making in clinical practice. Prevention & Treatment, 3, Article 29.Available on the World Wide Web: http://journals.apa.org/prevention/volume3/pre0030029c.html.

Norcross, J. (2000). Toward the delineation of empirically based principles in psychotherapy: Commentary on Beutler. Prevention & Treatment, 3, Article 28. Available on the World Wide Web: http://journals.apa.org/prevention/volume3/pre0030028c.html.

Woody, S. R. (2000). On babies and bathwater: Commentary on Beutler (2000). Prevention & Treatment, 3, Article 30. Available on the World Wide Web: http://journals.apa.org/prevention/volume3/pre0030030c.html.