PTSD Prevention report, follow-up

This is the report that I was referring to in my earlier post, Martin Seligman's PTSD Prevention Program Doesn't Work.

Institute of Medicine

(pp. 89 and following)

Comprehensive Soldier Fitness

 

In 2009 the Army launched the $125 million Comprehensive Soldier Fitness program (U.S. Army, 2009), the largest universal prevention program of its kind. At present it has already reached 1 million soldiers (Lester et al., 2011b). The goals of the Comprehensive Soldier Fitness (CSF) program are to prevent adverse psychological health consequences of trauma exposure—most notably, PTSD and depression—by increasing resilience in service members before deployment. The CSF program is based, in part, on the Penn Resiliency Program, which was developed by Martin Seligman at the University of Pennsylvania (Cornum et al., 2011). The Penn Resiliency Program is based on positive psychology as well as on cognitive behavioral theories of depression, and it includes training in assertiveness, negotiation, social skills, creative problem solving, the use of optimism and positive explanatory approaches, and decision making.

 

The CSF resilience-building program has four components that are designed to enhance service members’ mental, spiritual, physical, and social capabilities: (1) master resilience training, a 10-day, hands-on, face-to-face training course that includes the principles of positive psychology (Reivich et al., 2011); (2) comprehensive resilience modules (formerly known as Battlemind), which are training modules that focus on specific resilience skills using precepts of positive psychology, cognitive restructuring, mindfulness, and research on posttraumatic stress, unit cohesion, occupational health models, organizational leadership, and deployment in order to prepare service members for military life, combat, and transitioning home; (3) the global assessment tool (GAT), a confidential online 105-question survey that must be taken annually; and (4) institutional resilience training, which is expected to occur at every level of the Noncommissioned Officer Education System and the Officer Education System (U.S. Army, 2013b). Master resilience training for noncommissioned officers and mid-level supervisors is a “train the trainer” component of CSF for sergeants to use with their troops.

 

Versions of the program are also available for military families and Army civilians, although this committee found no evidence of their implementation with these groups. The CSF GAT measures psychosocial well-being in four domains: emotional fitness, social fitness, family fitness, and spiritual fitness. Results of the GAT are used to refer soldiers to programs aimed at enhancing their strengths and addressing their weaknesses, for example, training in flexible thinking if scores in this area are lower than the norm. A similar instrument, the Family GAT, is being developed for soldiers’ spouses and partners to provide advice about possible resources for building emotional assets.

 

Internal Evaluation of CSF

 

Although evaluations that were conducted by CSF staff and were not subject to peer review have demonstrated statistically significant improvement in some GAT subscale scores, the effect sizes have been very small, with no clinically meaningful differences in pre- and posttest scores. Accordingly, it is difficult to argue there has been any meaningful change in GAT scores as a result of participation. For example, in The Comprehensive Soldier Fitness Program Evaluation Report #3: Longitudinal Analysis of the Impact of Master Resilience Training on Self-Reported Resilience and Psychological Health Data (Lester et al., 2011b), in a pre–post comparison the maximum effect size (partial η2) of any outcome measured by the GAT was found to be 0.002 after exposure to the intervention. The only resilience or psychological health measures that saw significant improvement post-exposure were emotional fitness (a 1.31 percent improvement; 0.002 partial η2) and social fitness (a 0.66 percent improvement; 0.000 partial η2) (Lester et al., 2011b).

 

While Lester et al. (2011b) cite these figures as evidence of CSF’s effectiveness for prevention, this committee does not find these results meaningful, given the low level of improvement and the very small effect size. External reviews, discussed below, have raised similar questions concerning the effect sizes of reported findings and related problems in accurate interpretation of the impact.

More recently, in another internal non–peer-reviewed report, The Comprehensive Soldier and Family Fitness Program Evaluation Report #4: Evaluation of Resilience Training and Mental and Behavioral Health Outcomes, Harms et al. (2013) examined psychological health diagnosis outcomes for 7,230 soldiers who received the GAT before Master Resiliency Training was initiated (October 2010) and again approximately 6 months later (about April 2011) and who consented to use of their data for research. The researchers compared five psychological health diagnoses recorded in the U.S. Army Medical Department’s Patient Administration Systems and Biostatistics Activity (anxiety, depression, PTSD, alcohol abuse, and drug abuse) 3 months after return from deployment or completion of the second GAT for the 4,983 who had received the training (80 percent of whom had deployed) versus the 2,247 who had not (72 percent of whom had deployed). Findings revealed no change in the GAT factors and no difference in diagnosis among those receiving the intervention. Therefore, the subsequent mediation analysis performed by the authors cannot be interpreted as evidence of intervention/program impact.

 

External Reviews of CSF

 

In their review of CSF, Steenkamp et al. (2013) observed that the program that served as the blueprint for CSF, the Penn Resiliency program, did not, according to a meta-analysis, produce powerful effects in its own target, preventing depression in civilian adolescents and schoolchildren. The meta-analysis found that although the program reduced symptoms of depression, the effect size was small, and the program did not prevent, delay, or lessen “the intensity or duration of future psychological disorders” (Brunwasser et al., 2009, p. 1051). Prevention trials in adolescents and children find that an improvement in subclinical levels of depression is a more likely outcome than the prevention of a depression diagnosis in the future (Stice et al., 2009). With regard to the prevention of PTSD, Steenkamp and colleagues assert that no data at all support the effectiveness of the Penn Resiliency Program for adults; instead, they say, the best evidence for PTSD prevention can be found not in universal prevention programs like CSF, but in selective and indicated prevention programs, whose strongest effects are in preventing chronic PTSD in those who are already self-reporting clinically diagnosable stressrelated symptoms (Bryant et al., 1998).

 

Steenkamp and colleagues also criticized the GAT as not being designed to assess PTSD symptoms; it assesses only strengths and problems in emotional, social, family, and spiritual domains. “Thus the program evaluation could not adequately assess CSF’s success in preventing PTSD” (Steenkamp et al., 2013, p. 509). Steenkamp and colleagues also question the underlying assumption of the program that increasing resilience prevents onset of PTSD, noting that “it is possible to be psychologically high functioning and still develop PTSD” (p. 510).

 

In their article “The Dark Side of Comprehensive Soldier Fitness,” Eidelson and colleagues (2011) emphasize that CSF was initiated without the use of pilot testing to determine program effectiveness. Like Steenkamp and colleagues, they criticize the application of the Penn Resiliency program in the face of the small effect sizes found in the meta-analysis by Brunwasser et al. (2009). Eidelson and colleagues also criticized the lack of CSF review by an independent ethics board, especially in light of the mandatory nature of the program. They assert that resilience training may “harm our soldiers by making them more likely to engage in combat actions that adversely affect their psychological health” (Eidelson et al., 2011, p. 643).

 

Smith (2013) critiques the CSF program as potentially causing harm. She observes that CSF’s emphasis on positive emotions and reducing the frequency of negative emotions could be detrimental. Service members experiencing negative feelings could feel “marginalized and demoralized for failing to cope using CSF’s strategies” (p. 244). To support this view, Smith cites a study by Norem and Illingworth (2004) finding that when a positive mood is induced, individuals who are pessimists display decreased ability to problem-solve. Smith also argues that CSF shifts responsibility for psychological health away from external causes, such as multiple deployments and prolonged periods of combat stress, and onto the individuals, who blame themselves for not preventing their own disorder. She points out that service members who experience self-blame are at risk for further mood disturbances and poorer quality of life (Smith, 2013).