Addressing Work-related Traumatic Stress Nebraska - Key Findings

Authors

Quality Improvement Center for Workforce Development

Date of this Version

7-2023

Document Type

Article

Abstract

Background

Nebraska’s Department of Child and Family Services (DCFS) started working with the QIC-WD in 2017 to better understand its turnover problem and found that, among workers who left the agency in 2017, it typically happened within the first two years and the turnover rate in 2017 was about 30%. The team examined many contributing factors through a needs assessment and designed CFS Strong to improve employee retention by addressing work-related traumatic stress among the child welfare workforce.

According to the Secondary Traumatic Stress (STS) scale administered in December 2017, the overall presence of STS symptoms in DCFS staff was 48%. This means staff selected Often or Very Often for at least one of the 3 subscales: Arousal, Avoidance, or Intrusion. This measure of STS also assessed respondents’ presence of post-traumatic stress disorder (PTSD). For staff to meet this cutoff, respondents must have indicated a rating of 3 or higher on at least: 1 intrusion item, 3 avoidance items, and 2 arousal items. According to this cutoff, 53% of Nebraska DCFS staff were exhibiting signs of elevated STS.

The site logic model outlined the intervention-specific activities and the expected outcomes based on the overarching theory of change. The evaluation included a study of the implementation and a cluster randomized controlled trial (cRCT) to evaluate the effectiveness of the intervention. The evaluation was primarily a study of Resilience Alliance (RA) in 2019 - 2020, as other elements of CFS Strong (2020 – 2021) were interrupted by the pandemic, and the first known cRCT of RA.

Workforce Demographics

RA was implemented in offices in the Western, Central, Northern, Southeastern and Eastern Service Areas to ensure a mix of rural and urban teams. Supervisory teams (typically one supervisor and six workers) were randomly assigned to participate in CFS Strong and attend RA groups on a weekly basis; there were two teams in each RA group and the groups stayed the same throughout the intervention. Most DCFS workers were white females (female =87%, White=85%, Hispanic=8%, and Black=3%, Non-Hispanic, multi-race=2%, Asian=1%, Indigenous/ Pacific Islander=1%, and Other=1%). The average age of workers was 35 years old, and the average worker had been in the child welfare field for 4.6 years. Most DCFS staff (89%) had a BA or BS degree and an additional 10% had a Master’s degree, and 1% had a PhD. Mean staff tenure in DCFS is around 3.8 years.

Evaluation Results

Nearly 200 workers and supervisors were originally assigned to attend 24 RA sessions led by a trained facilitator over a period of six months. The mean attendance in the RA groups (n=199) was 13.9 sessions and the median was 16 sessions. RA attendance dropped throughout the course of the intervention and ultimately about half of participants attended 70% or more of the sessions. Attrition was related to staff members changing teams, leaving the job or agency, and conflicts with the group meeting day/time that prevented attendance.

RA session participants were asked to complete a survey about their experience after each session. The survey included fifteen items about the usefulness, practicality, and effectiveness of the RA phase of CFS Strong and related materials. Participants answered on an agreement scale where 1 = strongly disagree and 5 = strongly agree. Among those who attended and completed the survey, the mean total quality rating was between “3 = neither agree nor disagree” and “4 = somewhat agree.” The overall rating stayed within the same range but went from a 3.75 after the first half 12 weeks of RA to 3.41 at the end of the 24 weeks indicating a drop in satisfaction over time. The RA content was delivered in a specific order for the first 12 weeks with more flexibility and a replication of topics in the second 12 weeks of the program.

After participating in RA for six months, the expected impacts outlined in the project’s theory of change did not materialize.

• It was expected that RA would cause the child welfare workforce in Nebraska to experience lower levels of secondary traumatic stress and burnout after 6 months, however there were no statistically significant differences in change over time for STS or burnout between the treatment and control groups.
• It was expected that participation in RA would cause the child welfare workforce to report higher levels of resilience after 6 months, but that was also not supported.
• It was expected that participation in RA would cause the child welfare workforce to stay in their current position for a longer time. Participation in RA did cause those in the workforce who participated to report lower intent to search for work and/or leave their current position and to report higher intent to stay in their current position after 6 months.

These results are somewhat unexpected for the first known randomized control trial of RA.

To supplement resiliency skill building and keep attention focused on STS, RA participants engaged in Peer Support Groups (PSG) for an additional six months following RA. PSGs varied in their implementation as they were peer-led, met a minimum of once a month (with some variation), and were impacted by the COVID-19 pandemic. PSG attendance was less than RA attendance with only 33% of participants attending 75% or more of the sessions and 17% attending all PSG sessions. Among those who did participate and continue to meet during the pandemic, some positive anecdotal reports were captured (see text box). Across all groups with any response data, regardless of response rate, the item “Overall I was satisfied with this session” had high mean scores ranging between 3.7 and 5.0 (on the same scale as the RA surveys).

RA and PSG ran for a total of 12 months (2019 – 2020). Throughout this time a central office workforce leader sent weekly text messages to everyone assigned to participate in CFS Strong. These messages included memes, motivational phrases, and reminders of key RA messages. No data on the impact of this element of the intervention is available.

Finally, Restoring Resiliency Response® (RRR) sessions were piloted in three offices across the state between January and July 2021. This pilot was reimagined in light of the pandemic. It was originally intended to roll out statewide but there were implementation challenges related to identifying and training providers, so a smaller scale pilot was run instead with a limited, non-experimental study.

RRR sessions were open to individuals or small groups, based on request, and led by a trained therapist following a traumatic event, such as a child death. Of the fourteen requested sessions, 64% (n = 9) were held for individual workers. Group worker sessions were requested three times and group supervisor sessions were requested twice. Fifty percent of responses on the request form indicated that those who requested sessions were experiencing concurrent stressors such as several child fatalities, violence in the field, or recent personal trauma. No conclusions can be drawn regarding the impact of this element of the intervention on turnover due to the small and limited nature of the pilot.

A cRCT was employed to evaluate the effectiveness of Resilience Alliance and Peer Support Groups on long-term worker and supervisor turnover outcomes. Using human resources administrative data from the Nebraska Department of Health and Human Services the QIC-WD calculated the percentage of employees in each group (experimental or those who were assigned to attend RA and PSG, and a control or comparison) who left their job during a 30-month period including the twelve-month intervention and a follow-up ending December 31, 2021. The analysis found that CFS Strong did not lead to reductions in employee turnover over a 30-month period among caseworkers and supervisors. Furthermore, the QIC-WD examined whether the results differed by turnover intention. However, additional analyses showed that participants in CFS Strong with high initial intent to leave did not actually leave DCFS at different rates than those in the control group with high initial intent.

The QIC-WD also examined those who actually attended RA and found that workers and supervisors with high attendance (75% or more sessions) at CFS Strong (compared to no attendance) did leave the job less frequently over a 30-month period than those who did not attend. However, these two groups differ on their average number of years of experience and their “intent to leave” and “intent to stay” scores prior to starting RA. After controlling for these pre-CFS Strong differences, turnover results showed no difference between the two attendance groups. Thus, the observed differences in turnover are not due to better attendance at CFS Strong, but rather due to other, pre-existing differences between the high attendance and no attendance groups.

The Team

This project would not have been possible without the partnership of DCFS. They identified a statewide team that represented all of the regions and key positions in the organization. The team members committed to and participated in monthly meetings over multiple years to plan and support the initiative and its evaluation. The team was led by Site Implementation Manager, Sharon Kruse, and the implementation would not have been possible without the additional efforts of Talana Sayre, Mike Puls, and Brenda Books to develop supplemental content and recruit facilitators. The state also designated a Data Coordinator, Doug Beran, who was supported by James Baker, to provide essential data for the project.

The QIC-WD would like to acknowledge the work of the team members who contributed to the implementation and evaluation of CFS Strong. QIC-WD team members included Courtney Harrison, Becky Orsi, Kate Stephenson, Dustin Currie, Kayla Rockwell, Ashley Long, Jonathan Litt, and Robert Blagg.