Frontline Job Redesign Louisiana - Evaluation Overview

Authors

Quality Improvement Center for Workforce Development

Date of this Version

8-2021

Document Type

Article

Abstract

The QIC-WD evaluation was conducted with the support of the Louisiana Department of Children and Families to determine if a Job Redesign was effective in improving workforce and child welfare outcomes.

Research Questions

A key research question concerned the extent to which the job redesign reduced the amount of time caseworkers spent on administrative tasks and increased time spent on clinical work with children and families. The QIC-WD also tested questions related to the expected effects of the redesign on workers’ perceptions of their job complexity, role overload, and work-life balance, and how these perceptions relate to job satisfaction, case practices and worker tenure as outlined in the project Logic Model. The redesign was hypothesized to lead to improved case practice outcomes such as more frequent and regular family visits, faster provision of relevant services, and more accurate identification of families in need of short-term in-home services or out-of-home placement. Both practice changes and reductions in turnover were expected to lead to future improvements in child and family outcomes such as recurrence and time to permanency.

Evaluation Design

The evaluation design included a process component focused on readiness for change, participant reactions to the redesign model and perceptions of the adequacy of training and coaching supports provided. Outcomes were assessed with a pretest-posttest non-equivalent groups design. Workers providing investigative, in-home, and foster care services in three parishes implementing the job redesign were compared to a group of workers providing the same services in three parishes in the northern region and three parishes in the southern region of the state, selected to be comparable in terms of numbers of case carrying staff, numbers of cases the units handle, and urban-rural makeup of the community. (See the Site Overview for a map and list of control and intervention parishes.)

A variety of measures were used to capture process evaluation data, including a readiness survey, focus groups with redesign participants, and archival data from site-conducted coaching and community of practice activities. A series of time studies served a dual purpose, providing data on implementation fidelity, as well as outcome data on changes over the course of the project in proportion of time spent on administrative versus clinical duties in the experimental and comparison parishes. Practice changes were tracked via a case record review on samples drawn from the experimental parishes, evaluated using an interrupted time series design, and routinely produced management data reports from the experimental and comparison parishes. Data on worker outcomes, as well as factors that might mediate or moderate a relationship between the intervention and these outcomes, were obtained from surveys of staff in the experimental and comparison groups. Administrative data needed to evaluate worker turnover and child and family outcomes were limited by existing state data systems; however, turnover data were available and quarterly snapshots were produced which link workers to case information.

Timeline

The implementation of the redesign project rolled out in three phases beginning in June 2019. During this time, evaluation efforts focused on collecting formative feedback for refinement of the intervention, and assessment of implementation fidelity. Time studies were conducted with participants from the first two roll out groups during the latter half of 2019. These staff also completed a readiness survey and participated in focus groups to gather detailed information about their experiences and any issues that needed to be addressed. Focus groups did not occur for the third group due to Covid-19 restrictions and changes in working arrangements. Baseline surveys, planned for both the experimental and comparison groups in March 2020 were delayed to May 2020 to allow staff more time to adjust to Covid-19 related changes and were shifted from in-person to electronic administration. Follow-up surveys were administered at the end of the study period. Time studies resumed for both the experimental and comparison groups in July 2020 with electronic data collection. Case reviews began in March 2020 and continued through the end of the study period. The QIC-WD’s ability to look at long-term child and family outcomes was impacted by both the time needed to design and implement the job redesign intervention and delays due to changes in how work was performed due to the pandemic. However, outcomes were reviewed in a limited context using shorter timeframes for defining cohorts and follow-up periods than those defined by the CFSR indicators.