Case-supportive Technology Virginia - Key Findings

Authors

Quality Improvement Center for Workforce Development (QIC-WD)

Document Type

Article

Citation

© The Quality Improvement Center for Workforce Development (QIC-WD)

Abstract

Background

The Virginia Department of Social Services (VDSS) is a state-supervised, locally-administered child welfare system with 120 local departments of social services (LDSS). In 2016, VDSS had a turnover rate of 29% among their entry level Family Services Specialists. In response to child welfare caseworker concerns about excessive burdens associated with administrative, travel, and documentation tasks, VDSS elected to implement two technological interventions: transcription services in 2017-2018 and COMPASS|Mobile in 2019-2020. The QIC-WD developed a logic model, supported implementation, and conducted a short- and long-term outcome evaluation based on a theory of change that described how and why case-supportive technology was expected to lead to the desired outcomes. The intervention was rolled out in 18 LDSS, with representation from each of the state’s five regions.

Workforce Demographics

Data was collected from 422 caseworkers across the 18 participating localities. Of these, 87% were female, 52% were White, non-Hispanic, 34% were Black/African American, 8% identified as Hispanic, and 5% were either multi-racial or identified with other ethnoracial identities. Seventy-eight percent had a bachelor’s degree and 22% had a master’s degree. A little less than a third (29%) had a social work degree. Their average age was 39.5 years old.

Evaluation Findings

Data collected from vendors for each intervention illuminated the frequency of use of case-supportive technology. Diary entries were also completed by caseworkers in the months immediately following implementation to capture their immediate reflections on the supports and challenges associated with implementation. Results indicate that, over time, an average of 35% of eligible staff (N = 400) used transcription services but that the proportion of staff using it decreased over time, especially after the COMPASS|Mobile intervention rolled out and the COVID pandemic began. In contrast, an estimated average of 29% of staff used COMPASS|Mobile before the iPad-only based structured decision making (SDM) made iPad use mandatory. Following the implementation of SDM on the iPads in April 2021 and with the mandate that the SDM tool be completed on the iPads, usage increased dramatically, to approximately 79% on average. Moreover, the iPads served as a ready resource and means for the state to communicate critical COVID resources and COVID-related policy updates to caseworkers across the state in real time. It also enabled the quick rollout of a HIPPA- and HITECH-compliant tool for virtual visiting during lockdown.

Short-term outcomes were examined using the rapid diary studies, documentation data from the state’s Safe Measures information system, and three staff surveys (one just prior to transcription roll out, one before COMPASS|Mobile roll out and final follow-up). Rapid diary study results indicate that staff appreciated the state’s focus on supporting their work and that there were time savings related to travel (to and from the office) and documentation after COMPASS|Mobile was implemented. Staff felt their time was freed to accomplish other tasks, and some felt the accuracy of the information they documented improved due to the mobile work supports. Data reflecting the timeliness of case visit documentation revealed that the percentage of visit documentation that occurred more than a month after the visit was reduced 30-50%, depending on the type of case. The staff surveys suggest that over the course of the study, there were small but statistically significant improvements over time, including:

• reductions in work stress,
• increases in positive feelings about supervisors and co-workers, and
• increases in optimism.

In contrast, perspectives about transcription became less positive over time and no differences on turnover intention measures or perceptions of workload were detected.

COMPASS | Mobile and its Support for Tasks
(n = 304)

There is great variability in the LDSS’ human resource processes and data collection efforts. The centralized repository for human resources data is the LETS system, however the employee identifier that it utilizes is not used in any other data systems. Moreover, because the system historically did not distinguish between child protective services and adult protective services staff, the project team developed a monthly staff survey for the 18 localities to complete reflecting which staff left, moved positions, or were newly hired. As such, while it remains hard to estimate the exact number of staff who perform casework in any given month across the state, the results of the turnover analysis reflect data from the LETS data system and the monthly survey, resulting in a composite picture of turnover. In short, over the course of the study (February 2018 – September 2021), 39% of staff who were identified as child welfare staff departed the agency. Further, chi-square analyses did not detect an association between use of technology and turnover, a result suggesting that the interventions by themselves, may not have been adequate to address the full array of concerns that typically prompt departures from the agency. While this finding is disappointing, the array of data collected from Virginia offers the possibility of robust analytics examining other dynamics related to turnover.

On the other hand, organizational social context (OSC) data from the 18 participating localities suggests that overall, the dynamics of the workplaces operated within a fairly normal range for child welfare agencies. (OSC data was gathered from all QIC-WD sites.) Shifts in dynamics over time were generally favorable, but variation at the locality level was evident, with some localities consistently struggling or outperforming others over time. Still, in general, smaller localities tended to be associated with more positive and less adverse work environments.

There are numerous lessons learned from this experience. Clearly, the use of a common employee identifier across information systems is paramount. VDSS intended to add that field to their LETS system to enable them to pursue deeper efforts to understand dynamics associated with turnover over time and perhaps identify areas for further intervention. This study, as with most, also had to rely on proxies of family engagement, such as case contacts, rather than measures reflecting the depth or quality of the dynamic between workers and families. On the other hand, participating localities expressed immense gratitude for the state’s engagement of directors, supervisors, and caseworkers early on and throughout the COMPASS|Mobile planning and implementation process. The state also created a robust support system to ensure training and ongoing resources were available to support staff using COMPASS|Mobile across the state and their work should be considered a model for such endeavors.

In sum, while the data suggest that the intervention did not impact turnover, it had other important positive effects on other expected outcomes including:

• Enhancing staff beliefs that the VDSS cares about supporting its staff and is willing to invest in efforts to do so by giving all staff technology to facilitate real-time reporting, and providing staff flexibility in when and where casework was completed
• Reducing levels of personal and organizational stress as well as enhancing perceived support by supervisors and co-workers
• Workers reported that families were more engaged with them when using COMPASS|Mobile than they were before its implementation
• Time savings
• Accuracy of information
• Actual timeliness of documentation, as well as pivoting during the pandemic.

The Team

This project would not have been possible without the support of VDSS and partnership in participating LDSS’. The dedication, time, and investment of the case supportive technology team is a testament to the state’s commitment. The Virginia team was ready to implement transcription services at the time the QIC-WD began and readily partnered with the QIC-WD for evaluation support and the implementation of COMPASS|Mobile. LDSS’ and the state provided essential data and valuable insights throughout the project.

The QIC-WD would like to acknowledge the work of the team members who contributed to the implementation and evaluation of the Virginia case-supportive technology intervention. QIC-WD team members included Dana Hollinshead, Anita Barbee, Ida Drury, Mark Ells, Kayla Rockwell, Taylor Herhusky, Ashley Long, Jonathan Litt, and Robert Blagg. The VDSS Site Implementation Manager was Laura Polk, who was succeeded by Elizabeth Overall Lee and the VDSS Data Coordinator was Jessie Forman.