Research - Signs of Safety

Australian Centre for Child Protection Signs of Safety Program of Research

In 2013, in its fifth year of state-wide implementation, the Department for Child Protection and Family Support (DCPFS) has engaged the Australian Centre for Child Protection (ACCP) based at the University of South Australia to establish a three year program of research that will examine the impact and extent to which the Signs of Safety approach delivers reliable improvements and outcomes for:

children at risk of abuse and neglect and/or who are in out-of-home care
parents navigating the child protection system and process
carers looking after children and
practitioners within the Department and external partner agencies delivering the Signs of Safety approach and working in the complex field of child protection

In addition, the program of research will examine the implementation of Signs of Safety over the last five years, paying particular attention to the stages of implementation and the interplay of drivers in operation.

The program of research is led by Research Fellow Dr Mary Salveron and Deputy Director/Associate Professor Leah Bromfield. To date they have been working on the following:

Developing a research framework and model incorporating results logic or theory of change representation to evaluate the impact of the Signs of Safety approach on children, parents, families, carers, practitioners and other external partner agencies and key stakeholders. Results logic (or program logic) “is a systematic, visual representation of the underlying assumptions of a planned program’ (Adamson et al, 2010) and shows why and how a program is presumed to work. Identifying the theory of change underpinning Signs of Safety defines the model for research purposes and will underpin the West Australian and all future international research endeavour.
Children’s Study to test a rating tool to gather the views of children and young people about the degree to which their case workers engaged them and enabled their participation in child protection investigations.
Applying the body of knowledge on diffusion of innovation and implementation science to help explain the organisational journey of the Department as they implemented Signs of Safety across Western Australia.

Publications and presentations arising out of the research endeavour will be prepared and submitted with significant journals throughout the life of the program.

For more information contact Dr Mary Salveron: Mary.Salveron@unisa.edu.au or +61 (0)8 8302 2980.

Signs of Safety Fidelity Research Projects

As the child welfare system continues to implement cutting-edge approaches to its work, one important component of best practice is the measurement of fidelity to practice. Fidelity is the extent to which an intervention is being delivered as intended. Measuring to what degree practices are implemented as intended is critical to determining the best next implementation steps and the degree to which an intervention improves outcomes for children and families. As the Signs of Safety approach has been implemented by more and more jurisdictions, it is typical that agencies and managers ask various questions about fidelity. (See Figure 1)

Figure 1. Some dimensions related to fidelity

Through the vision and leadership of Casey Family Programs and Sacramento County Child Protective Services, and in consultation with the international Signs of Safety community of practitioners, the Fidelity Research Project is now well underway. The project has been established to create a series of assessment tools that will enable agencies to assess the fidelity of Signs of Safety practice of workers, supervisors, leaders, and agencies. Measuring how well and to what degree practices are implemented with fidelity at various levels in the organisation is critical to facilitating change and improvements in the quality and effectiveness of services, ensuring accountability, and reflecting progress toward attaining the shared goals of providers, individuals, and families served within the system.

The fidelity tools are being created sequentially, with current focus on tools for supervisors and parents.[1] (See Figure 2.) Thank you to all of the jurisdictions and individuals who have participated thus far. Your knowledge and involvement has been vital to this research!

An additional set of assessment tools are being developed currently: tools for assessing leadership (levels above supervisor), including a leadership self-assessment, an assessment of the organisation’s culture from the front line perspective, and then an assessment where staff (supervisors and above) could assess the leadership fidelity of the person that directly leads them.

Figure 2. Signs of Safety fidelity assessment phases of work.

Signs of Safety Supervisor Practice Fidelity Assessment

Casey Family Programs in the USA have worked with the Signs of Safety community to assist and guide us in the creation of fidelity tools for the Signs of Safety approach. This work is being done through the Signs of Safety International Research Collaborative, an international work group of Signs of Safety practitioners, agencies, educators, and researchers. Casey Family Programs have just released the first report that describes the development of a supervisor assessment fidelity measurement tool. The full report including the tool can downloaded here.

Practice-based Evidence

The Signs of Safety approach to child protection casework is a practitioner’s model meaning that it has evolved and continues to evolve based on how the approach is actually used and applied by practitioners, teams and agencies. As the model has evolved a range of written material have been published providing extensive ‘practice-based evidence’ (Ferguson 2008) providing rich qualitative data describing what the approach looks like in practice from the perspective of the practitioner and service recipient (see for example Christianson and Maloney, 2006; Teoh, et. al. 2003; Turnell and Elliott and Hogg, 2006, Turnell, 2007).

Existing Research and Evaluation

Since the model was first developed there has been many small studies in many of the jurisdictions using the approach including Western Australia, Drenthe, Netherlands, various Minnesota counties, Gateshead, England, Copenhagen boroughs and Open Home Foundation, New Zealand. The studies in these locales have shown that the Signs of Safety:

Increases worker’s morale
Increases practitioner clarity and decision-making
Improves and focuses relationships between practitioners and families
Improves collaboration between child protection and other professionals
Reduces rates of child removal
Reduces the duration cases are open to the agency

The findings of these studies (Bunn 2013; Caslor 2012; Gardner 2008; DCP 2010; 2011; 2012; Holmgård Sørensen, 2009; 2013; Idzelis Rothe et. al. 2013, Keddell 2012; Skrypek et. al. 2010; 2012; Turnell and Edwards 1999, Westbrook 2006; Wheeler and Hogg 2011) are explored in more detail in chapter five of the Signs of Safety Briefing Paper (Turnell, 2012). Chapter 5 is available as a separate download here.

Minnesota Studies

Minnesota counties have led the way in using and implementing the Signs of Safety approach in the USA beginning with Olmsted and Carver Counties with many others following their lead. The Casey Family Programs in conjunction with the Wilder Research have undertaken a series of evaluations and reports regarding the implementation of the Signs of Safety in Minnesota. These are:

In 2008 Western Australia’s Department for Child Protection and Family Services (CPFS) adopted the Signs of Safety as its assessment framework and practice approach for all child protection practice and has been undertaking system-wide implementation and reform built around the approach since that time. CPFS commissioned a comprehensive independent research study of the Signs of Safety implementation led by the Australian Centre for Child Protection at the University of South Australia that is currently underway (see below).

You Can’t Grow Roses in Concrete I & II

In mid-2014, Munro, Turnell and Murphy Child Protection Consulting (MTM) succeeded in securing English government innovations funding to work intensively with ten local authorities over eighteen months. The project became known as the Signs of Safety English Innovations Project. Reporting on transformational and necessarily extensive reforms in ten local authorities is potentially a vast task and the action research was a modest strand of the work. Therefore the reports provide a broad brush picture of what has happened, picking out key themes and illustrating the diversity of the implementations. They are structured as a list of key elements of the reforms, but attention is given to reporting on how successfully these elements must interact to produce the desired changes.

An independent evaluation is being carried out by a team from Kings College, London and that will explore more of the impact on children, young people and families. It also studies implementation and includes a contrast study of two local authorities using Signs of Safety and two who are not.

You Can’t Grow Roses in Concrete I
You Can’t Grow Roses in Concrete II

References

Bunn, A. (2013). Signs of Safety in England, an NSPCC commissioned report on the Signs of Safety model in child protection, London, NSPCC. Available at www.nspcc.org.uk

Caslor, M. (2011). The Metis DR/FE Project Evaluation. Building Capacity Consulting Services, Manitoba, Canada.

Christianson, B. & Maloney, S. (2006). One family’s journey: A case study utilizing complementary conferencing processes. Protecting Children, 21: 31-37.

Department for Child Protection, (2012) 2012 Signs of Safety survey results report DCP, Perth Western Australia.

Ferguson, H. (2008). The theory and practice of critical best practice in social work. In K. Jones, B. Cooper & H. Ferguson (Eds.), Best practice in social work: Critical perspectives (pp. 15 – 37). Basingstoke: Palgrave Macmillan.

Gardner, R. (2008) Developing an effective response to neglect and emotional harm to children, Norwich: UEA/NSPCC. Available at: www.nspcc.org.uk

Holmgård Sørensen, T. (2009), Familien I Centrum, Socialcentrenes Implementering af Løs- ningsfokuserede Metoder, Mål og Rammekontoret for Børn og Familier, Socialforvaltnin- gen, Københavns Kommune.

Holmgård Sørensen, T. (2013), Når forældre netværk skaber sikkerhed for barnet: en evaluering af ‘sikkerhedsplaner’ i arbejdet med udsatte børn familier I Københavns commune. Socialforvaltnin-gen, Københavns Kommune.

Idzelis Rothe, M., Nelson-Dusek, S. and Skrypek, M. (2013) Innovations in child protection services in Minnesota – research chronicle of Carver and Olmsted Counties. St. Paul, MN: Wilder Research. Link

Keddell, E. (2011a). Going home: managing ‘risk’ through relationship in returning chidlren from foster care to their families of origin, Qualitative Social Work available at: qsw.sagepub.com/content/early/2011/07/23/1473325011411010.abstract

Keddell, E. (2011b), Reasoning processes in child protection decision making: negotiating moral minefields and risky relationships, British Journal of Social Work available at: bjsw.oxfordjournals.org/content/early/2011/03/09/bjsw.bcr012

Keddell, E. (2012), Beyond care versus control: decision-making discourses and their functions in child protection social work, PhD Thesis, University of Otago, Dunedin.

Koziolek, D. (2007). Implementing Signs of Safety in Carver County, Child Welfare News, Center for Advanced Studies in Child Welfare, University of Minnesota, Fall 2007: 1-8.

Skrypek, M., Idzelis, M. & Pecora, P.J. (2012). Signs of Safety in Minnesota: Parent perceptions of a Signs of Safety child protection experience. St. Paul, MN: Wilder Research. www.wilder.org/Wilder-Research/Publications/Studies/Forms/Study/docsethomepage.aspx?ID=925&RootFolder=%2FWilder-Research%2FPublications%2FStudies%2FSigns%20of%20Safety

Skrypek, M., Otteson, C. & Owen , G. (2010). Signs of Safety in Minnesota: Early indicators of successful implementation in child protection agencies. St. Paul, MN: Wilder Research. www.wilder.org/Wilder-Research/Publications/Studies/Forms/Study/docsethomepage.aspx?ID=925&RootFolder=%2FWilder-Research%2FPublications%2FStudies%2FSigns%20of%20Safety

Teoh, A.H., Laffer, J., Parton, N. and Turnell, A. (2003). Trafficking in meaning: constructive social work in child protection practice. In C. Hall, K. Juhila, N. Parton, & T. Pösö (Eds.), Cli- ent as practice. London: Jessica Kingsley.

Turnell, A., Elliott, S. and Hogg, V. (2007). Compassionate, safe and rigorous child protection practice with parents of adopted children. Child Abuse Review, 16(2): 108-119.

Turnell A., Lohrbach, S. and Curran, S. (2008). Working with the ‘involuntary client’ in child protection: lessons from successful practice, pp. 104-115. In M. Calder (Ed.) The carrot or the stick? Towards effective practice with involuntary clients, London: Russell House Publishing.

Turnell, A. (2004). Relationship-grounded, safety-organised child protection practice: dreamtime or real-time option for child welfare? Protecting Children, 19(2): 14–25.

Turnell, A. (2006). Constructive Child Protection Practice: An oxymoron or news of difference? Journal of Systemic Therapies, 25(2): 3-12.

Turnell, A. (2007). Enacting the interpretive turn: narrative means toward transformational practice in child protection social work, PhD Thesis, Perth: Curtin University.

Turnell, A. (2012). Signs of Safety Briefing Paper v4 Elia International Ltd available here >

Westbrock, S. (2006). Utilizing the Signs of Safety framework to create effective relationships with child protection service recipients. MSW Clinical Research, University of St Thomas, St Paul Minnesota.

Wheeler, J. and Hogg, V. (2011), Signs of safety and the child protection movement, in C. Franklin, T. Trepper, E. McCollum W. Gingerich, (eds.), Solution-focused brief therapy: a handbook of evidence-based practice, New York: Oxford University Press USA.

[1] The Supervisor and Parent fidelity assessment tools project is being led by a Working Group that consists of Yvonne Humenay Roberts, Peter Pecora, and Kirk O’Brien from Casey Family Programs in Washington; Mike Caslor, Building Capacity in Manitoba, Canada; Dan McCormick, Safe Generations in Minnesota; Marita Sealey, Resolutions Consultancy in Western Australia; Audrey Lee, Department for Child Protection and Family Support in Western Australia; a special Sacramento County work group led by Kim Pearson; and Andrew Turnell. Eric Bruns from the University of Washington and Eileen Munro from the London School of Economics and Political Science are serving as project advisors.