Triangulation in action: Mixed method evaluation of a professional development program for teachers of students with special education needs

Abstract

This article presents a summary review of the design and results of an independently conducted evaluation of a national New Zealand (NZ) Ministry of Education funded contract for professional development of staff of students with special education needs in 49 schools.

The evaluation was conducted as a mixed method design (Johnson & Onwuegbuzie 2004) in the following three phases: a broad questionnaire issued to all participants associated with the development; eight focus groups with a sample of participants; and success case studies (Brinkerhoff, 2003) with four schools.

The most significant overall finding was that regardless of the approach to development engaged in (either action research, AR, or action learning, AL), there was an outstanding recurring characteristic of staff and supporters wanting to see the students excel. Other key participant self-report impacts from the small-scale projects on adaptation of the curriculum fell under the headings of improved social interaction and academic achievement for students, changes in values and attitudes (for students and teachers), and changes in teaching practice. Participants referred to the importance of school context factors (inclusive planning, management support) and internal and external experts as enablers towards the effectiveness of development. Barriers to effectiveness were noted as associated with initial national contract administration, the number of development initiatives involved in and lack of alignment between these varied initiatives. Maintaining and sustaining the effective impact of projects was seen as dependent on: ongoing commitment and follow-through by school management, governors and program teachers; having ongoing funding and support (internal and external); and bringing other staff on board.

The Phase Three success case evaluation revealed an important element that distinguished projects perceived to be highly successful by both the participants and Ministry of Education personnel. In this small proportion of projects the participant action researchers/learners utilised ‘informed’ decision-making. Although many participants in Phases One and Two justified their limited use of informed decision-making by noting that it was either too early to validate project outcome changes, or it was difficult to show causal effect (changes could be attributed to the development program), a hallmark of the four success cases was the use of strong data in the reconnaissance and evaluation phases of the AR and AL and improvement initiatives that were informed by both this data and relevant previous literature.