Development of an Observation Tool Designed to Increase Cultural Relationships and Responsive Pedagogy to Raise the Achievement of Māori Students in Secondary Classrooms in Aotearoa New Zealand

The paper discusses the development and conventions for use of a classroom observation tool designed to support secondary school teachers in Aotearoa New Zealand to develop respectful learning relationships and culturally responsive pedagogy in their classrooms. This tool was created within a programme of teacher professional development to support the improvement of indigenous Māori students’ achievement and engagement in learning. The Ministry of Education recognised the need for an extensive change in practices across the entire education sector that required a shift in thinking and behaviour. The observation tool was therefore designed to support formative assessment, focused on change, through deliberate and democratic professionalism. Initial data, whilst not conclusive, suggest this tool has the potential to support more effective cultural relationships and responsive pedagogy in classrooms thus improving learning and engagement among Māori students through increased self-efficacy, pride and a sense of themselves as culturally located.


Introduction: The Context
Education across the world continues to underserve specific groups of clearly identifiable students (OECD, 2013), as evidenced by achievement disparities that continue to be well documented in schools (Bishop, Berryman, & Wearmouth, 2014).It would appear that schooling has an unfortunate way of creating and perpetuating: Images of children in ways that are destructive, in ways that predispose some children to be successful, confident and engaged, and others to become lower achieving, timid or aggressive, reluctant and disengaged.(Shields, Bishop, & Mazawi, 2005, p. 1) Descriptions of high quality and low equity education systems, driven by deficit-oriented views of particular groups, are familiar to educators across the world (Sleeter, 2011).Indigenous students as a whole, although obviously not every individual within it, comprise one such marginalised group (Berryman, Nevin, SooHoo, & Ford, 2015;Sleeter, 2011).
In post-colonialist New Zealand, Māori have long been politically savvy, having had the vote from the beginning of the parliamentary system.However, the difference in lifestyles and life chances between the average Māori and Pākehā (Note 1) is long-standing (Chapple, Jefferies, & Walker, 1997).European colonisation, in New Zealand as elsewhere, is deeply implicated in representing Indigenous peoples "as a pathologized Other" (Shields et al., 2005, p. 2).The status of Māori within the New Zealand education system reflects a continuous cycle of recurring deficit themes in attitudes, expectations and beliefs.Māori have in turn been "civilized", discriminated against, segregated, assimilated and had their traditions and culture fore-grounded by the system, even if not always in ways that Māori themselves would regard as authentic (Bishop & Glynn, 1999).It is therefore no surprise that, in New Zealand, the differences between rich and poor, crime rates and educational achievement are not just drawn along economic lines, but racial ones as well.Since the foundation of these post-colonialist states, the poorest members of society are often the "colonised" indigenous populations.Māori, for example, remain disproportionately represented within the bottom quartile of society in academic achievement and employment prospects (Ministry of Education, 2016a); they do not remain in schooling as long as other students nor are they achieving as highly (Office of the Auditor-General, 2012, 2013, 2015, 2016a, 2016b).Despite many initiatives to raise Māori student achievement, English-medium schooling continues to return lower engagement and achievement rates for Māori than for non-Māori students (Udahemuka, 2016).For example, statistics published by the Ministry of Education (MoE) indicated that in 2015:  10.6 percent of Māori students left school with "little or no formal qualifications" compared to 3.7 percent of European New Zealand students  62% of Māori students left school with NCEA (Note 2) Level 2 or above compared to 83% of European students (MoE, 2016b) International measures confirm this picture of differential achievement.The outcomes of the Program for International Student Assessment (PISA) comparisons across the Organisation for Economic Cooperation and Development (OECD) countries continue to show New Zealand's education system as one that, in terms of education outcomes, achieves high levels of achievement for many students but not for all (OECD, 2004(OECD, , 2007(OECD, , 2010(OECD, , 2012(OECD, , 2015)).As reported in the 2012 PISA survey, for example, overall New Zealand achievement was above the OECD average in reading, mathematics and science, but the achievement of Māori students was both below the New Zealand average and the OECD average (May, Cowles, & Lamy, 2013).A major question arises for educators interested in promoting the future life chances of every young person, therefore: what can be done to promote improved educational opportunities and experiences for Māori students?

New Zealand Policy Response: Ka Hikitia
Over time a number of government initiatives and policies in New Zealand have focused on reducing the ongoing disparity of outcomes between Māori and non-Māori.However, neither a political policy for mandated change nor a set of learned strategies by school personnel has brought about a changed systemic education reality, despite a keen sense of urgency to see this happen in schools and Māori home communities (Berryman, Eley, Ford, & Egan, 2016;Bishop et al., 2014).
In recent years, charged with improving Māori students' experiences in the education system, the MoE launched Ka Hikitia-Managing for Success: Māori Education Strategy 2008-2012(MoE, 2008).The term "Ka Hikitia" translates as a means to "step up", "lift up", or lengthen one's stride' (MoE, 2008, p. 10).This strategy was positioned as "a call to action" (p.11) in order to step up "the performance of the education system to ensure Māori [students] are enjoying education success as Māori [culturally located]" (p.10).It challenged educators to focus collaboratively on making the difference by ensuring Māori students, "in their early years and first years of secondary school are present, engaged and achieving", and that "strong relationships with educators, whānau (Note 3) and iwi (Note 4) are supporting them to excel" (p.5).In so doing, the MoE recognized the need for an extensive change in practices across the entire education sector that required "a shift in thinking and behaviour, a change in attitudes and expectations" (MoE, 2008, p. 4).
However, effecting changes in the status quo in the education system that necessitate radical shifts in thinking is not easily achieved.The implementation of the strategy from 2008 was slower than anticipated, and did little to prepare schools' Boards of Trustees, principals or teachers either to identify what was required or to enact the policy.Consequently, whilst the release of Ka Hikitia was followed by some emerging achievement gains for Māori students along with "pockets of success" (MoE, 2013), its overall influence in effecting system change was disappointing (Auditor General, 2012, 2013, 2016a, 2016b) and disparity in achievement between Māori and non-Māori learners persisted at all levels of education.National statistics continued to reveal that Māori students did not do as well within the schooling system as non-Māori, and Māori learners did not remain in schooling for as long as other students (ibid.).As a result, Māori students overall were leaving school with lower qualifications and fewer future life choices for themselves, thus potentially casting a shadow over the future well-being of New Zealand society as a whole.
These considerations were important in the development and direction of the MoE's most recent Māori education initiatives.Through its Better Public Service Goals (State Services Commission, 2012), the government set the same achievement target for Pākehā, Māori and Pasifika students; that is, by 2018, at least 85% of all New Zealand 18 year olds would achieve success at Level 2 in New Zealand's national qualification, the National Certificate of Educational Achievement (NCEA), or an equivalent qualification, with "an unrelenting focus on lifting achievement for our priority groups (Note 5)" (MoE, 2012, p. 2).This goal presented some challenges given that, in 2011, only 57.1% of Māori students achieved NCEA Level 2.
To inform and refresh the Ka Hikitia strategy, and to attempt to accelerate success for Māori learners in particular, in 2013 the MoE (2013b) requested proposals from providers for a school-based professional development program to raise performance at secondary school level (13 to 18-year olds).The successful provider, a consortium led by the University of Waikato and including Te Whare Wānanga o Awanuiārangi and the University of Auckland, undertook to use an evidence-based, inquiry approach that would be responsive to each individual school.
Over the three years of the contract, the consortium, as required by the MoE (2013b), set out to bring together the learnings from government and locally-funded development and research initiatives addressing aspects of school life that, over the previous decade, had been shown to make a positive impact on Māori students' school experiences and achievement (Bishop et al., 2014).It aimed to grow and embed practices, processes and systems to sustain the increased levels of Māori students' achievement that were anticipated in this new approach.The professional learning and development model that was envisaged was designed to:  build in-school leader and teacher capability to embed culturally responsive pedagogy for Māori learners in classrooms;


promote culturally-informed and inclusive leadership, school governance and school-wide practices; and  foster the kind of positive engagement with local Māori communities that had been identified as crucial in previous research and development projects (Alton-Lee, Robinson, Hohepa, & Lloyd, 2009) in order that schools should benefit from their own communities' funds of cultural knowledge (Moll, Amanti, Neff, & Gonzalez, 1992).

Building on Success: Kia Eke Panuku (Note 6)
The underpinning aim of the consortium's resultant model, "Kia Eke Panuku" was for secondary schools to give life to the Ka Hikitia policy by addressing the aspirations of Māori communities and supporting Māori students to pursue their potential (Berryman & Eley, 2017).The evidence-based inquiry approach was designed to focus on what schools' leadership, teachers, and in turn, Māori students and whānau could do in response to evidence about strengths and/or challenges identified from the range of contexts and settings in which they each engaged.
Each school engaged in professional learning and development that was differentiated and adapted to the school's own evidence; built capacity and expertise within the schools; and invested in local people and their own solutions.This approach required that positive relationships needed to be developed between the consortium team and each school, within the schools, across schools, with whānau, hapū (Note 7) and iwi and with relevant MoE personnel so that all could maintain a clear focus on the kaupapa (Note 8).
Each Kia Eke Panuku school selected a strategic change leadership team to lead its own professional learning and development work.Kia Eke Panuku facilitators, known as "kaitoro" (Note 9) were trained by the external consortium to support and work with strategic change leadership teams.Together they used perception data (Note 10) to audit their current understandings of the nature of their existing in-school policies, processes, curriculum and structures against the following five inter-related elements, previously identified as important in promoting Māori students' engagement and achievement: In response to the findings of the audit, each school, with its kaitoro, developed an action plan that enabled the strategic change leadership team to specify what they were going to do, how they were going to work and with whom.The creation of the classroom observation tool discussed in this paper relates to the enactment of the third point above: the development of cultural relationships and responsive pedagogy for Māori so that classroom contexts for learning would be responsive to students' cultural backgrounds and foster positive relationships.

Culturally Responsive Pedagogy
Culturally responsive pedagogy has been defined differently over time and by a number of authors (Bishop et al., 2014;Castagno & Brayboy, 2008;Gay, 2010;Ladson-Billings, 2006;Sleeter, 2011).Gay (2010), for example, highlights the importance of teaching to and through the strengths of students and adopting approaches that are responsive to them as products of their diverse cultures.In order to do this, she recognises that cultural differences must be seen as assets to be valued.Gay (2010) suggests, "using the cultural knowledge, prior experiences, frames of reference and performance styles of ethnically diverse students to make encounters more relevant to and effective for them" (p.31), and, in so doing, being able to build "bridges of meaningfulness between home and school experiences as well as between academic abstractions and lived sociocultural realities" (ibid.).
Like Gay (2010), who exemplifies "a functional profile of culturally responsive caring-in-action" (pp.51-52), the consortium attempted to do the same.Building on understandings from Te Kotahitanga, a previous programme designed to raise Māori students' achievement, the Kia Eke Panuku team defined culturally responsive pedagogy as that which fosters positive relationships and is purposefully created by teachers/educators to provide learning contexts where:  high learning expectations and relationships of connectedness to Māori students are fundamental;  learners and their families have agency and the right to equity and self-determination through the sharing of power;


prior cultural knowledge and learning experiences of these learners form the basis of new learning and identity as learners;  sense-making is dialogic, interactive and ongoing between students and teachers and between teachers and teachers;  decision-making and classroom practices are informed by relevant evidence; and when  there are common understandings about the focus on student potential through shared roles, contributions and responsibilities.(Adapted from Bishop et al., 2014) Following the school audits, kaitoro provided support with a variety of purpose-built tools so that school teams could activate their action plans and meet their goals.These included the classroom observation tool described in this paper that, ultimately, was intended to facilitate the development, or further development, and embedding of culturally responsive pedagogy in classrooms.

Classroom Observation as a Tool
Across the world, classroom observation has a long history.Its significance can be judged in relation to its role and purpose at any one point in place and time.In the past few decades observation has often assumed a dual role.In classrooms, it can be both a tool for summative assessment of the quality of teaching and learning, in other words, a "form of surveillance" (O'Leary, 2014, p. ix) of practice, as seen through a Foucauldian lens (Foucault, 1977).Or it can be seen as a tool for enhancing teachers' professional learning and development through formative assessment and dialogue (Wragg, 1999).Or, indeed, it can attempt to include both a summative and a formative purpose (O'Leary, 2014).Thus, the different approaches can constitute, on the one hand, a "technology of control" used in summative evaluations of observable behaviour judged against an externally imposed framework at one point in time, or, on the other hand, a tool designed to support formative assessment focused on change through deliberate "democratic professionalism" (O' Leary, 2014, p. ix).This latter approach emphasises "collaborative, co-operative action between teachers and other educational stakeholders" (Day & Sachs, 2004, p. 7), to unify the interests of teachers in developing more responsive and effective pedagogy, and of students in making progress in their learning and academic achievement.
"The rules of observer-observee engagement are likely to differ according to who is observing whom, in what context and for what purpose" (O' Leary, 2014, p. 30).As Wragg (1999, p. 62) comments, "the actual or perceived power relationship between observer and observed is […] a reality that needs to be recognised"."Partnership observation" (Tilstone, 1998) and "observational partnership" (Lawson, 2011), which embody a wider philosophy of professional learning and which "should be conducted in a manner that equates to a professional dialogue" (Association of Teachers & Lecturers, 2008, p. 1) prioritise professional development over surveillance with obvious advantages, "a more open climate, greater trust between colleagues, and the development of strong professional relationships" (Marriott, 2001, p. 3).This is especially so when observation is sustained and collaborative to enable mutual trust and understanding between the participants, and open shared discussion that can contribute to improved teaching and learning.
The purpose of observation should determine how it is used (Wragg, 1999).It is essential that the approaches used to collect data are both fit for purpose, and as accurate as possible.As O'Leary (2014, p. 69) notes: models of observation underpinned by professional development goals tend to be based on a discursive approach whereby collaboration between observer and observe […] should be less hierarchically delineated and driven by a desire to nurture pedagogic knowledge and skills rather than simply passing judgement on the professional competence of the observe on the basis of an isolated observation.
In addition, there is a strong argument that all parties should understand the rationale underpinning the approach to observation that is chosen in any particular situation of circumstance, which data will be recorded, how and why.In the context of promoting culturally responsive pedagogy to support improved achievement of Māori students in New Zealand classrooms, without a common understanding of the purpose of, and agreement to, the observation, as Bishop, Berryman and Wearmouth (2014, p. 139) comment: teacher resistance to change is to be expected, when one considers that many teachers are tired of what Fullan (2007) calls "initiative-it is", where there is an on-going parade of "bold new initiatives" that are going to solve such a seemingly immutable problem as Māori student achievement.
Fundamental to willingness to change practice in classrooms is openness to experimenting with new pedagogies in classrooms.One might ask the question, "Why would teachers open themselves to taking risks and discussing areas for self-improvement?"Key components of any successful partnership observation, where the observer and observed can share understanding and develop further knowledge and expertise in pedagogy are "trust, commitment, common understanding and the identification of individual needs" (Tilstone, 1998, p. 60).It is obvious that teacher professional development must start with individual teachers because, without their willingness to develop, learn and grow, professionally, it seems hardly likely that they will engage in critical reflection on their practice.As Bailey, Curtis and Nunan (2001, p. 5) comment: People can be subjected to assessment, appraisal and evaluation against their will.But no one can be made to develop.Even if you have to complete a portfolio, you can't be made to develop by doing it.Teachers are too good at faking it.We have to be.We can fake development, and should do so, if someone tries to force us.But we develop as professionals, if, and only if, we choose to.
The observation tool, that is the subject of this paper, was viewed in Kia Eke Panuku as an essential tool in the development of learning contexts that are responsive to students' cultural experiences, prior knowledge and that foster positive classroom relationships.The tool was developed to support teachers' understanding of what such cultural responsivity involved.The consortium was only too aware that: … a major reason previous attempts at educational reform have been unsuccessful is that the relationships between students and teachers […] have remained essentially unchanged.The required changes involve personal redefinitions of the way classroom teachers interact with the children and communities they serve.In other words, legislative and policy reforms may be necessary conditions for effective change, but they are not sufficient.Implementation of change is dependent upon the extent to which educators, both collectively and individually, redefine their roles with respect to minority students […] (Cummins, 1986, pp. 18-19).
An obvious implication of Cummins' view here is the potential importance of using classroom observations to help teachers understand the change required "to retain learning at the forefront and to consider teaching primarily in terms of its impact on students' learning" (Hattie, 2012, p. 1).

Provenance of Classroom Observation Tool
Classroom observation tools intended to provide evidence of the effectiveness of classroom teaching and learning interactions are often developed from a positivist perspective that assumes that all behaviours can be objectively observed, accurately recorded, strictly quantified and categorised by an observer (Baer, Wolf, & Risley, 1968).This is the theoretical perspective underpinning an observation tool used in the first four phases of Te Kotahitanga, one of the previous government-funded reform initiatives (Bishop et al., 2014).That particular tool was intended to provide a written record of observed pedagogies, strategies and the range of interactions being used in classrooms, and then evaluated in relation to a given profile of effective teaching, together with information about student engagement and performance.The tool also sought objectively to quantify evidence of the relationships between teachers and Māori students in the form of accurately recorded behaviours (Berryman & Bishop, 2015).The process of observation to feedback was often unidirectional, driven by the observer, and time-consuming, leaving little time before the next observations due to begin.There was also an assumption that, by focusing only on the quantified written records of behaviours, pedagogy that was culturally responsive and promoted positive relationships would be readily appreciated by teachers and become more evident in their classrooms.However, this was not always the case.Even after having feedback from three years of observations many teachers recognised the components of the written records but had little understanding of how they worked together within the overall culturally responsive pedagogy, including the focus on positive relationships, that had the potential to engage Māori students in learning (Bishop, Ladwig, & Berryman, 2014).Further, as Wragg (1999, p. 62) comments, "The actual or perceived power relationship between observer and observed is […] a reality that needs to be recognised".In some schools, there was some suggestion that the imbalance of power between observer and observed was encouraging teacher dependence on the facilitators carrying out the observations, rather than autonomy.In observations such as these, there is an obvious link to Foucault's (1977) concept of the "examination", where teacher behaviour and performance are subjected to a process of objectification, reluctance to take risks and a decline in the creativity of pedagogy (Peake, 2006;Simons & Elliott, 1990).As Openshaw (2007, p. 47) noted, the performative nature of the tool and its use in the previous programme was criticised by some teachers as exemplifying "a process of blame and redemption; surveillance and control" rather than promoting a safe environment in which openly to share perceptions and critical discussion that would lead to deep understanding of what culturally responsive pedagogy implies in practice.
Subsequently, these issues became important considerations in the development of the new classroom observation tool that was taken in to Kia Eke Panuku.

Development of the New Tool
What was needed for Kia Eke Panuku was something different (MoE, 2015).It required knowing the people that the teachers and students in the classrooms were, and understanding their behaviour, interactions and their learning needs by engaging in joint dialogue and sense making.This involved kaitoro working with teachers in ways that acknowledged the importance of positive relationships, were collaborative and responsive, and assumed the centrality of the students and teachers.This called for a means to observe teacher and student classroom behaviours in a respectful way for use in subsequent discussions that would result in developing expertise and shared understandings between the observer and the observed, of what constituted culturally responsive pedagogy (Bishop, et al., 2014;Gay, 2010).Observations needed to occur in ways that would show "the attributes of schooling [within the classroom] that truly make a visible difference to student learning" so that teachers could "go about knowing and understanding, and then doing something about student learning" (Hattie, 2012, p. 1).
The new observation tool was developed to gather evidence of how culturally responsive pedagogy was understood and demonstrated by the teachers, and what influence this pedagogy has on Māori students.As O'Leary (2014, p. 69) notes, "models of observation underpinned by professional development goals tend to be based on a discursive approach …" This tool was intended to provide a basis for the feedback discussion that followed, to enable a teacher to talk about why s/he had behaved in particular ways and further elaborate personal understandings about culturally responsive pedagogy on the part of both the observer and the observed.The feedback stage is obviously the most important in terms of its influence over future development.Within this dialogic space (Bakhtin, 1981;Berryman, SooHoo, & Nevin, 2013), the inter-thinking (Littleton & Mercer, 2013) between teacher and observer would clarify and extend understandings of the pedagogy.While such learning can be understood from a sociocultural view of learning (Vygotsky, 1978), from a Māori cultural perspective it also demonstrates the process of "ako": the reciprocity involved through shared and inter-dependent learning relationships.Interestingly, the word ako is found in the Māori word for the person responsible for the teaching, "kaiako", the teacher, and the person who does the learning, "akonga", the learner, thus demonstrating the cultural context for the reciprocity and interchangeable nature of these roles and responsibilities.

Content and Structure of Observation Tool
The observation tool was designed to be used by a trained observer.Classroom behaviours were recorded as a running record, over five repeated, end-on-end, and three-minute periods (fifteen minutes overall).Following the lesson, the outcomes of discussion between the observer and up to five Māori students about the cognitive level of the lesson and work they had completed are also recorded.Several figures from the tool are included here to illustrate the conventions in its use.
Aspects of the tool from the previous programme were brought into this new tool.For example, there is a space to record with whom the teacher is interacting at particular points in time, whether transmission or dialogic pedagogy is evident, and whether there is evidence in the learning relationships that shows teachers: The new tool provides for records of the five three-minute observations to be completed in a continuous recording.Five identical sections were designed with a space for the observer to record, in the first thirty seconds of the observation, the time, the location of the teacher, the total number of students present, the total number thought to be Māori students, and how many are considered, according to their observed behaviour, to be engaged.As discussed above there is also space on each section for the observer to list the teacher and student behaviours observed in a behavioural running record in the remaining two and a half minutes of each observation.The front page of the observation contains two Likert 1-5 scales that are then completed by the observer to reflect the students' reports of their classroom learning.) that all parties should understand the rationale underpinning the approach to observation that is chosen in any particular situation or circumstance, which data will be recorded, how and why.As a result, this observer training was designed to serve two purposes:  to facilitate a shared understanding of what culturally responsive pedagogy looks like, or does not look like, in classroom practice;


to provide a record of observed behaviours that would be logged as objectively and dispassionately as possible.
One important issue in relation to the observer that has emerged from some research studies, for example O'Leary ( 2011), is that s/he should be experienced in the area being observed.In the current context, observers needed to have expert knowledge about culturally responsive pedagogy in order both to have credibility with those observed, and also be able to contribute to the scaffolding of new learning and inter-thinking on the inter-personal plane during subsequent feedback and discussions.They also needed to be skilled in objective recording of evidence.In this situation, it is really important that observers should record what they see rather than their subjective judgments of the value or effectiveness of the practice, using factual descriptions rather than opinions, and focus on the present rather than the past (Tilstone, 1998).In the feedback session following a classroom observation, the written record is intended to serve as a basis for discussion and co-construction with the teacher about what the teacher and Māori student behaviours signify.
To address both purposes and develop a shared picture of which observable behaviours constitute evidence of culturally responsive teaching relationships and interactions, an exercise based on existing prior knowledge and experience of culturally responsive pedagogy is undertaken with each new cohort of observers.The observers are asked to consider the most culturally responsive and effective teacher they have ever had.This is followed by a discussion of such questions as:


what did those teachers do that made them effective?On a large whiteboard, kaitoro ask observers to list these behaviours under a five-point Likert scale, where five represents "he most culturally responsive" behaviours.When observers are confident they have captured the attributes of their most culturally responsive teachers, the focus is on the other end of the range, "What does a one look like?"Following this comes, "what does a two look like, a three and a four?" Conversations can take more than two hours as behaviours are suggested, questions raised and shared understandings are sought, argued and agreed.While many interpretations of behaviours are made throughout these conversations, kaitoro stress that what must be recorded on the whiteboard are only the behaviours that could be seen and heard.
The importance of elaborating understandings about behaviours indicative of culturally responsive pedagogy that is enabled by this inter-thinking (Littleton & Mercer, 2013) cannot be understated.For the consortium facilitators to have developed a common general matrix on behalf of all schools may well have turned the process of classroom observation into a tick box exercise devoid of any deep understandings and potentially result in an inability to learn the role of the more informed other in scaffolding learning during the feedback and discussion with teachers that follow the observations.The feedback stage is obviously the most important in terms of its influence over future development."As a result of the discussion during feedback, teachers become more aware of their classroom practice (or not in some cases!) and are able to explore ways in which they can develop their existing knowledge and skills" (O'Leary, 2014, p. 83).

Feedback and Discussion
All the evidence collected from the observation, that is, the observer's notes and the Māori students' evaluations, is then used to mediate a feedback and discussion session between the observer and the teacher.Throughout this conversation, the observer and teacher choose examples from the recorded behaviours and discuss them together as they relate, or not, to what they both understand as relationships and interactions appropriate to a culturally responsive pedagogy.It is not expected that every recorded observation statement will be discussed, but rather that this tool will become the basis of a discussion where the observer and teacher together engage with the kind of interaction and inter-thinking (Littleton & Mercer, op cit.) that help to develop their understandings at an inter-and intra-personal level (Vygotsky, 1978).Following this discussion, a plan to support the teacher to continue to develop culturally responsive practice is negotiated.
On the back page of the observation tool is a space for recording notes from the feedback and discussion session as well as the bullet pointed list of culturally responsive and relational pedagogy to remind both the observer and the observed teacher what the focus of this conversation should be (Note 12).

Findings from Trialling the Observation Tool
The Kia Eke Panuku consortium team began introducing the use of the tool with schools in 2014.In 2016, a systematic evaluation began of the extent to which the process of classroom observation, feedback to teachers and co-constructing future plans based on evidence, appeared to have had an effect on teachers' pedagogy and on Māori students' engagement and improved achievement in schools.Whilst data collection was limited, claims about the efficacy of the process in bringing about changed pedagogy in classrooms and consequent improvement in Māori students' achievement must therefore be tentative.However, anecdotal evidence from Kia Eke Panuku school leaders and facilitators suggest that many of the schools' middle leaders have been keen to take responsibility for ensuring teachers within their own faculty/department are embedding culturally responsive and relational pedagogy and continue to see this work as the responsibility of the school-based teams.Some Māori students are beginning to report differences in their classroom learning experiences, also.
The process of professional development to use this observation tool occurred in over 80 of the 94 Kia Eke Panuku schools.Teachers from one school provided unsolicited feedback about the professional development experiences they had received in relation to the tool.One teacher said: Kia Eke Panuku has built really well on our past programmes where we have focused on building relationships with students.I have been very inspired and learned a lot.
Amongst other initiatives, this school had developed a student-mentoring programme and they were using evidence more often to understand how teachers' classroom practices could engage students and promote learning.One teacher commented that the professional learning was helping them to understand what to do once the focus on learning relationships with Māori students had been established.He contended that this was achieved by teachers beginning to understand why and how to change their classroom pedagogy through the use of these classroom observations.I really like the observation tool … I've got a lot of buy-in for this tool because you co-construct the next steps … the observer acts as a mirror for you, rather than just having to provide advice.I think it's really powerful.
In relation to students' experiences, members of Kia Eke Panuku engaged in conversations with 157 senior Māori students who had achieved successful outcomes across a range of academic, sporting and cultural activities from 64 of the schools.Of interest to this paper is that their reported experiences of educational success aligned closely with the Kia Eke Panuku definitions of culturally responsive pedagogies.Members of all groups commented on how important it was when teachers respected their cultural identity and prior knowledge, and how this had happened and could happen for others: We have teachers who have come from England and from other countries who have no te reo Māori (Note 13).
They learn te reo Māori and try to understand it.
We have a teacher who's gone over to the kura (Note 14) next door to learn te reo Māori.That shows me that he appreciates my whakapapa (Note 15), he appreciates where I come from, who I am as a person, and my culture.
That helps me understand where they're coming from as a teacher and what they're trying to teach me.I understand that they want to know who I am, so I want to know what they have to teach me.
Many students also talked about the importance of dialogic, sense making between teachers and students, for example: Our teachers, they're like our pou (Note 16) at school.They relate to us.Their teaching skills are fun, they make it like not so boring, they don't teach down to you.According to these students, teachers respected their right to equity and self-determination and were prepared to go above and beyond to ensure this was achieved.One reported, for example: She's helped me become a prefect, actually pushing me, saying, "The Māori students are always underestimated.I believe in you".Her belief in me really helped me, this year.
Students in every group noted how the focus was on potential being achieved through shared roles and responsibilities between families, teachers and learners: You can approach pretty much all the teachers at our school and they're willing to drop everything and help you out.They're always willing to help anyone, even if you are Māori, if you're not Māori, it doesn't matter.
All students agreed that familial-like relationships, connecting teachers with Māori students, were essential.Many students used the Māori term "whanaungatanga", a term that exemplifies the act of connecting as whānau or as family.One said: Something that helped me is the whanaungatanga with teachers, and also friends.They really pushed me to success.That's something that I think supported me.
While still another student offered how this was exemplified in practice: They're able to pronounce Māori names properly, because not being able to have a teacher pronounce your name properly is quite irritating.A lot of the teachers do try and are working hard to interact with Māori students.
Finally, a number of students talked about the importance of relevant evidence of achievement in school.Having access to their own evidence helped them to understand what they needed to do in order to succeed.Two of the comments in this regard were: We have a special initiative in our school.While this has not the case in all of these schools, evidence had begun to show improvements in Māori student achievement, thus bringing with it collective pride in ability to achieve: The Māori pass rate for NCEA has lifted in our school.It is lifting every year and I think that the word that's been thrown around is pride, and I do carry a lot of pride in seeing that I was part of that stat.
The gradual improvements in Māori students' measured achievements over the period of Kia Eke Panuku appear to bear out students' perceptions.A milestone report to the MoE in December, 2016 (Berryman, unpublished), indicates that, between 2012 and 2015, the achievement of Māori students in many Kia Eke Panuku schools steadily improved.Over this time, the proportion of Māori students achieving NCEA Level 2 in Year 12 increased from 53.3% to 62.3%.The increase for Māori was at a faster rate than non-Māori (9.0% compared to 5%) and the disparity of attainment decreased from 16.6% to 12%.The improved performance of Māori students was most evident in the schools that had been with Kia Eke Panuku for the longest.
Likewise, there was improvement in Year 9 and 10 students' literacy achievements, as measured on the New Zealand-wide "asTTle" tests that were originally developed specifically for the New Zealand context.In 2016, Year 9 Māori students in Kia Eke Panuku schools moved from an average rating of 3A, below the national average, to 4B, the national average.Year 10 Māori students moved from 3A (below average) to 4A.A smaller increase was seen in e-asTTle mathematics results: Year 9 Māori students in Kia Eke Panuku schools remained at 3P (below average), and Year 10 moved from 3A to 4B (Note 17).
Reflecting on the shifts in pedagogy that they had experienced throughout this time, one student said: It's hard not to do well with the support of the teachers and the people around you.They motivate you, and the motivation inspires you to do your best.

Conclusion
It has long been clear that the model of teacher professional development, where teachers are instructed what to do by experts, is not particularly effective for sustaining classroom or school reform.A review of literature on teacher professional learning and development conducted in New Zealand in 2007, where the initiative discussed in this paper was located, concludes that, of greater effectiveness is when teachers are assisted "to translate theory into classroom practice" (Timperley, Wilson, Barrar, & Fung, 2007, p. xxxiii).In this way the aspects of theory and practice, considered to be important, can be better integrated by teachers and used as the basis for making curricular and pedagogical decisions.Clear links between teaching and learning and teachers and learners is at the heart, where teachers can inquire into the effectiveness of their practice with particular groups of students for the purpose of confirming or redefining the effectiveness of their practice (Timperley et al., 2007).Sustainability is then more likely to be assured when these new understandings have been taken to depth and the "skills of inquiry to judge the impact of teaching on learning and to identify next learning steps" (Timperley et al., 2007, p. xxxiii) have been set.
This understanding is at the root of the classroom observation tool that was used as part of Kia Eke Panuku to raise the achievement and engagement of New Zealand's secondary Māori students.The intention was that, through the process of making links between what had been observed of practice and what both observers and observed understand in theory, to be effective culturally responsive pedagogy, both are able to deepen their collective understandings.An approach such as this emphasises the importance of learning alongside each other within non-judgmental and responsive dialogic spaces that can lead to reciprocal benefits for both.The data related to Māori students' achievement gains in classrooms in Kia Eke Panuku schools using this tool is limited and therefore no generalised claims can be substantiated.Nevertheless, the evidence that is available suggests that through the use of this tool, a platform for critical learning conversations was facilitated that was capable of having to unlearn some more traditional pedagogies, but also learning and/or relearning (Wink, 2011) pedagogies that can be more effective for the students who are at its heart.Some leadership teams and teachers in Kia Eke Panuku schools, at least, reported that they were redefining the way they related and interacted with Māori students, so that these and other students of diversity could take their rightful place as valuable contributors to society.


believe in and care for their Māori learners;  have high expectations for Māori learners and their learning;  have prepared well-managed learning contexts;  provide effective teaching strategies;  promote evidence-based learning; and, most importantly,  engage and accelerate improvement for Māori learners.

Figure 1 .
Figure 1.Example from one of the five identical sections

Figure 2 .
Figure 2. Cognitive level and work-completed sections

Figure 3 .
Figure 3. Observation to feedback and discussion They just expect you to do well.Help you when you're down, teach us to get back up.That's important for me because it shows that they have motivation, and they have a passion to understand students at a deeper level.It shows me that they take into consideration my culture and who I am as a person, as a Māori person.It shows that they appreciate that as well.It brings them to a level where you're able to respect them as a teacher and as a person as well.You understand that they value who you are and what you do in your life, so you value them and you want to learn from them.
It not only helps with your leadership skills, but it keeps track of your academic records, your attendance and everything.They really crack the whip and help you to stay on track.Teachers help us to understand what we need for university and what we need to gain scholarships or anything like that.My tutor teacher, he makes sure that if my grades aren't up to standard that I re-submit and get them up.Just teachers in general they've pushed me this year to do my best.