Interim evaluation of PAX Good Behaviour Game (GBG)

In order to prevent risk behaviour in pupils and increase their coping in society, the National Institute for Health Development (NIHD) (Tervise Arengu Instituut) is implementing the PAX Good Behaviour Game (GBG) in Estonia in 2015-2021. The programme is co-funded by the European Social Fund and the Ministry of Interior of Estonia. The game was first piloted in Estonia in the schoolyear of 2014/15. As of autumn 2017, when the interim evaluation was carried out, the PAX Good Behaviour Game has been implemented in 77 schools and 162 classes. By 2021, there will be a total of 220 classes and 4500 pupils participating in the game.

The aim of the interim evaluation was to gather practical recommendations on how to effectively use the PAX methodology in Estonia as well as to outline a continuous plan for distributing it on the system level. The interim evaluation was based on two previous schoolyears, i.e. 2015/16 and 2016/17. To fulfil the aim, the implementation processes were analysed to evaluate their efficiency in fulfilling the aims of the game methodology, and whether they are relevant and sustainable. The secondary aim was to evaluate sustainability in terms of the institutional readiness to implement the methodology in the foreseen distribution scope by the year 2021. To answer the evaluation questions, existing challenges were mapped on different levels (incl. the NIHD, mentors, teachers, representatives of the school boards, experts in the field) by using semi-structured interviews, focus group interviews and surveys.

GBG and other similar preventive programmes

There are several preventive programmes implemented in Estonia which help to develop self-control and social skills among children. Although awareness of such programmes is high in schools, they often lack a more detailed overview of the specifics of each programme which would help them select the best-suited programme for their pupils’ needs. Joining such programmes is usually not a long-term strategic decision, but the decision to join a programme is largely dependent on whether the teachers or members of the school board think that a programme is useful for their school or class, or not. Among other programmes, GBG is perceived as a stronger and more appropriate measure to shape the behaviour of children. However, this perception is affected by the lack of awareness of all the programmes aimed at developing self-control and social skills among children. The advantages of GBG compared to other similar programmes are the number of elements in use, the possibility to use the methodology as a natural part of the lesson, learning self-control and playful behaviour, evidence-based methodology, support from the mentor, and for the members of the schoolboard, no additional costs for participating in the programme.

The introduction of GBG

The interim evaluation shows that GBG has been launched successfully in Estonia. This success can be attributed to the well-functioning support network provided both by mentors and NIHD and the visible improvement in children’s behaviour and mood when implementing the methodology. When joining the GBG programme, the teacher’s motivation to implement the methodology is the most important factor, and this is influenced by the teacher’s interest in trying out new ideas and methods, the possibility to use GBG within the context of other lesson activities and the school’s or specific class’s need to implement such a programme. The majority of teachers have not indicated any difficulties when implementing GBG. Mentors have a substantial role in this regard as they continuously help to improve the implementation quality by reminding the teacher of the aims of GBG and thus support the implementation of GBG and increasing the teacher’s motivation to implement GBG throughout the schoolyear.

The quality and relevance of GBG

The interim evaluation shows that GBG methodology is relevant to both large and small, Estonian- as well as Russian-language taught, and city and rural area schools. The methodology is regarded especially relevant for first or second graders or even kindergarten children. There was somewhat more prejudice towards the relevance of GBG in the Russian-language taught schools and in classes above the second grade. In addition, there are cases where it is believed that GBG is not an appropriate methodology for children with special needs or behavioural disorders and this is the opposite what the methodology promotes. It is important further on to pay more attention to how these issues are handled in further communication about the GBG methodology.

The interim evaluation also shows that GBG is seen as an efficient prevention method for shaping children’s behaviour. However, it is seen by many as a means of discipline rather than a prevention tool and this reduces the relevance of fulfilling the long-term goals of the methodology. The mentors, nevertheless, consider the quality of implementing the GBG methodology to be high in most cases. Most of the teachers have not made modifications to the methodology. Modifications to the original methodology are generally derived from the need to meet the needs of some specific classes or teachers. In many cases where the methodology has been modified, it has been relevant when done with the assistance of a mentor, and the goals of the programme have been kept in mind. Interviews with various parties indicated that teachers often do not view the omission of some elements as making modifications to the methodology, so the actual number of teachers who have made modifications to the methodology is higher. Not using some elements of GBG derives from the teacher’s inadequate understanding of GBG as a complete methodology, lack of skills for using different elements, short period of time for implementing GBG, as well as the specifics of the class and the teacher. However, due to the mentors’ good work, it is not a common practice to make modifications to the methodology or stop using the GBG methodology.

GBG is generally viewed to have a short-term positive impact which derives from its short implementation period in Estonia. The results of the evidence-based impact evaluation carried out by NIHD will be published in 2018. The perceived impact of GBG is currently regarded as very high. Most of the positive impact is seen with regards to the reduced need to keep order in the class, improved contact with children, increase in the teachers’ self-confidence, and a general improvement in pupils’ mood. There has been no significant negative impact caused by implementing the GBG methodology.

Added work load from implementing GBG

The interim evaluation shows that the added work load from implementing GBG or supporting the teacher is moderate and no added work load is perceived by the teachers, members of schoolboards, support personnel or the mentors. Even though there is a minor increase in the workload for teachers who start implementing GBG, it decreases or remains at the same level when implementing GBG skilfully, as it can be easily integrated into classroom activities and allows to focus on teaching, rather than keeping order in the classroom. For many teachers, it is burdensome to fill the GBG scoreboard as it is dependent on the teacher’s skills to use the GBG scoreboard meaningfully.

Mentors consider their work load low or at an expected level, and the tasks deriving from the mentor’s role are considered reasonable. However, the mentors’ work load may soon increase due to their increased role in carrying out awareness-raising activities. Therefore, it is necessary that mentors themselves would be able to decide whether and to what extent they are able to take on additional tasks. In addition, the mentors’ main job and the additional salary for the extra tasks should be taken into account. The work load of the schools’ support personnel is minor. Exceptions occur only in schools where a member of the school’s support personnel is also a GBG mentor. However, these two roles are mostly regarded as separate positions. Members of the schoolboard have not perceived an increased work load since implementing GBG.

Teachers’ and mentors’ cooperation

Mentors and teachers place high value on cooperation. Also, other relevant parties, e.g. members of the school board, value the mentoring’ system. The key to successful cooperation is openness from both parties and the skill to view the mentor as a helper, not as somebody who is there to check on the teacher’s work. The members of the schoolboard value mentors’ feedback and the help which they provide to the teachers. Mentor’s support is important for the teacher to secure a continuous and high-quality implementation of the GBG methodology in the school environment. It is important to the mentors that teacher’s decision to join the programme has been voluntary and that the members of the school board would be increasingly involved in the implementing of GBG in schools.

National Institute for Health Development’s co-operation with teachers and mentors

The activities carried out by the National Institute for Health Development (NIHD) to implement the programme, organise the trainings and reflection days are highly valued by the teachers. These activities offer diversity, give additional energy and motivation and increase the level of self-reflection, thus decreasing the work load in a long-term perspective. The activities carried out by NIHD are also highly valued by the mentors, as support and information provided to them is always viewed as sufficient. Teachers interact with NIHD mainly through receiving information materials distributed by NIHD, and by participating in the training courses and reflection events organised by NIHD. All three aspects are regarded as high-quality, motivational and sufficient by the teachers. On the other hand, the interim evaluation shows that the methodology functions on the programme level and if the teacher lacks a support structure for motivation, support and exchanging thoughts, the use of the GBG methodology is reduced.

Administrative work load and awareness-raising activities of the NIHD

NIHD place high value on mentors’ input in carrying out awareness-raising activities and developing the methodology, so that it would better meet the specifics of the Estonian school system. Starting from 2017, the role of the long-term mentors will be increased in carrying out awareness-raising activities. This will result in increased time resource left for NIHD to carry out specific activities to further develop the program to better suit the specific needs of the Estonian school system. The distribution of work for project management and coordination activities at NIHD is well allocated.

Awareness of the school board members

School board members’ awareness of the implementation of GBG depends on whether the decision to join the programme is made by the members of the school board themselves, and whether they are strong supporters of the methodology. If the decision to join the programme is made by the school board members, they are more aware of the programme and its implementation process in the school.  Most of the school board members are well-aware of the aims and expected impact of the GBG programme, but awareness regarding other programme-related aspects is lower. 75% of the school board members find that their awareness of implementing the GBG programme should be higher. Whereas some of the teachers found that the school board members should not be aware of, or involved in all GBG-related activities, the mentors as well as NIHD found that the school board members should be more aware of the GBG implementation process in their school. If the school board is well-aware of the implementation and the elements of the methodology, they can, if needed, provide additional support for the teacher. The interim evaluation identified several positive examples from schools where a larger level of involvement of school board members and support personnel had had a significant positive effect on the sustainability and growth in the network of GBG implementers.

School board members’ and support personnel’s support for the teacher

School board members’, support personnel’s and parents’ support for the teacher depends on the school personnel’s involvement in implementing GBG in the specific school. Generally, the whole school personnel are supporting the implementation of the GBG methodology in the school. However, while in some schools, the school personnel or the school board members are directly involved in, or well-aware of the GBG implementation process, in other schools, the teacher is implementing the methodology independently. The support and understanding provided by the school personnel helps to improve the long-term implementation of the methodology and leads to better results.

Parents’ engagement/involvement

Even though engaging/involving parents is the initial part of the GBG methodology, it is not a wide-spread practice in Estonia. Parents are mostly being involved through information sessions, e.g. at parent meetings or student guidance sessions. In some occasions, the teacher makes extra effort to include the parents, for example, by carrying out a short training session together with the teacher and the mentor, inviting the parents to participate in the GBG lesson, or giving them updates on the implementation process via a blog. Conversely, around one quarter of teachers do not include parents at all. At the same time the school board members as well as teachers view parent inclusion as an important aspect. Parents have a key role in the child’s life and the success of a programme, model or game depends on the parents’ support. Various parties indicate that parents are not aware of different preventive programmes nor their content to an extent that would allow them to discuss the related topics with their children and if needed, use some of the elements at home in order to give a holistic view to the programme implementation and improve the child’s behaviour. It is positive, that already around two thirds of the teachers give parents tips on how to use the GBG methodology at home and 90% of teachers forward written compliments (originally Tootles;  kiidud in Estonian) to the parents. It is noteworthy that some parents are already using GBG elements at home, e.g. the timer and written compliments. Parents have indicated that they are motivated to use GBG elements as they can visibly see the impact and they have also noted an improvement in the child’s behaviour at home.

Distribution of the GBG in Estonia

The interim evaluation shows that the current systems for distributing the GBG in Estonia are sufficient to meet the indicators (number of classes and students) set for year 2021. Distribution is supported by a target group oriented introduction strategy, involving teachers at the awareness-raising events to share their experiences, the positive impact of GBG, and to a minor extent, the fact that participation in the programme is free. Members of the school boards are supportive of the use of the GBG methodology, but if they were more involved in the implementation process, the distribution and sustainability levels of GBG could be increased. Information on GBG should also be distributed via subject-specific societies and teacher’s cooperation networks.

The interim evaluation shows that there are somewhat more prejudices in the Russian-language taught schools which may be somewhat an obstacle to the effective distribution of the programme. This is partly influenced by the context in which the Russian-language taught schools operate in, which has also been identified in previous studies. Schools should improve communication both with other schools as well as members of the school board within the school, including explaining the wider goal of the GBG methodology and establishing connections between the methodology and modern education theories. In addition, it should be more widely emphasised that GBG is appropriate for different cultural contexts, and that the methodology has been successfully implemented in various countries where children’s temperament varies greatly, as well as across different school systems. Also, it is necessary to include Russian-language taught schools’ teachers to the network of the Estonian-language taught schools to foster networking and exchange of experience.