Models of Practice

What is design-based research?

Design-based research (DBR): 

• addresses research problems in education that are ‘both scientifically and practically significant’ (McKenney & Reeves, 2013) 
• produces research outcomes that affect practice.

DBR uses an iterative, cyclical process consisting of: 

• design
• evaluation
• redesign
• mixed-methods data collection.

In DBR, researchers collaborate with practitioners in real-world educational contexts. 

The design of the two Models of Practice, including each practice brief, was informed by the design principles of Falconer, Finlay, and Fincher (2011). 

The DBR process in the Models of Practice project

The Models of Practice (MoP) were developed using the design–evaluate–redesign method described above, in the following steps:

1. Design: The initial ideas for the practices were developed into Prototype 1.
2. Evaluation: The content in Prototype 1 was validated. The practices were refined and redesigned according to the findings of this content validation process. The result was called Prototype 2.
3. Redesign: The social validity of the Prototype 2 practices was tested. The Prototype 2 practices were refined and redesigned according to the findings of the social validation process, and became Prototype 3.
4. Mixed-methods data collection: Prototype 3 was trialled in classrooms, with the early years (EY) comprising 29 practices and the middle years (MY) comprising 36 practices.

This process is explained in more detail below for each Model of Practice.

The graph shows the stages of the process.

The table below shows how the practices were refined as they went through the process.

Stage 1: Design–evaluate–redesign process

The team aimed to answer the following research question:

Which practices should be embedded in the Model of Practice to support teacher decision-making in relation to the effective education of students on the autism spectrum as they move between early and middle years classrooms?

To answer this research question, the team used the following steps:

Design

• Initial design completed by researchers. The team identified:
  • relevant, comprehensive, teacher-based practices with evidence to support them, including two potential lists for young children with disabilities¹
  • five practice-based publications²
• The design was scrutinised for alignment with the organisers of Belonging, Being, and Becoming.
• 163 practices were scrutinised by the team.
• 31 potential practices were progressed.

Evaluate: Content validity 

• 31 practices were evaluated by five experts in the autism spectrum and education.
• The context validation index was calculated, and practices were retained only if their content validity was excellent according to the index.
• 31 practices were progressed.
• Practices were refined and redesigned. One practice was separated into two.
• 32 practices were progressed.

Evaluate: Social validity 

• 32 practices were evaluated via an online survey completed by 129 Australian teachers. Teachers were asked if they considered each practice to be informed by evidence, and if they used the practices in their classrooms.
• 32 practices were retained.

Redesign

• Practices were refined and redesigned considering feedback.
• Practice briefs were developed.

^{1. Division for Early Childhood Recommended Practices [DEC], 2014; Center on the Social and Emotional Foundations for Early Learning, 2013.}

^{2. Hurth, E., Izeman, Whaley, & Rogers, 1999; Iovannone, Dunlap, Huber, & Kincaid, 2003; Long & Simpson, 2017; Simpson & Crutchfield, 2013.}

The early years practices that were trialled are listed in ‘Key elements’ above.

Read more about each practice on the relevant practice page of inclusionED. 

Stage 2: Trialling the Model of Practice with Australian teachers

Trial design

Before the trial, the team wanted to find out what teachers thought of the Model of Practice. The team:

• surveyed teachers. Teachers rated their knowledge and confidence. The survey included some items from the published rating scale called the Teachers’ Sense of Efficacy Scale (Tschannen-Moran & Hoy, 2001)
• interviewed 33 teachers. Teachers commented on their knowledge and confidence with the specific student group.

During the trial, the team wanted to find out about the level of support teachers received. To find out which methods of support work best, the team offered teachers: 

• face-to-face coaching
• online coaching
• access to online resources and practice briefs only.

After the trial, the team wanted to find out about teachers’ experiences using the Model of Practice. The team:

• surveyed 33 teachers
• interviewed 27 teachers.

At the end of the trial:

• 15 teachers were 'active users' of the practices
• 4 teachers were 'superficial users'
• 8 teachers were 'non-users'.

Participants

38 teachers from 23 schools in New South Wales, Queensland, and Victoria took part in the trial.

Information about schools and teachers that participated in the trial is shown in the table and graphs below.

Findings

Initial impressions of the early years Model of Practice

Teachers' initial impressions of the early years (EY) Model of Practice and the practices associated with each of the three organisers – Belonging, Being, and Becoming – are summarised below. 

Familiarity with practices

The Model of Practice and/or practices were familiar to teachers and made sense to them. Some teachers recognised the model (and the organisers of Belonging, Being, and Becoming) because they know the Early Years Learning Framework (DEEWR, 2009).

‘I love that it's aligned with the Early Years Learning Framework.’ 

Other teachers recognised that the EY Model of Practice was a new practice framework but the practices themselves were familiar. 

‘It’s quite familiar, all the descriptions, but I’ve never seen it laid out like that, under Belonging, Being, and Becoming.’

‘It was all quite familiar, I thought – yeah, that makes common sense; yep, you need to do that; yep.’

Important and useful

Teachers noted the importance or usefulness of the individual practices to early years education.

‘I really think that the class rules and those class schedules and everything are just so important.’

‘I just think the social-emotional learning is very important to me and my kids.’ 

‘I agree with all the practices that are mentioned there. I think they’re all … equally as important as each other.’

Some teachers thought the practices in the Becoming organiser were secondary to those in Being and Belonging.

‘In order for them to be calm, I think they need to have to be able to have that feeling of Being and Belonging, so I do feel like that’s third in line to the other things, because … I think the Becoming part has to come as a result of the other things being in place.’

About one third of the teachers considered the practices useful for all students.

‘I thought it sort of comprises everything we want for not just … ASD students, but also for any student in a class.’

Teachers’ experiences of using the EY Model of Practice

27 teachers were interviewed about using the Model of Practice.

Facilitators supporting teachers

Teachers thought the EY Model of Practice was a valuable resource. 

‘It’s benefited the children, it’s benefited me, and I’ve got a great resource that’s here, that’s self-explanatory, easy to read.’

EY practices were beneficial for all students.

‘Those routines help all Preps, especially … my boy with autism … but it benefited the whole class.'

Teachers thought that foundational practices made the Model of Practice particularly suitable for early-career teachers, recent graduates, or pre-service teachers. 

‘I hope all new teachers can get their hands on it.’

‘I actually showed my student teacher, and she found it really helpful.’

Teachers placed high value on the professional support they received from coaches. 

A close association was observed between active use of the EY Model of Practice and the professional support teachers received: All teachers who received face-to-face support implemented parts of the model in their classrooms. 

Barriers to teachers' use of the EY Model of Practice

The main barrier to teachers using the Model of Practice was lack of time or lack of support.

‘It wasn't the fact that I actually thought about not going back to it … It was just time constraints – with so much stuff happening with work.’ 

‘I think it was a bit of the time restraints, and trying to get everything done also with our everyday teaching and commitments, and assessments, and reporting – just everything of the everyday teaching world, I think, made it quite difficult.’ 

‘I think the model itself is fantastic, but it would be good to have someone come out and explain it to us – actually someone coming out and explaining it to us is much better than us reading it and trying to implement it ourselves.’

Practices adopted by teachers

The following practices were adopted most often by teachers:

Teacher knowledge, confidence, and capability

The ratings given by teachers to the Model of Practice revealed significant changes in teachers’ perceptions of their knowledge, confidence, and capability related to teaching students on the autism spectrum in the early years. 

Before teachers used the EY Model of Practice

Before they used the Model of Practice, around two thirds of teachers rated their knowledge and confidence between moderate to high.

‘I probably would have just put average, I’d say – I don’t feel panicked about it, but I know that there’s still a lot to learn about it, for me.’

‘Some days I feel like, yup, I’ve made a positive impact, and other days I’m like “Oh my God,” … “What have I done?” I guess it's up and down all the time.’

After trialling the EY Model of Practice

Active users of the Model of Practice reported increased knowledge, skills, or feelings of capability.

‘Just by accessing the model and the practice briefs has maintained what I already knew, but it's also deepened my knowledge. And the hyperlinks to outside sources extends that even further.’

‘I think because being able to identify why things weren't working has really helped. So, it's certainly improved my confidence, but also not being so hard on myself with a few things as well.’

Did coaching make a difference to teacher uptake?

Teachers who received professional support from coaches to select and implement practices were more likely to adopt the model. 

• All 9 teachers who received face-to-face support in the trial implemented the practices.
• 3 of the 4 teachers who received support over the phone implemented the practices.

‘We just unpacked the briefs a lot more. I think we were … quite lost before we had the coach – it was very valuable having her coming out. I think we were just trying to focus on too many briefs at once.’

‘I think the model itself is fantastic, but it would be good to have someone come out and explain it to us.’

Stage 1: Design–evaluate–redesign process

The team aimed to answer the following research question:

Which practices should be embedded in the Model of Practice to support teacher decision-making in relation to the effective education of students on the autism spectrum as they move between early and middle years classrooms?

To answer this research question, the team used the following steps:

Design

• Initial design completed by researchers. The team identified potential teacher practices. Practices included in the design stage had to meet five conditions:
  • evidence-based (see definition below)
  • suitable for general education classroom teachers (rather than learning support teachers)
  • whole-class strategies
  • mainstream school–focused
  • single-step program.
• The design was scrutinised for alignment with the organisers of Rigour, Relevance and Relationships.
• 135 practices were scrutinised by the team.
• 44 potential practices were progressed.

How is 'evidence-based' defined?

… a practice is considered evidence based if it was supported by: 

(a) two high quality experimental or quasi-experimental design studies conducted by two different research groups, or 

(b) five high quality single case design studies conducted by three different research groups and involving a total of 20 participants across studies, or 

(c) a combination of research designs that must include at least one high quality experimental/quasiexperimental design, three high quality single case designs, and be conducted by more than one researcher or research group.    

 

                                                          Wong, Odom, Hume...Schultz (2015) p.1958

Evaluate: Content validity 

• 44 practices were evaluated by five experts in the autism spectrum and education.
• The context validation index was calculated, and practices were retained only if their content validity was excellent according to the index.
• 6 practices were progressed.
• Practices were refined and redesigned.
• 38 practices were progressed.

Evaluate: Social validity 

• 38 practices were evaluated via an online survey completed by 129 Australian teachers. Teachers were asked if they considered each practice to be informed by evidence, and if they used the practices in their classrooms.
• 36 practices were retained.

Redesign

• Practices were refined and redesigned considering feedback.
• Practice briefs were developed.

The middle years practices that were trialled are listed in ‘Key elements’ above.

Read more about each practice on the relevant practice page of inclusionED. 

Stage 2: Trialling the Model of Practice with Australian teachers

Trial design

Before the trial, the team wanted to find out what teachers thought of the Model of Practice. The team:

• surveyed 31 teachers. Teachers rated their knowledge and confidence. The survey included some items from the published rating scale called the Teachers’ Sense of Efficacy Scale (Tschannen-Moran & Hoy, 2001)
• interviewed 33 teachers. Teachers commented on their knowledge and confidence with the specific student group.

During the trial, the team wanted to find out about the level of support teachers received. To find out which methods of support work best, the team offered teachers: 

• face-to-face coaching
• online coaching
• access to online resources and practice briefs only.

After the trial, the team wanted to find out about teachers’ experiences using the Model of Practice. The team:

• surveyed 15 teachers
• interviewed 27 teachers.

Participants

34 teachers from nine schools in New South Wales, Queensland, and Victoria took part in the trial.  

The middle years (MY) Model of Practice trial took place over an eight-week period in Terms 3 and 4.

Scheduling for each teacher to receive coaching was problematic in MY. One member of staff from each participating school was nominated as an Autism Instructional Leader (AIL). The AIL received coaching support, and then supported other teachers at their school.  

Information about schools and teachers that participated in the trial is shown in the table and graphs below.

Findings

Initial impressions of the middle years Model of Practice

Teachers' initial impressions of the MY Model of Practice and the practices associated with each of the three organisers – Rigour, Relevance, and Relationships – are summarised below.

Familiarity with practices

The practices were familiar to most of the teachers interviewed. Several teachers reported that they already used some of the practices. 

Even though the practices were familiar, reading through the MY Model of Practice prompted them to evaluate their use of the practices. 

‘When I was reading over these things I said: I do that, I do that. I think I said that I need to do a few of those better.’

Good teaching practice

Teachers reported that the Model of Practice represents good teaching practice in general. They thought that the Model of Practice was:

• accessible
• user-friendly
• presented in a way that made sense to teachers.

‘I really liked it. I think that it just makes sense.’

The Relationships organiser was recognised as an area of importance that should be a focus in teachers’ development.

‘We also need to make sure that we have a good relationship not just with the students, but the family and even the community.’

Teachers’ experiences of using the EY Model of Practice

The 15 teachers surveyed included 6 AILs.

Teachers who used the practice briefs described them as useful and said they would share them with colleagues. Teachers identified three key enablers and barriers to their engagement with the Model of Practice.

Facilitators supporting teachers

Teachers thought the MY Model of Practice offered good strategies, which were presented in well-laid-out, accessible briefs.

‘The three areas are well named with the three Rs – a good memory hook. When I discovered the practice briefs, they made a lot more sense to me. Colour coding was helpful. Spoken in everyday language. Addressed many facets of practice. Layout great – not overcrowded.’

Teachers valued the support provided by AILs and coaches.

‘Face-to-face dialogue is the most effective way to learn and be guided.’

The Model of Practice was considered to be a great reflective tool. While teachers were familiar with many of the practices, the Model of Practice served as a reminder to teachers to reflect on their practice, think about what they were doing well, and identify strategies to further their practices.

‘It allows the teacher to reflect on their own practices to ensure they cater for all students.’

Barriers to teachers' use of the EY Model of Practice

Teachers reported that a lack of time and support was the most significant challenge.

Most teachers had to use their own time (e.g. lunchtime) as no release time was available. One of the better supported teachers noted that the principal took their class.

The timing of support sessions was also important.

‘Information sessions were too spread out. I would constantly need to be reminded what it was and re-explained some things when we started it.’

Some teachers also reported a greater need for explicit instruction on how to use the Model of Practice.

‘Felt a little at sea with some things, especially in the beginning – an expert would have been helpful to give a big picture or summary of the practices and briefs and how they work together.’

Practices adopted by teachers

The following practices were adopted most often by teachers:

Teacher knowledge, confidence, and capability

The ratings given by teachers to the Model of Practice revealed significant changes in teachers’ perceptions of their knowledge and confidence related to teaching students on the autism spectrum in the middle years. Of the 15 teachers surveyed:

• nine reported an increase in their knowledge
• 12 reported an increase in their confidence. 

Before teachers used the EY Model of Practice

Before they used the Model of Practice, teachers identified limited confidence due to:

• challenges with balancing the various needs in the class
• a lack of support
• limited experience with behaviour management 
• limited experience with students with additional needs 
• insufficient time to spend with students.

After trialling the EY Model of Practice

Teachers reported that their confidence increased due to:

• establishing relationships with students
• engaging with families and carers
• using structured classrooms
• collaborating with colleagues
• receiving support from the school.
• No teachers rated their knowledge or confidence as decreasing.

Did coaching make a difference to teacher uptake?

Only the AILs received professional support and were interviewed about this. AILs who received coaching reported complete satisfaction with:

• communication
• collaborative style 
• effectiveness
• productivity 
• capacity building.