Project Details

A project, completed in 2012, to highlight effective systems of on-job assessment and show how these systems can build organisational capability and improve outcomes for learners. A collaboration of the New Zealand Council for Educational Research (NZCER) and the Building and Construction Industry Training Organisation (BCITO).

Aims:

The project aims were to:

  • examine industry training assessment systems at an organisational level
  • extend on the findings of a previous project supported by the Ako Aotearoa National Project Fund – ITO Workplace Assessment Structures (Vaughan & Cameron, 2010)
  • observe assessment practice during the introduction of a new assessment model
  • understand the associated processes and learning outcomes of the new assessment system
  • develop a set of guidelines which can be used for enhancing on-job assessment of apprentices.

Methodology:

The project methodology involved:

  • gathering data from observations of the assessment system in practice
  • observing both formal and less-formal assessment situations, and examining the interactions associated with these situations
  • conducting interviews with key stakeholders in the assessment process, including assessors, apprentices, moderators, and trainer/evaluators.

Team

bcito

Andrew Kear

Project Leader

Building and Construction ITO
NZCER

Ben Gardiner

New Zealand Council for Educational Research (NZCER)
bcito

Loretta Garrow

BCITO
NZCER

Dr Karen Vaughan

New Zealand Council for Educational Research (NZCER)

Status

Completed

Funding

$102,000.00 (excl GST)

$102,000.00 National Project Fund

Key Findings

Key finding from the project included:

  • Training advisors as assessors: Transferring the role of assessor from employers to training advisors has enabled the BCITO to offer a range of training and management opportunities that were not previously financially or operationally viable. Training advisor/assessors can now combine both of their roles to the best effect.
  • The assessment team: Each team comprises an apprentice (learner), a training advisor (assessor), an employer (trainer and evaluator) and a moderator. The assessment team concept repositions everyone’s roles in relation to the key purpose of promoting learning through assessment to produce a well-trained workforce that knows the right things and is skilled in the right ways. It raises awareness of learning and its relationship to assessment.
  • The Training Plan: The Training Plan is one of the key tools that help members of the assessment team to work together to a common purpose. The Training Plan contains overview details about the apprenticeship such as unit standards, credits and dates of assessment. It also contains more detailed records of workplace visits, discussions held, progress comments, evidence used to determine competence, and assessor judgements about competence.
  • The walk-around: The assessment begins with a ‘walk-around’ for about 10 minutes with the apprentice showing the assessor around the site, pointing out the work they have done and providing details about individual tasks involved in the work. This process enabled the training advisor to directly observe apprentice competence through ‘naturally occurring evidence’ derived from the apprentice’s real everyday work, often over a period of time.
  • A custom-made record of work: Trainees record their practical, onsite work as part of the evidence-gathering process that informs assessment decisions. These records of practical work are reviewed by the training advisor during assessment visits, to help to build a picture of the apprentice’s knowledge and skills in relation to the qualification requirements.
  • Assessor support: Moderators support assessors by providing advice around managing relationships with apprentices and employers, gathering of evidence, and the use of assessment tools. In a more formal moderation role, Training Plans and assessment judgements are reviewed.
  • National Moderation Workshops: Through its National Moderation Workshops, the BCITO regularly brings its national group of training advisors/assessors and moderators together for professional development opportunities. These workshops run up to several times a year and occur in addition to moderation sessions at Area Meetings, held on a quarterly basis. These are part of a deliberate strategy for creating an assessment community of practice.

Key Recommendations

Extending the system to other industry training organisations: The BCITO have provided evidence that a principles-based assessment system can work. Their system not only gives life to the four principles of good assessment systems but illustrates how they operate holistically across the different parts of an assessment system. Before embarking on the change to the assessment model in 2009, the BCITO surveyed their employers, and over 80 per cent supported the concept of a changed system. The idea of the assessment team was designed to still include the employer in assessment decisions and make clear the importance of their role as an evaluator of evidence. If there are other ITOs considering moving away from using employers as assessors, it is important that they too canvass the views of their employers on possible new arrangements and learn from how the BCITO have created a new, valued role for employers. The BCITO’s system may be suitable for a wide variety of industries, largely because it relies on professionalising the assessors and developing a solid community of practice.

Building a community of practice within the assessment system | The BCITO has already begun approaching the other trades within its coverage, seeking to extend the assessment model into their industries. In their proposal they have emphasised the training in assessment practice that the BCITO provided to assessors, the well-developed moderation system, and the assessment workload relief for employers. The proposal has been accepted by all trades. A key part of this acceptance has been that the assessment team approach provides industry stakeholders with reassurance that specific trade knowledge will always form part of assessment decisions. Interested ITOs should consider all their stakeholders and how they too might build a community of practice within their assessment systems.

Report

A Transformational System for On-Job Assessment in the Building and Construction Industries

A research report prepared by Karen Vaughan, Ben Gardiner and Jan Eyre.

(PDF, 2.61 MB, 85-pages).

  • 3 December 2012
Read more

Guide

Taking Charge of your Apprenticeship

A guide prepared by Andrew Kear, Karen Vaughan and Ben Gardiner.

(PDF, 2.81 MB, 20-pages).

  • 3 December 2012
Read more