AUT’s approach to assessment design in an AI landscape

In the current assessment landscape, the key challenge for universities is how to assure that the learning that has taken place reflects the expected outcomes of a student graduating in a particular discipline.

AUT’s response to this challenge is to take a whole-of-institution approach to assessment redesign. This work is grounded in the assessment principles, policy and procedures that provide a foundation on which to build a sustainable approach to the integration of AI into assessment and feedback practices.

The assessment policy and the two-channel approach

The definition of assessment from the AUT policy is "The ongoing and systematic process of gathering evidence of what a student knows, understands, and is able to demonstrate”.

AUT has adopted and adapted the University of Sydney’s two-lane model to the use of AI in assessment and has reframed this so that assessment sits within one of two channels:

• Channel 1: Assessment tasks completed within a set time period and under set conditions where there is a clear point of invigilated observation of learning. The use of AI tools is not permitted, and assessments are designed to demonstrate expected learning without student use of AI tools
• Channel 2: Assessment tasks that take place across a range of settings (in-person, online, live or asynchronous) in which the use of AI tools is permitted. Assessments are designed to provide insight into students’ learning process beyond the final submission, ensuring that learning can be understood, observed and evidenced in AI-enabled settings..

Importantly, both channels are valuable and contribute to developing a well-rounded, future-ready graduate. Assessments in both show evidence of different types of valuable learning.

Assurance of learning at AUT

• AUT is framing the assurance of learning as the “understanding, observing and evidencing student learning”
• At AUT, assessments in both channels will assure learning we do not, therefore, use the terms secure/open or controlled/uncontrolled
• For AUT assessments in both channels assure different types of valuable learning. Both channels are therefore controlled by design
• If we can't understand, observe or evidence the learning occurring in an assessment - which is the role of assessing – we can’t assure the learning, and the assessment will need to be redesigned so that it sits within channel 1 or channel 2
• What assurance of learning means for a particular assessment task will depend on a range of factors – including its place in the programme and the learning outcomes it is measuring
• Whether an assessment task assures the intended learning is a decision made by the programme team and the faculty Board of Studies, within AUT’s Academic Quality Guidelines

Points of observation of learning

• To assure learning, we need to understand, observe and evidence the student learning that takes place as part of an assessment task
• In invigilated settings, this assurance takes place at a single point in time (a performance, an exam, a supervised practical test)
• In non-invigilated settings, this assurance will include evidence of student learning that has taken place in preparation for the assessment task
• To support assurance of learning AUT has developed the Points of Observation of Learning (POOL) model
• The POOL model is the mechanism that surfaces the expected learning that a student should complete during the development of an assessment task
• POOLs are evidenced through the assessment rubric and carefully weighted to ensure they are valued and become part of the final submission

Supporting programme and course teams

AUT is taking a systematic approach to working with teaching staff to redesign assessments. This includes workshops that take a programme view of assessment and shorter design sprints that are focused on revising assessment tasks.

At the programme level, teams:

• Work together to create and evaluate a current state assessment map
• Consider the appropriate placing of channel 1 and channel 2 assessments, in the context of assuring the programme learning outcomes
• Redesign assessments that sit within channel 1 and channel 2

At the assessment task level, teaching staff:

• Review a current assessment task to evaluate how learning is assured
• Consider potential points of observation of learning and how they could be implemented
• Redesign of the assessment to include these points of observation of learning

Providing clarity for students

AUT believes that the approach outlined above provides greater clarity for students completing assessments in non-invigilated settings. Academic integrity of assessed work is upheld through intentional design that assures learning, rather than a reliance on invigilation, declarations or detection.