Overview
Assessment quality can have a major impact on students, as it is assessment that defines the subject from the student’s perspective. Unfortunate experiences of assessment can be reflected in poor SFUs for a subject.
To ensure quality, the following considerations are applied to assessment for subjects or programs:
- Alignment – the assessment tasks and criteria validate student attainment of subject learning outcomes. Attainment of subject learning outcomes is an important step towards achievement of program learning outcomes, an important aspect of course accreditation.
- Consistency – assessment criteria and standards are referenced to agreed rubrics or marking guides.
- Moderated and calibrated – grading decisions are consistent between markers
- Reviewed – through a range of processes including student feedback and peer review to ensure continuous improvement of assessment tasks.
Previous sections of this guide have articulated a range of techniques for using assessment to support and guide student learning. This section focuses on how informal and formal approaches to collegial peer review can lead to improvements in assessment practice. It also elaborates on the process for making changes to assessment for subjects at Western if, following the processes in this guide and the peer review processes below, leads to the conclusion that revisions to assessment are necessary.
Why is this important?
To ensure there is alignment with external standards such as AQF and HESF
Under the revised Higher Education Standards Framework (HESF), the University has the following obligations in relation to the quality assurance of its courses, assessment methods and grading:
1.4.3 Methods of assessment are consistent with the learning outcomes being assessed, are capable of confirming that all specified learning outcomes are achieved and that grades awarded reflect the level of student attainment.
5.3.3 Comprehensive reviews of programs of study are informed and supported by regular interim monitoring of the quality of teaching and supervision of research students, student progress and the overall delivery of subjects within each program of study.
5.3.4 Review and improvement activities include regular external referencing of the success of student cohorts against comparable programs of study, including:
a. analyses of progression rates, attrition rates, completion times and rates and, where applicable, comparing different locations of delivery, and
b. the assessment methods and grading of students’ achievement of learning outcomes for selected subjects of study within programs of study.
Quality Assurance at Western
Through the process of curriculum and subject development a range of informal processes are applied to review assessment. These can be applied when an issue arises relating to assessment, such as high failure rates or poor student performance on particular assessment items. Influential factors may be the assessment task itself or the learning activities that the students need to undertake to prepare for it. Possible collegial review processes are summarised in Table 3.
Table 3. Collegial review processes for quality assurance of assessment.
| Assessment stage | Who can provide QA |
| Subject learning outcomes: – Are these right for the subject? | EA, Colleagues |
| Assessment design: – Do the assessment tasks validly assess the learning outcomes? | EA, Colleagues |
| Rubrics: Are the criteria appropriate? – Are the criteria and standards clear and differentiated? | EA, Colleagues |
| Guidelines: Are assessment guidelines clear to the students? | DAP |
| Moderation of marking: – Did the marking team interpret the rubric in the same way? – Did marking identify issues with the assessment? | Subject Coordinator and marking team |
| Review: – Is the assessment effective for the subject? – Does it fit well with the subject design? – Have issues with clarity of assessment, rubrics, and marking been addressed? | Colleagues through peer review DAP, Subject Coordinator and Colleagues through the 360 Degree Subject Review (see Figure 5) and Annual Course Reports |
Peer Review Initiatives
A range of approaches to peer review exist to enable continuous improvements of subjects and assessment. These are both informal and formal processes that can be applied when subjects and programs are in need of review, or when individual academic staff would like peer feedback on aspects of their subject, such as the effectiveness of the assessment. Peer review can be a formative or a summative process, depending on the circumstances and purpose of the review. Peer review processes for quality assurance ensure that a collegial approach can be used for continuous improvement. A wide range of topics related to teaching may be the subject of a peer review, including:
- Teaching (online and f2f)
- Curriculum (programs, subjects)
- Teaching resources
- Learning activities
- Facilitation of student interactions and discussions
- Assessments
Peer review of assessment has been a major focus, given the importance of assessment to student learning.
Peer Review of Assessment
Given the importance of subject assessment for students, assessment is a natural focus for peer review. Peer review of assessment can be conducted informally with colleagues, or through a more formal process that will provide documentation that may be important for tenure, promotion or research purposes.
Informal peer review of assessment may be as simple as arranging a discussion with colleagues on the intention of the assessment task, how it aligns with learning outcomes, how the students respond to the task and how this validates student learning. The discussion could lead to suggestions for improvement.
Additional informal peer review of educational practice activities at Western could influence assessment practice. These include:
- Week four open door (and online open door) in which staff can see examples of teaching practice in other subjects
- Peer review with a friend for a discussion of case studies
- Peer review buddies for collegial peer review within a school
- Teaching squares for interdisciplinary collegial peer review groups
- Support from peer review champions within schools.
These activities are designed to facilitate peer review of teaching and assessment practice. They may be of value for people who wish to review and/or update assessment within a subject or across a program. Contact Learning Futures if you would like to know more about these peer review activities.
A formal process of peer review can be engaged with through the Innovative Research Universities (IRU) network. The IRU provides a portal to enable academic staff to request a peer review of assessment for a subject or to offer to be a peer reviewer. Providing peer review is a valuable professional development activity that also enables insight into teaching and assessment provided by academic colleagues at other universities.
Requesting peer review through the formal IRU process provides documented evidence of the outcomes of the review. Following this up by making improvements to the subject assessment, possibly by requesting another peer review the following year, enables evidence of systematic improvement to be gathered. This may be valuable support for applications for promotion or tenure.
Providing peer review is a service to the discipline and the university. This also may be valuable evidence to support an application for promotion.
The IRU portal calibrates assessment by asking for 12 marked assignments, 3 from each band, the university’s grading guide, the subject learning outcomes and how these relate to the program learning outcomes, and a context statement. The reviewer completes a report that either agrees or disagrees with the grade awarded to each assignment with an explanation of any disagreement, and feedback on the clarity of the learning outcomes. Also through the portal a more in-depth curriculum review can be conducted. In relation to assessment, this could use the seven questions developed by Wilson, Bedford and Readman (2018) in Table 4.
Table 4. Validated questions for peer review of assessment.
| Research by Wilson, Bedford, and Readman (2018) has led to the development and validation of seven questions for peer review of assessment using an online platform. These are: 1. Are the subject learning outcomes aligned with the relevant program learning outcomes? 2. Are the subject learning outcomes appropriate to the AQF level of the subject? 3. Does the assessment task enable all students to demonstrate attainment of the subject learning outcomes and the program learning outcomes? 4. Is the description of the performance standards appropriate to the specific subject learning outcomes and program learning outcomes? 5. Is the method of assessment capable of confirming that all specified program learning outcomes and subject learning outcomes are achieved? 6. Do the grades awarded reflect the level of student attainment? 7. What can be done to improve the assessment of the subject? |
Contact the Office of Quality and Reviews if you wish to use the IRU portal for peer review..
Peer review of subjects
An important aim of quality assurance is that it be data driven to enable review decisions for a subject to be made based on evidence, such as shortcomings in assessment results, student feedback, or tutor reports of difficulties students are experiencing. A wide range of data is available, which can be used for analysis of subject activity that can form a firm basis for a collegial approach to subject review. The 360 Degree Subject Review has been developed to draw on available data and to guide a comprehensive review process, as illustrated in Figure 5.
Figure 5. The 360 Degree Subject Review Process.
By developing and piloting structured review processes, staff at Western have a range of effective tools for quality assurance and continuous improvement when this is necessary.
Resources
Wilson, G., Bedford, S., and Readman, K. (2018). External Peer Review of Assessment. Council of Australasian University Leaders in Learning and Teaching (CAULLT). https://www.caullt.edu.au/wp-content/uploads/2018/11/Resource-Kit_Peer-Review-of-Assessment_091118-Gail-Simon.pdf