USING LEARNING ANALYTICS TO IMPROVE STUDENT LEARNING
Iterative cycles of improvement take place in TEL environments in the same way that you might currently undertake the review process in your campus-based units. The feedback students provide about your TEL unit at the end of the semester can be valuable in helping you improve and refine your teaching.
Soliciting student feedback during the beginning and mid-semester has the additional benefit of allowing you to hear your students’ concerns while there is still time in the semester to make appropriate changes. As you review your unit, you should also reflect on the TEL pedagogy principles and the design blueprint that you have constructed at the beginning of the handbook.
THERE ARE THREE COMPONENTS OF THE REVIEW OF YOUR TEL UNIT:
STARTING YOUR ANALYSIS
The following diagram on learning evaluation is a useful place to start as you begin the review of your unit.
If you are reviewing the formative learning that took place during your unit, for example, you will likely be more interested in evaluating your unit through feedback provided informally. Evaluation about summative results, however, is more likely to come through assessment tasks.
The following practice guidelines will provide you with a list of considerations that you should think about to get your review process started.
You can also use the reports that are currently available in vUWS. These are available through the left-hand menu of your vUWS site under ‘Evaluation’ and can provide you with basic information on student login information, grade centre summaries and various vUWS-related activities such as discussion boards or blogs.
The information that you have gathered from students themselves can be combined with data generated from vUWS reports to provide you with a global picture of student engagement in your vUWS unit.
ANALYSIS OF ENGAGEMENT
If you find there is specific content that students are not engaging with, consider alterations to make your unit more student-centred.
For more on design for student engagement refer to QM General Standard 5. QM General Standards 2, 3, 4 and 8 are also relevant.
Consider the following checklist questions:
- Have you ensured that all material is accessible?
- Is support material easily accessible?
- Did you engage with students regularly throughout the unit?
- Were formative and summative assessments clearly outlined?
- Was the unit guide clearly accessible?
- Were expectations about involvement in discussion boards clearly articulated?
- Were knowledge sharing activities designed to promote student engagement?
- Was the pedagogical principle of authentic learning followed? How might you change your activities to be more relevant and engaging? You should refer to the section on design of student-centred learning activities.
ANALYSIS OF ASSESSMENT
To analyse the assessment in your unit it is important to distinguish between formative and summative assessment.
To analyse formative assessments the following can be used to guide your analysis:
- Feedback from 3-minute evaluations. Did students, for example, have any concerns about particular topics or ideas? How will you address these concerns in the development or review of learning activities in your unit?
- Student performance on formative quizzes can be used as a diagnostic tool to identify areas that students are having difficulty with. How can you use the results from formative assessments in subsequent teaching sessions?
- Informal feedback can be used to gauge levels of student engagement or interest in topics. How will you integrate this feedback into your learning activities?
In the analysis of summative assessments in your unit the following questions can be used to guide you:
- Were assessment tasks aligned with learning activities and learning outcomes?
- Were assessment activities authentic and relevant? How could you build more engaging assessment tasks?
- Were students clear which tasks were formative and which were summative? Was this clearly communicated?
- Was the schedule of assessment activities clearly outlined?
- Students can be overwhelmed with high-stakes assessments. Have you reflected on providing a series of summative tasks throughout your unit?
- If students participated in group assessments, were the instructions and expectations for engagement clearly outlined and articulated?
Here is a planner and an example to guide you through your review of your assessment design.
|Name of assessment activity||Issues with activity||Questions to ask||Improving the assessment activity|
|Quiz to examine demonstration of student knowledge of methods||Questions were too abstract||Create a series of summative tasks throughout the unit rather than one high-stakes quiz.
Is it tied to the learning outcomes and learning activities?
Is the rubric clear? Can it be revised?
|Create a series of summative tasks throughout the unit rather than one high-stakes quiz.
One task is to create a semi-structured interview in student area of interest, conduct interview, develop themes and reflect on the process.
Revise assessment rubric.
For more on assessment design refer to QM General Standard 2 and 3.