Using learning analytics in courses via Review, Amend, Apply – GeoEd Trek

I’m authoring this blog post at a time of the academic year that is the halfway point for universities on the semester system and wrapping up/starting again stage for schools on the quarter system. Both of these timeline markers are ideal for reflection upon how our instruction has been proceeding, and where we go from here.

Of course, before we make shifts in our pedagogical approaches that will ideally enhance student peformance and engagement, we need to gather data. Whether through qualitative or quantitative approaches, data is valuable to inform our practices of teaching and learning. This blog post will not address the ethical considerations of collection and use of data, but note that it is critical for anyone that wishes to use data collection for research purposes and dissemination to connect with their institutions, Office for Research Protections for IRB review/approval.

Our course management systems are making it even easier to explore course-related data, as universities are designing interfaces that, with one click, allow us to see data for each student and the class as a whole. For example, for my courses, I can see which courses students enrolled in the prior semester. I can learn which math courses a student has taken prior to my course. I can learn the demographics of my students enrolled, which ones are student athletes, etc.

So we have data! Now, what do we do with it (meaning, what can we learn)?

Indiana University has published a Pressbook titled Review, Amend, Apply: A Framework for Using Analytics in the Classroom. Freely open and available for download, this online book presents a strategy for instructors to identify student needs so that those needs can be addressed with evidenced-based practices. The book is divided into the following sections, which include suggested resources for moving forward:

• Review the information available to you about your students’ behavior.
• Amend the information you’ve gathered with student feedback to ensure accuracy.
• Apply evidence-based interventions in your classroom based on your insights.

My institution (Penn State University) has a Data-Informed Pedagogy Faculty Community, and we are currently exploring this book in monthly Microsoft Teams sessions and discussions. Penn State has also added another “R” at the bottom of the list – Review by comparing how your course data looks before and after taking action.

I encourage instructors to move forward with two steps. The second step would be to look through this Pressbook and use the Review/Amend/Apply framework. The first step needs to be the development of the research question – we should never start with the data, or survey our students just to survey them. The Review section will be most helpful if an instructor has a question in mind (Is there an assessment type that students often skip/don’t complete? Which study methods lead students to earn the best scores? Are students watching the videos I post online and viewing them through to the end? etc.).

Implementing practices such as Review, Amend, Apply have the opportunity for us to create a more inclusive, equitable, and supportive learning environment. We can’t address all of the questions we have in just one semester, but by taking some time to work forward with thought and intention, we can improve our own pedagogical practices while increasing student success each time we teach.