One powerful tool instructional designers and faculty can use to ensure quality digital learning is a rubric specifically developed for their context. The Online Learning Consortium (OLC) — one of Every Learner Everywhere’s partner organizations — uses research-based quality standards and benchmarks to create rubrics that it calls scorecards. Some of their quality scorecards focus on online course design, teaching practices in online courses, and using courseware.
But suppose none of the existing scorecards are quite right for an institution, program, or department. That was the case for online courses at Western Oregon University, says Dylan Barth, Vice President of Innovation and Programs at OLC. At the request of WOU’s Dean of Library and Academic Innovation, Barth has been working recently with a team there to develop a customized scorecard tailored to their needs.
“Sometimes there are nuances at an institution that our scorecards don’t address,” Barth explains. “There may be indicators that some institutions aren’t ready for and would need additional programming for, or they may want to add to one.”
Beginning in early 2024, he has been collaborating with the WOU Center for Teaching and Learning’s Seyed Abdollah Shahrokni, Instructional Technology Specialist: Faculty Development Coordinator and Dr. Amy Dawson, Assistant Professor, Biology. After their initial meetings, it became clear that they would benefit from a remix approach to creating exactly what WOU needed.
The remix process
Remixing works, because one advantage of the OLC scorecards is that they are openly available and licensed for the public to use. There are three standardized quality scorecards currently, and a fourth on reviewing courses is in development. The current scorecards are:
- Online Course Quality Review (OSCQR), a course-level rubric designed by Alexandra M. Pickett, director of Online Teaching for the State University of New York Online to help review and improve the instructional design and accessibility of online courses.
- Quality Course Teaching and Instructional Practice (QCTIP), which focuses on the delivery of courseware and helps evaluate the overall classroom experience.
- Digital Courseware and Instructional Practice (DCIP), which focuses on teaching by outlining quality indicators for building an effective classroom experience.
Barth and the WOU team began by engaging instructors from different disciplines to identify goals and to help them understand the existing scorecards. That process determined that an entirely new scorecard wasn’t necessary and that WOU online courses would benefit from adapting the OSCQR and QCTIP scorecards.
Barth praises the result for having two components — an extended version and a less overwhelming version with fundamental elements. While the complete scorecard might have 70 unique indicators, an instructor can use the introductory version to implement the essentials and ease into using a quality scorecard.
“If an instructor implements just those important pieces, it’s going to make a big difference, but then there’s room to grow,” Barth says. “It’s going to be an available resource, and training will accompany it in a way that will hopefully continue those conversations.”
That professional development supported 11 instructors who redesigned their online courses around the rubric for the Summer 2024 term. More professional development and review is planned for the fall, and WOU hopes to officially release the rubric to all instructors for planning Spring 2025 courses.
How online learning quality scorecards impact learning
Barth sees quality scorecards as a conversation builder based on a common language instructors use when discussing their work, “to get people thinking in the right direction, and to get where they need to be to help their students succeed.”
One impact is that those conversations start to influence more than just online courses. After instructors use the scorecards, he says, “they often come back and say, ‘Wow, I’ve learned so much about teaching face to face from learning about how to teach online.’”
Barth points out that WOU did not use a top-down approach for the remix. Instead, this process went to those teaching the courses, got their feedback, and eased them into it, making the scorecard optional at the beginning. They were able to discuss current classroom issues they hope to address with the scorecard, and, as the process continues, it becomes a benchmark for measuring improved teaching and learning.
Barth explains that a program of support is essential to helping people understand how to use the scorecard and help them through any apparent gaps. Creating the draft scorecard with specific indicators relating to WOU while separating out essential and optimal indicators makes it easier to implement it gradually across the campus.
“We know where things are going with these courses and what success looks like,” says Barth, “making it easier for the whole campus to go that route.”
Learn more about services from Every Learner’s partners