Every Learner Everywhere
Association of Public and Land-grant Universities
Digital Promise

Assessing Adaptive Learning Readiness at Your Institution? Look for These Criteria.

When a college or university considers using adaptive learning technology in its classrooms, a basic question it must confront is if it is ready. Are the culture, capacity, and resources in place to succeed?

Every Learner Everywhere works with its partner institutions to assess their readiness to implement adaptive learning. In 2019, for example, they worked with 10 “Lighthouse” institutions that piloted the use of adaptive learning in over 40 entry-level courses. (Lighthouse institutions are the first colleges and universities served by the network and that produced insight and data about equity-centered implementations of digital learning technologies.)

The pilot of these courses was preceded by a needs assessment to understand the adaptive learning readiness and capacity at those colleges and universities.. Three of the network partners, Achieving the Dream, the Association of Public and Land-grant Universities (APLU), and Digital Promise crafted the needs assessment and provided technical assistance.

Megan Tesene, Associate Director of the Personalized Learning Consortium at APLU, says the consulting partners met with cross-functional project teams that included faculty, instructional support staff, and other stakeholders. “Each of us had guiding questions about things we wanted to look at on campus,” she says.

“Who’s interested in this work, who’s involved in it, and what are the institutional capacities? We did a lot of questioning and looked at the breadth of the different resources they had on campus. We also looked at the ability and commitment to integrate existing resources into supporting the initiative.”

Below are seven of the criteria that emerged and that institutions should consider using to evaluate adaptive learning readiness.

1. Alignment 

Success will depend on communication, collaboration, and coordination across multiple groups of stakeholders at a college or university.

“There needs to be an understanding at the administrative level that faculty need space and time to do this work,” Tesene says. “It’s not just plugging a tool in. It must be integrated into the pedagogy and into the course design.”

Tesene recommends coordinating with the campus teaching and learning centers as well as with IT to make sure the courseware integrates into the learning management system (LMS).

Likewise, teaching and learning centers must be available to support course and curriculum design and to ensure the adaptive elements align with course objectives. “Instructional designers are excellent partners that are often underutilized,” Tesene says.

Barbara Means, Executive Director, Learning Sciences Research, at Digital Promise, says, “Ideally, alignment is institution wide and reflected in the college’s goals and strategic plan.”

Program profile — A Team-Based Approach to Redesigning and Aligning a Gateway Course

2. Leader support 

Successful projects need champions who stay active.

  • They need to have relationships with colleagues in other parts of the university.
  • They need to galvanize support for redesigning courses.
  • They need to advocate for the advantages of adaptive learning.
  • They need to advocate for faculty and students during the process.

Means recommends looking at whether that champion “is at a high-enough level at the institution to garner resources for the effort.”

‍Program profile — Championing Adaptive Learning Courseware: One Department Chair’s Story

3. Capacity of the project team

In most institutions, a small group of people will lead the effort to implement adaptive learning. Is that team prepared for success?

Means describes several factors to consider:

  • Can they teach other staff how to redesign courses?
  • Do they have knowledge of specific adaptive courseware?
  • Are they knowledgeable about best practices?
  • What’s their capacity to gather and examine data?
  • Is the team multi-faceted, with the variety of expertise that will be needed?

Program profile — Teamwork to Seed Change: How Foreign Language Faculty at UCF Are Improving Learning Outcomes

4. Capacity to redesign courses at scale

One major benefit of adaptive learning technology is the potential to close equity gaps by improving learning outcomes for large numbers of Black, Latino, Indigenous, and poverty-affected students. An interesting experiment with a couple of instructors working in isolation will have a limited impact.

To scale up adaptive learning across an institution, it needs to have processes in place for reviewing outcomes data, learning from it, and using it. Tesene says embedding continuous improvement into the design of adaptive learning is vital to success. Faculty need to know what they want to accomplish, measure progress, and refine courses.

“Is there an instructional design team on hand that understands what adaptive learning is and can offer help on building that out?” she asks. “Does the institution have the infrastructure and technological capacity to integrate adaptive courseware into their LMS? Do they have a plan for which tool they want to use and how they plan to use it?”

‍Resource — Improving Critical Courses Using Digital Learning & Evidence-based Pedagogy

5. Capacity for using data

The data generated by adaptive learning can support continuous improvement of teaching and learning at the course and program levels. It can help identify individual students whose academic performance is lagging, and it can help identify where a section or multiple sections of a course lose large numbers of students.

The Every Learner Everywhere network partners help institutions assess their adaptive learning readiness and their capacity to use data by asking questions like:

  • How common is it to analyze student learning and course success rates?
  • Do department chairs know the student success rates for their gateway courses?
  • Do departments have access to data from the institutional research office?
  • Is evaluation expertise readily available?
  • Is there support for using data from digital learning systems formatively to improve instruction?

Tesene says a simple reality check is to ask, “Which experts do you already have who are knowledgeable and understand how to interpret data? And how easy is it for faculty to just log in and see what their success rates are?”

An institution should also ask about its data culture — how data is shared across campus and if there are open and transparent conversations about data and how it can inform teaching and learning.

Related reading — Disaggregating Learning Data to Support Equity Efforts: Resources for College and University Instructors

6. Capacity to support faculty development 

If the data reveals opportunities for improvement, that means implementing adaptive learning successfully will require ongoing professional development by faculty — both permanent and adjunct.

Project teams should evaluate if:

  • enough professional learning opportunities (such as workshops, mentorships, communities of practice, and training sessions) are available for both adjuncts and regular faculty and instructors;
  • incentives (e.g., mini-grants, release time) are available to support faculty and staff involvement in redesigning courses to incorporate adaptive courseware; and
  • these supports attend to issues around quality implementation of adaptive learning.

First look at campus culture, Means says: “Are there professional learning opportunities there, and are faculty actually working with staff from the teaching and learning center?”

Tesene advises institutions to look at compensation for the extra labor and time needed to redesign courses for adaptive learning technology.

“Does the campus offer recognition in the form of stipends or awards?” she asks. “It takes time, effort, and energy to do this well. Faculty require institutional supports as well as recognition for their commitment to teaching and learning.”

‍Related reading — Models for Effective Faculty Development While Implementing Adaptive Learning

7. Capacity for inclusive learning

The mission of Every Learner Everywhere is to help institutions use new technology to improve outcomes for Black, Latino, Indigenous, poverty-affected, and first-generation students. 

The network turned to adaptive learning because of the potential to close achievement gaps in courses with students who have a wide range of readiness. (The pilot particularly targeted gateway courses. These introductory courses in a program of study often have hundreds of students in a lecture-style format, and they are often a barrier to progress for minoritized and poverty-affected students.)

Institutions implementing adaptive learning must look at ways to support every student regardless of where they are starting. In addition to supporting students with diverse levels of prior learning, courses should strive to be culturally responsive.

This means course content and instructional methods are designed to engage students in ways that are meaningful to them. 

The adaptive learning readiness assessment used with pilot institutions asked:

  • Across the institution, to what extent are departments and faculty encouraged and supported in efforts to improve teaching and learning for minoritized and poverty-affected students?
  • Do department chairs and faculty know the success rates for their foundational courses for key student subgroups?
  • Are faculty and staff being guided by research on pedagogical approaches that reduce equity gaps?

Related reading — Curricula That Account For All Students: A Look at Culturally Responsive Teaching in Higher Ed

Closing the gap on your adaptive learning readiness

Evaluating whether your college or university is ready to implement adaptive learning and other digital learning technologies is a process of identifying your strengths, weaknesses, and where the gaps are. 

The good news is many savvy people are already doing this work well at other institutions and are happy to share their knowledge and experience. For example, during the pilot project, peers from Lighthouse institutions used an online tool to share their experience, advice, and expertise with each other.

Tesene says that support and cross-pollination “is absolutely happening all the time. Bringing peers together has been really compelling because they speak the same language and can relate to and advise one another based on their experiences.”

Both Means and Tesene believe every college or university is ready to try adaptive learning to some degree. No institution is topped up with every capacity in every area from day one. Institutions often start with a single class or program.

But to have a broader impact on student success, you have to prepare to implement what you learn at scale. That will mean developing the culture, resources, and leadership described above.

In the meantime, start somewhere: Gather evidence about what’s working, and share that with others on campus. With each iteration, you’ll learn more, become more comfortable with adaptive learning, attract supporters and champions, and deliver a positive impact on student outcomes.

Download the Adaptive Courseware Implementation Guide

Originally published April 2020. Updated August 2021 with additional information and references.