The data generated by digital learning tools has the potential to support personalized learning and help close equity gaps in higher education. However, many colleges and universities are not yet realizing the full potential of learning analytics. In response, a new Learning Analytics Strategy and Equity Toolkit from Tyton Partners and Every Learner Everywhere outlines guiding principles and strategies for developing a learning analytics approach that closes equity gaps.
Tyton Partners, a research and consulting firm part of the Every Learner Everywhere network, conducted a survey and led the development of the toolkit in collaboration with over 20 learning analytics and equity experts. Professionals from universities, accrediting agencies, and nonprofits such as the Association of Public & Land-grant Universities contributed.
The principles and strategies, according to the report, “provide an evidence-based framework for a scalable, equity-focused learning analytics adoption.” They are offered to close a gap demonstrated in the survey:
“Although many institutions have adopted learning analytics in some capacity . . . few have the required technology infrastructure and knowledge-building framework to act on student data at scale. In addition, there are few examples of wide-scale learning analytics adoption that are expressly focused on equity and eliminating race and income as predictors of student success.”
Along with the survey results and the guiding principles and strategies for using learning analytics, the report also includes three case studies and a self-assessment guide that colleges and universities can use to determine readiness to use learning analytics.
During the 2019-2020 academic year, Tyton Partners conducted a national survey to understand the current state of learning analytics and how data can be used to address inequitable outcomes across different student groups. The survey explored where learning analytics is being used and adopted by faculty at institutions, both individually and at scale. Tyton wanted to understand barriers to adoption and to identify what enables increased adoption of learning analytics.
As described in the final report, some of the possible use cases of learning analytics include:
- Help students self-regulate their learning and improve on strengths
- Personalize student learning paths
- Influence curricular decisions
- Evaluate student learning across the institution and within different student groups
Learning analytics can help instructors understand where students are struggling or identify groups of students that are performing poorly in class and adjust their teaching to address these differences.
When the data is disaggregated to view specific populations of low-income students, first-generation students, and students of color, the data can highlight equity gaps where interventions can be focused.
“Faculty and administrators need access to disaggregated data,” says Kristen Fox, Director at Tyton Partners. “If they’re not able to disaggregate across student groups, they can’t look at the difference between students of color and white students, or at differences between first-generation students and non-first-generation students.”
Fox described an example from one institution where learning analytics demonstrated a higher than average DFWI rate (the rate of D or F grades, withdrawals, or incompletes) for Black male students in an introductory psychology course. Further analysis revealed that many were getting passing grades but still consistently dropping out. The department is now testing a hypothesis that more culturally relevant course content will improve learning outcomes and course completion.
Current state of the field
Research into the current state of learning analytics revealed that most institutions are not realizing the full opportunity of learning analytics yet. While many faculty and administrators said they were aware of the potential of learning analytics or that they were using learning analytics individually, there was a large gap between awareness and strategy and between strategy and execution.
- While over 80 percent of respondents used or were personally aware of the use of student data at their institutions, only 40 percent reported there was a plan to use it for achieving equity.
- Fewer than 25 percent said their institution had clear goals for using student data.
- Few respondents were making use of disaggregated data across different student groups to address equity issues.
- There was a wide gap between the number of people who said they believe learning analytics could improve student learning and who said their institution has clear goals for adopting learning analytics. The gap was 39 percent in public four-year institutions, 40 percent in private 4-year institutions, and 33 percent in public 2-year institutions.
Strategy-to-Execution Gap for Adoption of Learning Analytics
Four guiding principles for using learning analytics
As a result of a collaboration with a group of higher-education professionals and experts, Tyton Partners have led the developed a set of four guiding principles and supporting strategies for implementing a robust learning analytics program focused on equity.
1. Explicitly set goals to achieve equity across student groups
To reduce gaps in learning outcomes across student groups, it is important to establish and communicate goals across the institution. “It’s not just about advancing outcomes for all students,” Fox says. “It’s about ensuring that learning outcomes are being achieved within and across sub-populations of students.”
The survey, however, showed that closing equity gaps was only the sixth most common use of learning analytics.
Primary Uses for Learning Analytics
2. Ensure inclusion and support for faculty, administrators and students
Faculty, students, and administrators need ongoing support to analyze and act on data. That includes professional development, data privacy policies, and devising research-based learning interventions in response to the data.
However, only 5 percent of faculty and 7 percent of administrators reported robust training opportunities about learning analytics at their institutions.
Some of the strategies recommended include creating cross-disciplinary communities and nurturing a continuous learning culture to spread and sustain best practices.
3. Establish Policies that address data ethics and privacy
Lack of familiarity and comfort with policies on key ethical issues such as consent, privacy, transparency, and responsible use of data inhibits the adoption of learning analytics. Faculty are hesitant to act on the data they have access to if they feel uncertain about these policies.
Only 42 percent of survey respondents were aware of policies regarding the use of student data in the classroom at their institutions, and 48 percent of faculty said they don’t know their institutions’ policies.
4. Improve appropriate access to student data
“Users today are leveraging data from a wide variety of systems, creating complexity in their ability to use data,” the report says. That complexity means that access to data was the third largest barrier to uptake of learning analytics by faculty and administrators.
Colleges and universities should have processes for ensuring that technology and infrastructure eases the ability of users to leverage student data.
Some of the recommended strategies include developing a centralized source of data, dashboards tailored to institutional goals, and appropriate access to both learning and demographic data.
After all, the point of learning analytics is to personalize learning and close equity gaps, and requires being able to make finer distinctions with the data. “For all students to be successful, you can’t just teach to the average student,” Fox says. “Different students are mastering concepts at different rates of speed and coming in with different understanding, different experiences.”
The Learning Analytics Strategy Toolkit includes three case studies of innovative learning analytics in action. They show, among other things, how:
- Rio Salado College in Arizona developed a Dynamic Assessment Data Display (DADD) that enables faculty to identify performance gaps in individual students or student groups.
- The University of Michigan’s MyLA portal helps students track their engagement within a course.
- The University of Texas at Arlington is searching for variables such as a sense of belonging and help-seeking behavior that may explain gaps in success for students of color in introductory algebra courses.
Also included in the toolkit are self-assessment instruments that can help colleges and universities find their own innovative uses for learning analytics. That may involve working groups that routinely evaluate an institution’s goals, finding the existing power users of learning analytics in an institution, or creating training and professional development to support equity efforts in digital learning.
“The team has designed the self-assessments to enable you to assess where you are in terms of the four guiding principles,” Fox says. “Then start where you already have momentum. Look at what systems, activities, processes, and courseware you already have in place, and start there to build on existing strengths.”