How One University Uses Adaptive Learning Dashboard Data to Revise Math Courses

When the Department of Mathematics and Statistics at Cleveland State University was considering the potential for digital learning technologies to improve pass rates in its introductory Mathematics courses, they didn’t have to look far. By digging into the dashboard data in the adaptive learning courseware they had already been using for years, they were able to successfully launch a pilot that showed almost immediate results.

In fall 2019, mathematics faculty at Cleveland State University began an effort to use dashboard data from its adaptive learning software — ALEKS —  to make mid-semester revisions in assignments, quizzes, and tests. In doing so, the department achieved its goal of normalizing pass rates at above 75 percent for several sections of precalculus courses.

And they expect to help even more students succeed by using data from adaptive learning software to identify where courses can be refined in future semesters.

A “mini-run” to test the adaptive learning dashboard data

Because the Cleveland State University Mathematics and Statistics Department was already thinking about how to get more out of its adaptive learning software, when the Association of Public & Land-grant Universities (APLU) offered support and expertise, instructors were ready to engage in a pilot. (They are also one of the Lighthouse institutions that began working with Every Learner Everywhere and its network partners in 2019.)

They expanded their previous experience with ALEKS, an artificial intelligence, web-based program and learning system that supplies course content for several introductory college courses including math, chemistry, and business. Before Spring 2019, the Department of Mathematics and Statistics rarely used the dashboard data from ALEKS to identify areas where individual sections were struggling during the semester.

As part of preparing for its adaptive learning pilot project, math faculty used the last four weeks of the Spring 2019 semester to conduct a “mini-run” on three classes: MTH 95-Applied Algebra, MTH 167-Precalculus I (College Algebra), and MTH 168-Precalculus II (Trigonometry). Instructors for those courses practiced pulling and reviewing dashboard data from the adaptive courseware and revising assignments to make sure the process would work.

Juan Amador, Interim Director, Operation STEM, Louis Stokes Alliances for Minority Participation, Department of Mathematics and Statistics, at Cleveland State University says, “Using the ALEKS dashboard was just amazing.” The way it revealed topics that students were struggling with and differences between sections or student groups convinced the department they were ready to fully implement adaptive learning for the Fall 2019 semester.

Related reading: Disaggregating Learning Data to Support Equity Efforts: Resources for College and University Instructors

Applying dashboard data to make mid-stream adjustments

Before using ALEKS data, the Precalculus Coordinator at Cleveland State University would construct homework assignments and quizzes for the semester by randomly selecting problems from the software’s question bank.

In fall 2019, however, the Department of Mathematics and Statistics began selecting homework problems weekly as data showed where students were struggling. The adaptive learning pilot project focused on three courses: MTH 165-Intensive Precalculus I (a newly implemented co-requisite course consisting of all topics learned in MTH 95 and MTH 167), MTH 167, and MTH 168.

Instructors used the adaptive learning dashboard data from those courses to evaluate learning within different course sections and among specific student groups to influence curricular decisions. During the Fall 2019 semester, for all three courses, the following steps were carried out each week:

• Sunday through Thursday, students worked in ALEKS on a learning objective and practice problems.
• Friday morning, the Precalculus Coordinator used dashboard data to identify four of the “most missed” homework problems of each course section.
• The coordinator then combined those homework problems with two other review problems covering concepts necessary to master the topics students were struggling with. This six-problem homework assignment was posted Friday morning in ALEKS. (This included both online and written work.)
• Monday morning, instructors collected the written homework for review and identified two of the six written homework problems as “most missed” using dashboard data. The two “most missed” problems were presented on a pencil and paper quiz every Wednesday.

Therefore, each week during the semester, six sections of Precalculus courses, totaling 160 students, cemented math concepts through the use of adaptive learning: analyzing dashboard data and using it to provide additional reinforcement and to adjust homework and quizzes to respond to “most missed” problems.

Instructors began to respond quickly to adjust their assignments and quizzes that narrowed those learning gaps. For example, “Let’s say our current objective was focused on graphing sine and cosine functions,” Amador says. “Before class, the instructor would access the ALEKS data and notice the poor completion rate and the number of failed attempts for certain topics.”

Knowing this, instructors would provide additional instruction time on the topic in the next class meeting. This often led to improved completion rates in the next objectives, homework problem sets and quizzes — and ultimately to increased completion rates of the course at the end of the semester.

Results from the adaptive learning pilot

Cleveland State University found the results of using dashboard data from adaptive learning software to inform mid-semester revisions encouraging. Although the COVID-19 emergency impacted pass rates for courses in the Spring 2020 semester, the Department of Mathematics and Statistics did see positive pass rates for all three Precalculus courses in the Fall 2019 semester.

• The newly implemented MTH 165 course measured pass rates of 81 percent at the end of the Fall 2019 semester.
• The new MTH 167-Precalculus I: College Algebra and MTH 168- Precalculus II: Trigonometry measured pass rates of 80 percent at the end of the Fall 2019 semester.

Amador says courseware data shows finite differences among the different sections. He says, “Historically, there were always topics we know were challenging, but with dashboard data, you could see more details.”

Not only can instructors drill down to specific section levels to tailor and adjust assignments, quizzes, and tests, but they can drill down to the student level. They can identify a struggling student who needs additional support. And they’re using that data to reach out and refer them to resources they need to succeed.

Students like using the adaptive courseware because it’s more interactive. In ALEKS, when a student is doing homework and doesn’t understand a concept, they can ask the software to show practice activities that reinforce the concept.

Students also appreciate the initial assessment they take at the beginning of the course. For example, if mastery of 200 topics is necessary to succeed in Math 167 – Precalculus I – College Algebra course and an individual student has previously mastered 50 of those topics according to the initial assessment, the software will prioritize the 150 remaining topics and tailor the practice activities for that individual.

(For an updated study of the results of the adaptive learning pilots, see the report Teaching Practices of Faculty Adopting Adaptive Courseware.)

Using dashboard data for continuous improvement

Through testing, refinement, and evaluation, math faculty at Cleveland State University continue to use the data dashboard across the multiple sections of MTH 165 – Intensive Precalculus, MTH 167 Precalculus I, and MTH 168 Precalculus II.

Next they plan to make informed course-level revisions using the data from the adaptive learning software. For example, the data in MTH 167 – Precalculus I suggests that students will benefit from a small number of additional prerequisite remedial topics, such as adding fractions or converting miles per hour to inches per second. Amador says these topics will aid student understanding of challenging topics later in the semester that the ALEKS data is showing as stumbling blocks.

Then, at the end of the Fall 2020 semester, Amador expects the math faculty at Cleveland State University to go through the same process of evaluating the results, refining their use of the data, and making additional revisions to their courses to improve student learning and retention of concepts.

‍Originally published October 2020. Updated August 2021 with additional information and references.