How Can Universities Change Data Culture? Lessons from NMSU

Like many institutions, New Mexico State University (NMSU) has what Patrick Turner, Associate Vice President of Student Academic Success, describes as a “sporadic” data culture.

“Institutions are data rich, but information poor,” says Turner. “We collect all of this data on students coming through our doors — race, gender, family income, grades in class — but we only use about 10 percent of the data we collect.”

In his experience, peers in the institutional research office are overworked both from internal needs and from the demands of regulators and accreditors. That makes it hard for faculty and administrators to get useful data in a timely way.

As a consequence, a “shadow” IT infrastructure springs up as individual programs implement their own solutions, which can lead to unverified data from different sources.

“I started hearing conversations about people’s frustration — in the Deans’ Council, in the Provost Council, on the retention committee,” Turner says. “I started realizing it is a huge issue.”

None of these are challenges unique to NMSU which is why the Association of Public and Land-grant Universities (APLU) offers Building an Academic Data Culture to Support Student Success. NMSU had previously worked with APLU on other professional development programs, so when Turner was offered the opportunity to take part in this in fall 2023, he jumped at the chance.

Making meaningful connections

Building an Academic Data Culture to Support Student Success is one of a menu of professional development services provided by Every Learner Everywhere with the partners in its network.

The Data Culture service is a three- to five-month series of workshops and coaching support. It helps colleges examine their institutional data, build academic data literacy skills, and use data to address a specific student success issue at the institution.

The service is customized for the particular needs of a campus or cohort. In this case, between September and November of 2023, APLU worked with a group from NMSU’s institutional research office, student affairs, and academic advising. Their goals were to better understand its data culture and to build skills through a specific data project.

The program consisted of five workshops and four coaching sessions during which the NMSU team worked with a facilitator from APLU and a peer coach from Bowling Green State University with expertise in data-driven decision making.

The scope of the challenge

One example of sporadic data culture is Turner’s own experience each semester  administering the 30 courses in the university’s first-year experience program.

“Each course has over 50 students and I want to collect data in a timely manner,” says Turner. “But every semester, I have to submit this long request, which takes about 8 to 12 weeks.”

This long process makes it difficult to analyze the data and quickly respond to student needs.

Another challenge is the proliferation of tools. During this engagement with APLU, the NMSU team discovered nearly 20 data platforms on campus not managed by institutional research or IT. Individual departments had purchased commercial data platforms, hired their own data collection personnel, or built their own tools. This can lead to inconsistencies, which can be problematic when making data-driven decisions.”

“Everybody’s data was not aligned,” says Turner. “It really starts making you ask, ‘Okay, this is the retention rate you have, but I have a couple of different numbers. Which one is the true number?’”

Applying data to gateway courses

NMSU hasn’t solved all these challenges yet but are moving in that direction in recent months. The capstone of the eight-week program with APLU was a project to investigate a major student challenge on campus by using data.

The NMSU team chose to explore a gateway trigonometry course, which historically had a high DFW rate. In the previous year NMSU had taken part in another APLU program, which helped redesign the course and offered teaching interventions to the instructors. Turner wanted to better understand that course, see what was working, and discover what indicators led to students failing or withdrawing.

“We had already collected data from the students, we had already collected data from the course, we had already collected data from the faculty members, but we still wanted this team to dig deep into it,” Turner says.

The team created a dashboard to track data collected on this course, allowing them to break down five years’ worth of data in different ways: by semester, section, and student demographics. Similar dashboards were created for two other gateway math courses with high DFW rates.

The data told the working group a clear story: poor mid-term grades predicted which students were likely to fail the course or drop out.

The next steps for the team are to examine student engagement data during the first weeks of class, and ask students what challenges they faced in completing the course. Based on those results, the team is identifying interventions that will help improve student performance.

One outcome of the work so far is that Turner found the campus had an existing data governance board that was inactive. After the institution’s current presidential search, he would like to reactivate it and begin consolidating data collection.

In the meantime, Turner wants to work on what he calls the “low-hanging fruit,” expanding the interventions to the APLU capstone project to gateway courses beyond trigonometry.

“These are things we can do without spending a lot of money now,” he says. “Let’s start with refining what we have in place now. Because I believe we’re doing great work. It just needs to be reimagined.”

Changing data culture in higher education

To move from data rich and information poor, universities will have to use the other 90 percent of data. That’s where big opportunities are available to colleges and universities, and neglecting it isn’t really an option anymore anyway, Turner says.

“In order to really support student success and close the equity gap, you’ve got to look at the data and you’ve got to look at it responsibly,” he explains.

“We still have too many students falling through the cracks, but we can no longer blame the students. It starts with looking at the data, facing what the data is showing, and making intentional efforts to create interventions.”