Blog

Interview with Colleen Carmean, Founder, Ethical Analytics Group and former Associate Vice Chancellor for Academic Innovation, University of Washington-Tacoma

As a recent article listed in our Spring 2020 issue of Trends in Higher Education points out, most colleges and universities excel at collecting data, but struggle when it comes to effectively using that data to improve teaching and learning or business operations. According to the article, the key to success is a robust cross-institutional strategy that engages multiple stakeholders.

But what does that look like in practice? We reached out to Colleen Carmean, former associate vice chancellor for academic innovation, University of Washington-Tacoma, for her insights. Colleen recently founded the Ethical Analytics Group, which works with higher education institutions to improve student and campus success via thoughtful use of data.

When you were at the University of Washington-Tacoma, you were heavily involved in improving the use of data across the institution. What was the impetus for that effort?

The “obligation of knowing” where students struggle rests so heavily on higher education right now. As a new, diverse population takes on significant debt and makes extraordinary sacrifices to earn a bachelor’s degree, we’ve learned that analytics can create change that helps those students to the finish line. In 2015, the University of Washington took on learner analytics as a tri-campus solution (with Civitas Learning) that allowed us to integrate our data across so many previous silos, to examine hidden barriers, and to discover real-time predictions regarding populations that were quietly struggling. The results were surprising.

What sort of data did you uncover?

For instance, we discovered that on our campus, young men under the age of 21 are seven percent more likely than their peers to return in Fall if they’ve taken one or more online courses the previous year. We discovered that students across all demographics and academic performance levels are six percent more likely to return in Fall when they sign up for courses at least two weeks before the beginning of a term. And, we discovered that students in the “murky middle”—those who qualify for Pell aid, but not enough Pell aid—are by far our most vulnerable population.

How did having this kind of “real-time” data improve the effectiveness of data analytics at the University of Washington?

The predictive analytics we worked on forced us to recognize that the institution, rather than the student, needs to take responsibility for a system that lets students down. Students in college today aren’t the same as they were in their parents’ or grandparents’ generations, and yet in many ways we continue to operate as if they need the same courses and services. Analytics gives us evidence that requires new attention. It’s not the “dead data” of mandatory reporting regarding the sorting and tallying of students, but real-time success predictions based on the students currently on our campus.

What are some of the unexpected benefits of improving data analytics?

A significant benefit seems to be that, in a time of financial struggle for so many campuses, data analytics—done well—helps us move budgets set with good intentions (but scant evidence) toward initiatives where we are able to see the needle move on student success. When leadership can embrace sometimes surprising data and create a 21st-century campus that supports the students here today, we’re doing the right thing and preparing for the disruptive change we see happening in higher education.

What are some of the greatest challenges in developing effective data analytics?

Crucially, administrators have to recognize that analytics is a new paradigm, and it doesn’t necessarily fit within legacy, siloed structures. It’s not institutional reporting, it’s not simply implementing a technology solution, and it’s truly not worth the effort if an institution can’t innovate to support the diversity and challenges of the students now on our campuses. Seriously, administrators would do well to assign the Baer and Carmean (SCUP, 2019) Analytics Handbook to an impact team—so many case studies, ethical issues, and student concerns are explored by thinkers at the intersection of opportunities realized and mistakes made by campuses blazing the trail.

Any final thoughts?

Failing studying what higher education has learned in the past five years? Bringing in expert outside help at any point in the design and implementation of analytics work could be a very insightful decision! Most of us don’t realize that the hard work of configuring our data and installing an analytics tool turns out to be the easy part. The “Where’s Waldo” work of designing ethical and appropriate responses to new insights is challenging and requires collaboration across institutional spaces, data barriers, budgets, and skills. It demands commitment from leadership to thrive, and it helps to learn from others’ mistakes in creating the framework that can move newfound evidence to meaningful impact.