By Kim Thanos, CEO
January is a natural time for reflection: How are we doing? What impact are we making? What can we do differently and better in the new year?
And what does impact mean, anyway?
Impact = Number of students?
When the curtain closed on 2017, we tallied just over 170,000 students using Lumen course materials at the institutions we support. We’re pleased to see this number of Lumen-supported students more than double every year. Millions more use Lumen’s openly-licensed content through our free public catalog (11 million users during the Fall term alone). That’s one way of measuring impact.
Impact = Textbook cost savings?
Another way to measure impact, common in the open education community, is in terms of textbook cost savings: How much did students save by using our open educational resource (OER) materials rather than expensive publishers’ textbooks?
A lot has been written about how much students actually save using OER at a time when relatively few students buy all their assigned textbooks new. Amazon, the used book and textbook rental markets all complicate this picture further. But if we calculate the difference between Lumen course support fees (standard pricing $10-25) and the average cost of a textbook ($100), we estimate Lumen-supported students saved $15 million in 2017.
That’s a tidy sum for one year. It goes far towards helping individual students buy groceries, pay for rent, childcare and transportation, and reinvest in achieving their educational goals.
Impact = Successful students?
For many educators, the most fulfilling part of the job isn’t about dollars or numbers at all. It’s about working with individual students and seeing the transformative impact of education in their lives.
Lumen’s mission is to enable unprecedented learning for all students.
Initially we worked towards this mission by replacing expensive textbooks with OER. We found that opening access and removing cost as an obstacle helped–up to a point. Research found students performed as well or better in Lumen-supported OER courses, compared to courses using traditional textbooks. But we also came to the realization that replacing static publisher materials with static open materials doesn’t lead to unprecedented learning.
Since 2015, our focus has shifted towards improving the learning experience around OER.
- Students learn better by doing, and so we’re building interactivity with frequent feedback into open content.
- At-risk students often struggle with whether they belong in college, and so we’ve created time-saving messaging tools that provide guidance, encouragement, and deeper faculty-student connections.
- Students become better lifelong learners as they develop metacognitive muscles, and so we’re using learning data to help them reflect on their progress and make good choices about how to develop mastery.
- By analyzing how students perform using Lumen course materials, we can pinpoint problem areas and iteratively improve content, assessments, and learning tools to strengthen learning.
Evidence of Impact
In the end, impact is a combination of these factors. In 2017, we crunched a lot of data figuring out how we might begin to quantify our impact on student success with Waymaker courses, where all these investments in OER and learning design come together.
Here’s a powerful finding: At Cerritos College during 2016, 73% of students enrolled on day one passed the course successfully in sections using Waymaker. This compares to just 55% in non-Waymaker sections. Driving this result was a huge differential in drop rates. Just 10% of students in Waymaker sections dropped the course, compared to 34% in non-Waymaker courses. The effective learning experience in Waymaker courses helped to keep more students enrolled and progressing successfully through the course.
In another analysis of student outcomes, we found that using Waymaker course materials erased the performance gap that typically exists between Pell-eligible and non-Pell eligible students. Pell eligibility is often a marker for at-risk students, and it is difficult to find learning interventions that make a measurable difference improving learning for these students on a large scale.
Looking Ahead in 2018
What we found in 2017 about our impact on learning only scratches the surface of what’s possible. We have definitely found that better content and learning design make a difference. We have validated yet again that the faculty role is key to a successful learning experience. We continue to learn so much from the dedicated, creative faculty members who are always seeking better results for their students. We have begun to understand more about the right supports for students and faculty. We are constantly reminded that there is still much to learn about effective learning.
This year we’ll have many more data points and opportunities to discover, understand and demonstrate learning impact. Looking forward to it.