Expanding Our Commitment to Continuous Improvement: Partnerships and Community

By David Wiley, Chief Academic Officer

 

Learning Materials Are Hypotheses

All learning materials are collections of hypotheses. They represent a series of guesses about what will effectively support student learning. Should students be reading text? Watching videos? Playing with simulations? Arguing with peers? Creating and sharing artifacts? Practicing and reflecting on associated feedback? And exactly what should they read, or watch, or play with, or argue about, or create, or share, or practice? And how much should they do each of those things? And what order should they do them in? And who should they do them with? And which topics are even candidates for inclusion in these materials, and which definitely belong in the materials for a different class? Answers to each of these questions, and thousands more, are hypothesized in each and every set of learning materials.

Initial hypotheses are seldom – if ever – correct. Hypotheses need testing and refinement. And they need to go through that testing-refinement loop many, many times.

As hypotheses, learning materials are no different – they need iterative improvement, too. However, educators have historically failed to think about learning materials as hypotheses. That is to say that we don’t think of learning materials as things that should be tested and refined over time. (Yes, new editions of learning materials are released frequently. But everyone – from students, to faculty, to the creators of the learning materials themselves – is keenly aware of the fact that the primary motivation behind the creation of most new editions is undercutting the market for used materials, and not anything related to learning.) Part of the reason we’ve never thought of learning materials as hypotheses that could be tested and refined is that historically we’ve been legally prohibited from refining them.

Open educational resources (OER) are learning materials that are licensed in a way that provides people with permission to engage in activities that traditional copyright prohibits. Among the activities permitted by open licenses are revising and remixing. In other words, refining learning materials is possible in a world of OER. And when refine-able OER are combined with established and emerging ways of conducting educational research, we can finally begin thinking about learning materials as hypotheses that can be empirically tested and refined over cycles of continuous improvement. And we can make open learning materials more effective over time.

Building Partnerships to Improve Learning

Because there’s been a relative lack of thinking about learning materials as hypotheses in the past, there’s a relative lack of methodologies and tools for conducting the kind of educational research underlying the test-refine cycle for learning materials. That’s why I’m so excited about the recently announced collaboration between the Simon Initiative at Carnegie Mellon University and Lumen Learning on open source tools for data-driven continuous improvement and related research. By collaborating on tools and approaches, and making these openly available, we hope to jump-start a more rigorous, empirically-minded approach to the creation and continuous improvement of OER.

You can learn more about this new partnership here.

Empowering Our Community to Improve Learning

I’m also really excited about the work Lumen has done to enable our community to participate in this process. There’s a great literature I’ve really enjoyed reading over the last several months, including topics ranging from Benkler’s take on why the collaborative production of open learning materials is so difficult to a dissertation on helping people overcome the struggles associated with making their first contributions to open source software projects. (If you’re interested in these topics, the best things you can do are (1) hang out on Mara Averick’s incredible twitter feed and (2) read everything Igor Steinmacher has written.)

We spent several months synthesizing this literature into actionable processes. Then we started doing data-driven continuous improvement in small workshop settings with faculty (which is incredibly fun, btw). For the workshops, Lumen has done all the data analysis, visualization, and other preparation (getting assessment items and content that need improvement into Google Docs so they can be improved collaboratively) ahead of time. We then kick off the workshops by briefly explaining the work we’ve done beforehand, asking people to break into small groups (2 to 3 people) per learning outcome that needs help, and then supporting them through the process of digging in, generating their own hypotheses about why students might be struggling, and either making needed improvements or describing them in enough detail that we can make them later.

Now we’re opening this continuous improvement process to our whole community at https://lumenlearning.com/improvinglearning/. When all the prep work is done for you, I think you’ll be surprised at how big a difference you can make in 30 minutes.

While I realize I’ve used the word “excited” a lot above, I’m particularly excited about the Tenure and Promotion Documentation we’re able to provide for people who participate in the process. If Lumen takes on all the preparation and coordination costs, and if we can provide meaningful incentives for faculty to participate in the continuous improvement process, we might actually be able to create a sustainable model for continuous improvement of OER that invites and encourages contributions from a wide range of faculty voices.

Finally, and perhaps the most exciting thing of all: the list of learning outcomes that need help provided for each of the courses in the Community-based Continuous Improvement site can be an incredibly useful tool for faculty who are looking to integrate some OER-enabled pedagogy into their classes. Because renewable assignments can be so much work for both students and faculty, faculty may choose to use only one or two during a term. This leads to the question “which topic’s OER should I have students revise and remix?”. Knowing where students have struggled most can help faculty decide where to invite students to take the deeper dive associated with OER-enabled pedagogy. And these assignments can also provide opportunities for students who do exceptional work to submit their revisions and remixes for inclusion in the master course.

Learn More: Webinar Recording

In this webinar recording, from Thursday, May 16, 2019, I share more information about how this continuous improvement process works and how we hope to involve educators in making Lumen course materials better and better at supporting learning.

 

The header photo for this post is by Jon Tyson on Unsplash.