Massive Open Online Courses (MOOCs) have the potential to democratize education by improving access. Although retention and completion rates for free users have not been promising, these statistics are much brighter for paid users who receive a certificate upon completing the course.

A new study co-authored by Ali Goli, an assistant professor of marketing at the University of Washington Foster School of Business, aims to understand how paying for the certificate option can improve engagement with course content.

How can payments affect students in online courses? There are a few possible ways, but one challenge is that payments can act as a screening mechanism — users with higher confidence in their ability to obtain the certificate are more likely to pay for the certificate. This means that simply comparing the learning outcomes of people who pay vs. don’t pay is unlikely to be helpful, because students who choose to pay might be much more serious about their education that students who don’t. “The fact that users self select into the payment plan makes it challenging to do a fair comparison of education outcomes across free and paid users,” says Goli.

To overcome this obstacle, Goli and co-authors Pradeep Chintagunta of the University of Chicago and S. Sriram from the University of Michigan obtained detailed data from courses offered through Coursera by a large national public university. The data records all user interactions with the website as they progress in the course. This creates a unique opportunity to observe students’ output (grades) as well as their input (time investment). This feature enables the authors to understand if the gap between free and paid users is explained by differences in ability or differences in motivation levels. The basic idea is simple: if students are spending equal amounts of time learning the material but get very different grades, that can be attributed to a difference in ability. On the other hand, if students are spending very different amounts of time learning the material, then that suggests a difference in motivation.

“To our surprise, we found that ability explains very little of the gap between free and paid users, and the gap is mostly explained by either intrinsic or extrinsic motivation,” says Goli. “The question then becomes to what extent can payment impact students’ extrinsic motivation level?”

The authors consider two explanations for why students might be better-motivated if they had to pay: the sunk cost fallacy, and a direct certificate effect.

The sunk cost fallacy is a common psychological principle that explains why people insist on continuing with behaviors not because they’re enjoying them, but because they don’t want to “waste” the money or effort they spent previously. In this online education context, the sunk cost effect would imply that paid users might demonstrate higher engagement merely because they made the payment; i.e., students would try harder because they don’t want their payment to be “wasted”. The authors show that this effect holds true: in the weeks following the payment, students are more likely to remain engaged with the platform. However, while the sunk cost effect increases engagement levels by about 20%, the effect is short-lived and vanishes quickly after the payment. According to Goli, the transient nature of the sunk cost effect may indicate that recurring payments or increasing the salience of the payment could improve user engagement level in the long run.

Goli and his co-authors then investigate if offering a certificate can lead to an increase in engagement. “We wanted to know to what extent do students have their eyes on the prize — that is, how much do they care about fulfilling the certificate requirements?” Goli says. Using the detailed nature of the data, the authors can compare how the engagement levels of free and paid users change as they cross the minimum grade required to pass the course. They find that the certificate can improve engagement by about 10 percent. “These findings indicate that there is a lot of room for improving education outcomes by optimizing the way certificates are offered and payments are processed in online courses. With the recent surge in usage of online education platforms caused by the COVID-19 pandemic, MOOCs that figure out how to retain students will be the ones that can sustain their growth in the future,” Goli concludes.

For more details, read the full research paper here.

Authors:

• Ali Goli (Foster School of Business, University of Washington)
• Pradeep K. Chintagunta (Booth School of Business, University of Chicago)
• S. Sriram (Ross School of Business, University of Michigan)