Does online learning work? The evidence

The US Department of Education meta-analysis

A report prepared for the US Department of Education searched research from 1996 to July 2008 and found more than a thousand empirical studies of online learning. After screening for a face-to-face comparison, measured learning outcomes, a rigorous design and enough data for an effect size, 51 independent effects remained, 44 of them from older learners. On average, students in online conditions performed better than those in face-to-face instruction, with an average effect size of +0.24, statistically significant at the 1 percent level.^[1]

Blended instruction showed a larger advantage (+0.35) than purely online instruction (+0.14) over face-to-face teaching. Studies in which online learners spent more time on task averaged +0.46, against +0.19 where face-to-face learners spent as much time or more. For graduate students and professionals the mean was +0.17. The authors cautioned that conditions often differed in curriculum, pedagogy and learning time, so the advantage may not be rooted in the medium. Only five published studies involved school-age learners.^[1]

Later and wider syntheses

Means and colleagues (2013) published a related meta-analysis of 50 effects from 45 studies. Learners were about evenly split between students in college or earlier education and those in graduate or professional training, with average ages from 13 to 44. Students in online conditions performed modestly better than those in face-to-face instruction. The advantage was significant for blended learning compared with face-to-face teaching but not for purely online compared with face-to-face. Blended studies tended to include extra time, resources and learner interaction, which confounds the comparison.^[2]

Cook and colleagues (2008) pooled 201 studies of Internet-based instruction for health professionals. Against no intervention, pooled effect sizes were 1.00 for knowledge, 0.85 for skills and 0.82 for learner behaviours and patient effects. Against non-Internet formats, they were 0.12 for knowledge, with a confidence interval just above zero, and not statistically significant for satisfaction, skills or behaviours. Heterogeneity was large (I squared of 79 percent or more), and the authors concluded effectiveness appears similar to traditional methods.^[3]

The Means review set inclusion rules: it counted experimental studies with random assignment and quasi-experiments that controlled statistically for pre-existing group differences, and most of the instruction it covered lasted more than a month. The authors called for more research on different blended models and for experimental tests of design principles for different kinds of learners.^[2]

Design features: interaction and blending

Bernard and colleagues (2009) compared distance education courses that differed in their interaction features, such as student to student, student to teacher and student to content. Across 74 studies, the weighted average effect on achievement was 0.38, and results varied. The authors found that stronger interaction was associated with higher achievement, and interpreted this as increased cognitive engagement. The reading about engagement is an interpretation presented in the paper, not a direct measurement.^[4]

A systematic review of school-based research by Topping and colleagues (2022) covered 1,355 studies, of which online learning made up 7 percent and blended learning 13 percent. Digital technology beat regular instruction in 85 percent of studies, matched it in 8 percent and did worse in 3 percent. Blended learning was considerably better than online learning, and self-efficacy was strongly related to outcomes. The review counts studies rather than pooling effect sizes and concerns schools, so it should be read as supporting evidence only.^[5]

The Topping review adds some detail on where effects were found. Effects were largest in primary and early-years settings, and computer-assisted instruction showed the strongest effects of the approaches covered. The authors suggest that the main benefit of digital technology may lie in task flexibility and learner autonomy, which can foster self-regulation, though they note this may not suit every student.^[5]

Completion and dropout

Jordan (2015) gathered enrolment and completion data for 221 massive open online courses. Completion, defined as the percentage of enrolled students who completed, varied from 0.7 percent to 52.1 percent, with a median of 12.6 percent. Longer courses had lower completion, more recent courses had higher completion, and courses using auto-grading only also had higher completion. The first and second weeks appeared critical to engagement, after which the active proportion and the assessment-submitting proportion differed by less than 3 percent.^[6]

Lee and Choi (2010) reviewed empirical studies of online course dropout in post-secondary education from the preceding ten years. They identified 69 influencing factors in three groups: student factors, course or programme factors, and environmental factors. Strategies they examined included understanding each student's challenges and potential, providing quality course activities with well-structured support, and handling environmental issues and emotional challenges. The review concerns post-secondary courses and does not address employer-assigned training.^[7]

What the evidence does not show

Each synthesis has limits that matter for interpretation. The Department of Education search ended in 2008 and the Cook review in 2007, so neither examines current platforms. Comparisons often differed in several ways at once, including time, materials and pedagogy. Cook and colleagues reported large heterogeneity, meaning the average hides wide variation between studies. These findings are about average outcomes across studies, not about how any individual course or provider performs.^[1][3]

Completion figures also need care. The 12.6 percent median comes from open online courses and cannot be assumed for assigned workplace training, where learners, incentives and course length differ. The dropout review likewise concerns post-secondary education. Neither source measures completion in company programmes, so those figures should be treated as context rather than as benchmarks. Anyone evaluating an online programme would do better to ask for its own completion and outcome data.^[6][7]

Questions