Comparative studies of learning outcomes in distance and in-person classes. A substantial body of literature is available that generally supports no significant difference in learning outcomes in online and in-person classes. The “No Significant Difference” website does an excellent job explaining what the no significant difference phenomenon is and also provides helpful answers to other questions about the concept of no significant differences.
Listed below are citations for several comparative studies involving distance education and face-to-face classes. Some of the older studies may not have included effect size in their research findings. The articles below are grouped in terms of:
- No significant differences in learning outcomes in online and in-person classes.
- Significant differences favoring online.
- Significant differences favoring face-to-face, in-person classes.
- Mixed Results.
If more than one study was cited in the meta-analyses, the article has been included in the category that fits the majority of the findings cited.
No significant
differences
Interactive learning online at public universities
Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2014). Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management, 33(1), 94-111.
The authors randomly assigned 605 students at six public institutions to a hybrid or face-to-face introductory statistics course and found no significant difference in learning outcomes and achievement between the hybrid and face-to-face classes.
Grade based student learning outcomes
Cavanaugh, J., & Jacquemin, S. (2015). A large sample comparison of grade based student learning outcomes in online vs. face-to-face courses. Online Learning Journal, 19(2).
Cavanaugh and Jacquemin evaluated data for over 140,000 students enrolled in about 6,000 courses taught by the same instructors online and face to face at a large Midwestern four-year public institution from 2010 to 2013, reporting no statistically significant difference between grades in those online and face-to-face sections based on student performance.
Learning equity between online and on-site
Jones, S. J., & Long, V. M. (2013). Learning equity between online and on-site mathematics courses. MERLOT Journal of Online Learning and Teaching, 9(1).
Jones and Long compared course grades for students enrolled in online and face-to-face sections of a business mathematics course over 10 semesters and found no significant difference in course grades in the online and face-to-face classes for the last seven terms.
Study of face-to-face versus virtual laboratories
Miller T. A., Carver, J. S., & Roy A. (2018). To go virtual or not to go virtual, that is the question: A comparative study of face-to-face versus virtual laboratories in a physical science course. Journal of College Science Teaching, 48(2), 59-67.
Miller, Carver, and Roy studied pre- and post-test science content assessment scores for 96 students who completed weekly laboratory experiments in either virtual or face-to-face environments for a physical science class for nonscience majors. Those students also completed survey questions about attitudes toward science and preferences for lab modality. Results indicated no significant difference between the two groups of virtual and face-to-face lab students on gains in science content knowledge, attitudes toward science, or preference for lab modality.
Can a hands-on physics lab be delivered as a distance lab
Moosvi, F., Reinsberg, S. A., & Rieger, G.W. (2019). Can a hands-on physics project lab be delivered effectively as a distance lab? International Review of Research in Open and Distributed Learning, 20(1).
Using a sample of 233 project reports selected from over 450 students, Moosvi, Reinsberg and Rieger compared achievement outcomes for students completing hands-on experiments in distance labs versus face-to-face campus labs and found no significant differences in the scores assigned to their capstone projects. Focus group feedback and ways to improve the teaching of data analysis are also shared.
Comparison of student attitudes and performance
Nennig, H. T., Idarraga, K. L., Salzer, L. D., Bleske-Rechek, A., & Theisen, R. M. (2020). Comparison of student attitudes and performance in an online and a face-to-face inorganic chemistry course. Chemistry Education and Practice, (21), 168-177.
The authors compared scores on ten identical core exam questions for about 100 students enrolled in online or face-to-face sections of an organic chemistry course. While different instructors taught the online and face-to-face classes, textbooks were also the same, as was some of the curriculum. The authors analyzed mean test scores for those ten anchor items and overall class grades, and determined that the difference between means was not statistically significant. Attitudes toward chemistry were also comparable for the two groups.
Twenty years of research on academic performance
Shachar, M., & Neumann, Y. (2010). Twenty years of research on the academic performance differences between traditional and distance learning: Summative meta-analysis and trend examination. Merlot Journal of Online Learning and Teaching, 6(2), 318-334.
Shachar and Neumann conducted a meta-analysis on 125 research studies that were published from 1999 to 2009 and concluded that there were no significant differences between distance and traditional education, noting that achievement outcomes in distance education were actually higher, using their conservative estimates.
Learning outcomes in an online vs traditional course
Stack, S. (2015). Learning outcomes in an online vs traditional course. International Journal for the Scholarship of Teaching 9(1).
Using a quasi-experimental design, Stack compared final exam grades for students enrolled in the same course with the same instructor and found no significant difference in grades for students enrolled in the online versus the face-to-face section.
Learning online, offline, and in-between
Yen, S. C., Lo, Y., Lee, A., & Enriquez, J. (2018). Learning online, offline, and in-between: Comparing student academic outcomes and course satisfaction in face-to-face, online, and blended teaching modalities. Education and Information Technologies, 1-13.
Researchers compared the grades and satisfaction of students (N=86) across three sections of the same class offered in different modalities: online, face-to-face and a hybrid version. The same instructor taught each section and graded all of the material to limit any other influences, although the students self-selected into the sections. No differences were found for grades or student satisfaction, and the authors proposed a 2-factor model for success including interaction and flexibility.
Significant differences in learning outcomes in online and face-to-face classes favoring online
A comparison of online and face-to-face approaches
Bolsen, T., Evans, M., & Fleming, A. M. (2016). A comparison of online and face-to-face approaches to teaching introduction to American government. Journal of Political Science Education, 12(3), 302-317.
Bolsen et al. compared achievement and engagement results for over 1,500 students enrolled in a core political science course in traditional lecture, breakout, blended and online sections that also used different textbook formats. Results included higher levels of engagement in the online sections and higher levels of political knowledge in the online and blended formats than in the traditional and breakout sections, although students in the online course sections had a higher drop rate than that of the other formats.
A comparison of online and traditional chemistry
Faulconer, E. K., Griffith, J. C., Wood, B. L., Acharyya, S., & Roberts, D. L. (2018). A comparison of online and traditional chemistry lecture and lab. Chemistry Education Research and Practice, 19(1), 392-397.
Pass rate, withdrawal rate, and grade distributions of 823 students in similarly formatted online and face-to-face introductory chemistry classes were used to identify outcome differences based on modality. Students in the online version of the class were more likely to get As or Bs. Withdrawal rates between modalities were the same.
Randomized controlled trials of U‐Pace instruction
Fleming, R., Kienzler, S., Stoiber, L., Fleming, R. R., Pedrick, L. E., & Reddy, D. M. (2018). Randomized controlled trials of U‐Pace instruction: Outcomes in two gateway courses. Journal of Computer Assisted Learning, 34(6), 799-806.
Researchers at a Midwestern institution conducted randomized controlled trials to compare student outcomes in traditional face-to-face classes versus online competency-based classes designed with a heavy instructor presence. Those students in the online competency-based course scored higher on the cumulative exam and were several times more likely to earn a final grade of an A or B than those students in traditional modalities.
Evaluation of evidence-based practices in online learning
Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Washington, D.C: U.S. Dept. of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service.
The authors reviewed over 1,000 studies published from 1996 through 2008 and conducted a meta-analysis narrowing their focus to 99 studies, 45 of which had enough data to determine 50 effect sizes. They indicated that on average, students enrolled in online or blended classes with both online and face-to-face elements performed somewhat better than students enrolled solely in face-to-face classes, although they expressed caution in the findings due to small sample sizes, etc.
A comparison of student academic performance with traditional, online and flipped
Sharp, J., & Sharp, L. (2017). A comparison of student academic performance with traditional, online and flipped instructional approaches in a C# programming course. Journal of Information Technology Education: Innovations in Practice, 16, 215-231.
Sharp and Sharp compared academic outcomes of undergraduate students (N=271) enrolled in an introductory computer programming class offered in three different formats: online, face-to-face, and flipped. The findings suggest that outcomes may be better for those enrolled in online or flipped modalities.
Does online learning impede degree completion
Shea, P., & Bidjerano, T. (2014). Does online learning impede degree completion? A national study of community college students. Computers & Education, 75, 103-111.
Using data from the Beginning Postsecondary Student Survey, Shea and Bidjerano reported that community college students who took online or distance courses their first year significantly increased their odds of graduating compared to students who enrolled solely in the classroom their first year.
Analysis of research on the effectiveness of distance education
Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H.S. (2005). What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record, 107(8), 1836-1884.
Zhao et al. reviewed over 1,300 articles published from 1966 through 2002 and conducted a meta-analysis narrowing their focus to 51 studies and indicated that in two-thirds of the research, students enrolled in distance education outperformed students in face-to-face classes, with opposite results for one-third of the research.
Significant differences in learning outcomes in online and face-to-face classes favoring face-to-face
An Investigation of the relationship between grades and learning mode
Bourdeau, D. T., Griffith, K. V., Griffith, J. C., & Griffith, J. R. (2018). An Investigation of the relationship between grades and learning mode in an English composition course. Journal of University Teaching & Learning Practice, 15(2), 3.
Using over 2,900 grades from an English composition class, researchers compared pass rates, withdrawal rates and grade distributions for students based on the modality of the class: online, synchronous video, or face-to-face. Students in the face-to-face version of the class were more likely to get Bs, and they withdrew at lower rates than students in the online format.
Comparing student performance in online and face-to-face delivery
Helms, J. L. (2014). Comparing student performance in online and face-to-face delivery modalities. Journal of Asynchronous Learning Networks, 18(1), 147-160.
Helms compared achievement outcomes for students enrolled in his psychology course and reported significantly lower course grades for students in the online versus the face-to-face course, partly due to the online students missing one or more assignments.
The effectiveness of distance education across community colleges
Xu, D., & Jaggars, S. S. (2011). The effectiveness of distance education across Virginia's community colleges: Evidence from introductory college-level math and English courses. Educational Evaluation and Policy Analysis, 33(3), 360-377.
Xu and Jaggars reviewed English and math course grades for nearly 24,000 students enrolled in Virginia’s community college system during a four-year time period and indicated that achievement outcomes were better in those courses for those enrolled face to face than online.
linguistics: online vs. face-to-face
Johnson, D., & Palmer, C. C. (2015). Comparing student assessments and perceptions of online and face-to-face versions of an introductory linguistics course. Online Learning Journal, 19(2), 33-50.
Johnson and Palmer compared achievement outcomes for students enrolled in an introductory linguistics course and reported lower course grades for students in the online versus the face-to-face course, partly due to the lower prior GPA of those students enrolling in the online class.
The impact of online learning on students’ course outcomes
Xu, D., & Jaggars, S. S. (2013). The impact of online learning on students’ course outcomes: Evidence from a large community and technical college system. Economics of Education Review, 37, 46-57. doi: 10.1016/j.econedurev.2013.08.001
Xu and Jaggars analyzed achievement for over 18,000 students enrolled in Washington state’s community and technical college system during a five-year time period and concluded that persistence and course grades were better for those enrolled face to face than online.
Mixed Results
Comparing student success between courses offered online, blended and face to face
Ashby, J., Sadera, W. A., & McNary, S. W. (2011). Comparing student success between developmental math courses offered online, blended and face to face. Journal of Interactive Online Learning, 10(3), 128-139.
Ashby, Sadera and McNary analyzed grades for students enrolled in an intermediate algebra course at a community college and reported that students enrolled in the blended class did less well than students in the face-to-face and online classes if grades for those who did not finish were counted, and that students in the face-to-face class did less well than students in both other modalities if grades for those who did not finish were not counted as attrition rates were higher in the blended and online classes.
How does distance education compare to classroom instruction
Bernard, R. M., Abrami, P.C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., & Huang, B. (2004). How does distance education compare to classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379-439.
Bernard et al. analyzed 232 research studies published between 1985 and 2002 and reported that achievement outcomes for face-to-face classes were higher than those in distance courses for synchronous classes and opposite outcomes for asynchronous classes. Effect sizes for synchronous favored face-to-face, and effect sizes for asynchronous favored online.
Online versus face-to-face public speaking outcomes
Broeckelman-Post, M. A., Hyatt Hawkins, K. E., Arciero, A. R., & Malterud, A. S. (2019). Online versus face-to-face public speaking outcomes: A comprehensive assessment. Basic Communication Course Annual, 31(1), 10.
Student outcomes in a face-to-face public speaking course were compared to those in an online version of the same course. Each version of the course was standardized using the same resources and assessments, and student grades suggested no significant difference. The online students had higher levels of engagement, but they held slightly higher DFW rates. There was no significant difference in ratings of speaking performance, although the authors admit that online students had a chance to record their speeches while face-to-face students performed live.
Comparison of an online and a traditional MSSW program
Cummings, S. M., Chaffin, K. M., & Milam, A. (2019) Comparison of an online and a traditional MSSW program: A 5-year study. Journal of Social Work Education, 55(1), 176-187.
Researchers studied outcome differences for several graduates of an MSSW program over a 5-year period. Students were either in an online or face-to-face program, and the measures used were administered to graduates of the program every year. Face-to-face students had higher knowledge scores, while online students had higher skills scores and better perceptions of preparedness. The authors argue that the small effect sizes suggest that differences between online and traditional student outcomes have minor practical relevance related to student achievement.
No significant difference— Unless you are a jumper
Fendler, R. J., Ruff, C., & Shrikhande, M. M. (2018). No significant difference—Unless you are a jumper. Online Learning, 22(1).
Individual differences and academic outcomes were identified for 504 students taking either a face-to-face or online version of an undergraduate finance class, and researchers built predictive models based on this information to forecast how each student would perform in the other course modality. Based on the models, over half of the students would have performed about the same, yet others would have had jumped at least one grade (positive or negative) based on their individual differences.
Grades, student satisfaction and retention in online and face-to-face
Garratt-Reed, D., Roberts, L. D., & Heritage, B. (2016). Grades, student satisfaction and retention in online and face-to-face introductory psychology units: A test of equivalency theory. Frontiers in Psychology, 7, 1-10.
The authors studied achievement and retention rates of 866 students in an undergraduate psychology course that was offered in equivalent face-to-face and online formats and reported no significant difference in grades between delivery modes, except for a non-equivalent group activity, and a difference in retention rates between the two groups.
Retention, progression and the taking of online courses
James, S., Swan, K., & Daston, C. (2016). Retention, progression and the taking of online courses. Online Learning Journal, 20(2), 33-50.
James, Swan, and Daston compared over 656,000 student records from 14 institutions, calculated retention rates for those enrolled solely online, solely face-to-face, and in both course types, and reported mixed results; students enrolled solely online at community colleges were slightly less likely to be retained, students enrolled in both course types were more likely to be retained at primarily online institutions, and there were no significant differences in retention rates between delivery methods at four-year institutions.
Current status of research on online learning in postsecondary education
Lack, K. A. (2013). Current status of research on online learning in postsecondary education.
Lack reviewed 30 research studies published between 2003 and 2012 that met specific criteria and reported the researchers’ results in a helpful table (results are mixed).
Comparison of didactic, technical, role modeling, and ethics
Pereira, A. S. & Wahi, M. M. (2018). Comparison of didactic, technical, role modeling, and ethics learning acquisition in undergraduate online versus face-to-face modalities. Journal of Higher Education Theory and Practice, 18(5), 56-69.
Pereira and Wahi studied didactic, technical skills and ethics knowledge acquisition for 106 students who completed an introductory computer information systems class either online or face-to-face during academic year 2017 at a public Eastern university. Overall results indicated no significant difference between the face-to-face and online classes on information, skills, role modeling and ethics knowledge acquisition. However, the online and face-to-face classes differed on performance on individual tasks with the online group performing significantly better on certain tasks and the face-to-face group performing better on others. End-of-semester self-assessments indicated a preference for online classes.
An Investigation of the relationship between grades and learning modes
Roberts, D., Griffith, J. C., Faulconer, E., Wood, B. L., & Acharyya, S. (2019). An Investigation of the relationship between grades and learning modes in an introductory research methods course. Online Journal of Distance Learning Administration, 22(1), n1.
Researchers compared passing rates, grade distribution, and withdrawal rates for students (N=2,097) taking an introductory research methods course in different formats: online, in-person, synchronous video home, and synchronous video classroom. Passing rates of online and in-person modes were not statistically different from each other, but they were higher than either of the video modes. Students earned more “A”s in the online mode, and withdrawal rates were the same between online and all other modes.
Effectiveness of learning in online academic courses
Soffer, T., & Nachmias, R. (2018). Effectiveness of learning in online academic courses compared with face‐to‐face courses in higher education. Journal of Computer Assisted Learning, 34(5).
Soffer and Nachmias examined student outcomes from three different courses that were offered in online and face-to-face formats the same semester. One of the classes showed improved grades in the online format, but there was no difference in completion rates. Self-report measures were also evaluated, and the authors found differences in satisfaction and perceived learning between the modalities.
Online learning in postsecondary education
Wu, D. D. (2015). Online learning in postsecondary education: A review of the empirical literature (2013-2014).
Wu built on Lack’s 2013 meta-analysis (see above) by reviewing 12 research studies published in 2013 and 2014 that met the same criteria Lack used and again reports the researchers’ findings in a helpful table format (results are mixed).