Can Web Courses Replace the Classroom in Principles of Microeconomics?

Can Web Courses Replace the Classroom in Principles of Microeconomics?

By Byron W. Brown and Carl E. Liedholm*

The proliferation of economics courses offered partly or completely on-line (Katz

and Becker, 1999) raises important questions about the effects of the new technologies on

student learning. Do students enrolled in on-line courses learn more or less than students

taught face-to-face? Can we identify any student characteristics, such as gender, race,

ACT scores, or grade averages, that are associated with better outcomes in one

technology or another? How would the on-line (or face-to-face) students fare if they had

taken the course using the alternative technology? This paper addresses these questions

using student data from our principles of microeconomics courses at Michigan State

University.

I. The Courses

This study analyzes examination performance of students in three different modes

or technologies of instruction in principles of microeconomics. We call the modes of

instruction live, hybrid (for reasons that will become clear), and virtual.

Each of these modes of instruction employs different instructional materials, but they all

have some features in common, such as the same textbook, Mankiw (2001), use of

multiple-choice examinations, and e-mail and course web sites for communication.

The live course, taught in two sections by Liedholm in Fall 2000, met face-to-face

for three class hours per week. Although the classes were large, the instructor directly

engaged them in the learning process by using animated PowerPoint slides, videos, and

group demonstrations, and by calling on individual students.

2

Brown taught the hybrid course in Fall 2000. It supplemented face-to-face

lectures of two class hours per week with a variety of on-line materials. Most important

of these on-line materials was an extensive collection of interactive, collaborative

practice materials called Problems in Microeconomics

(http://www.msu.edu/course/ec/201/brown/pim). In these problems the parameters in the

relevant functions are pseudo-randomized so each student receives a unique version of

each problem set, and students were encouraged to work together on the problems.

Completing the problem sets was a course requirement. The remaining on-line materials

included an extensive set of PowerPoint slides available on-line as a supplement to the

textbook, and extensive files of repeatable practice quizzes.

The virtual course, offered in Fall 2000 and Spring 2001, was the product of a

staff of professional web course producers, designers, programmers, and pedagogical

experts operating under the direction of the authors. What makes the comparison of the

live and virtual courses especially interesting is that we were able to incorporate

streaming video of Liedholm's lectures in an on-line format that included synchronous

viewing of textual material. Thus the students taking the course completely on-line got to

enjoy as nearly as possible what the live students got in class. The virtual course also

included Problems in Microeconomics, and the other on-line materials available to

students in the hybrid course, including the repeatable practice quizzes.

II. The data and model

Our data set consists of information on the test scores and personal data from

official university records from 363 students in Liedholm's live course sections, 258

students from Brown's hybrid course, and 89 students from two semesters in the virtual,

3

on-line course. Table 1 shows the basic statistics on most of the variables. The students

in this study we believe were, in most respects, typical of students in the principles course

in general. Overall, forty-eight percent of the students were women, about thirty percent

were majors in business, eighteen percent were from social science, and seven percent

from engineering. The live classes had a significantly higher percentage of Black

students, and athletes. The virtual sections had significantly higher ACT comprehensive

scores, and had completed significantly more credits towards graduation then either the

live or hybrid sections.

The dependent variable, total score, is the percentage of the total number of

questions answered correctly from thirty-seven questions that were included on all of the

students' examinations. The questions are available at

http://www.msu.edu/~brownb/vstudy.htm. In order to analyze depth of understanding,

we divided the pool of questions into three groups according to the degree of

sophistication in using economic concepts we believed a student would need to answer

each question. The first group, subscore 1, included sixteen questions that were either

straightforward definitions or concept identification. An example would be recognition

of the definition of elasticity of demand. The second group, subscore 2, consisted of

eleven questions that required a simple application or extension of a microeconomic

concept or tool. An example would be the calculation of elasticity of demand from data.

Subscore 3 consisted of questions requiring a more complex application of a concept. An

example would be a question that asks about the implications for total spending on a

good when price changes under specific assumptions about elasticity of demand. While

the overall percent correct falls as we move up the groups from 1 to 3, this grouping does

4

not correspond exactly to one based simply on question difficulty, an alternative we also

explored.

Our empirical specification is that the students' scores depend on the students'

characteristics, and in which section of the course they found themselves, live, hybrid, or

virtual.

III. Results

Table 2 reports results of OLS estimation for most of the important variables. We

explored a large number of non-linearities, and interactions and found that none of these

made any meaningful difference to the results. Significant censoring or ceiling effects

were present in the data for the subscore 2 and subscore 3 subgroups. We used Tobit

estimation to account for this, but there were no important differences from the OLS

results.

A Chow test of the differences in coefficients among the three modes of

instruction shows that the live and virtual

...