Now that the big question of effect of
technology on learning has been answered, this report concludes that research can now focus on the best ways
to integrate the many technology tools
into learning and teaching practices.
Online Learning Works Best
with a Teacher’s Guidance
Barbara Means, Yukie Toyama, Robert
Murphy, Marianne Bakia, and Karla
Jones tackle the impact question from
an online learning angle in “Evaluation
of Evidence-Based Practices in Online
Learning: A Meta-Analysis and Review
of Online Learning Studies,” which the
U.S. Department of Education’s Center
for Technology in Learning published
in 2010. Interactive, collaborative, and
multimedia features are relatively new
to online learning environments, so the
authors could find only nine studies
that apply to K– 12 education (between
2006 and 2008), and only five of these
were sufficient to report effect sizes,
despite that “over a million K– 12 students took online courses in 2007–08”
(Means et al. referring to Picciano &
Seaman, 2009). In addition to the five
K– 12 studies, the meta-analysis included 43 studies of postsecondary learners.
The article addresses the relative effectiveness of online learning compared
with face-to-face instruction and with
blended instruction and found that:
• Online learners performed slightly
better than face-to-face learners.
However, instruction delivered only
online is neither better nor worse
than face-to-face instruction. Authors
attribute the small advantage to the
greater flexibility, and therefore more
time, students apply to learning.
• Blended instruction, which combines face-to-face with web delivery,
shows a larger effect size than web-only delivery when compared to
face-to-face-only instruction. This
larger gain may justify the greater
expense of blended instruction
compared with self-directed
online learning.
• Collaborative or instructor-directed
instruction showed a larger effect size
than independent online learning.
The authors stress the limitations of
this meta-analysis, including the fact
that the sample of students in these
studies is too small to demonstrate
confidence in how the findings apply to younger learners. In fact, effect sizes for the subsample of K– 12
learners did not show any advantage
for web delivery, and we can’t assume
that what is effective for adult students
necessarily applies to children as well.
Another important consideration
for interpreting these findings is that
“treatment conditions,” such as time
on task and pedagogic approaches,
cannot be separated from the method
of instructional delivery (face to face,
online, or hybrid). The authors assert that it is the combination of web
delivery with curriculum, time, and
pedagogy that explains the differences
in outcomes.
In their review of studies (which
was not included in the meta-analysis
for technical reasons), the authors
conclude that videos and quizzes do
not improve learning delivered online.
However, giving individual online
learners the opportunity to control
online activity and to reflect and self-monitor their learning increases gains.
case for integrating technology into
instruction. Both studies show that
technology improves student outcomes the most when teachers integrate technology tools into the learning and teaching experience. They
also support the idea that substituting
technology for quality teaching is not
as effective. Recent and continuing
changes in technology, such as social
media and interactivity, probably
increase the web’s effectiveness for
learning, but there is not enough research yet on these developments.
If you are implementing a technology program, including well-designed
evaluation research will enable you
to make specific claims about the
outcomes of your investment for your
stakeholders and contribute to the
growing knowledge of the field. When
you’re thinking about how these findings might apply to your environment,
one place to start is the NETS Essential Conditions, which provides guidelines to set the stage for success.
Implications for Practice
These two studies are important to
teachers, administrators, and program planners because they make the
Resources
NETS Essential Conditions:
iste.org/NETS/
EssentialConditions
“Evaluation of Evidence-Based Practices in On-
line Learning: A Meta-Analysis and Review of
“What 40 Years of Research Says about the
Impact of Technology on Learning: A
Second-Order Meta-Analysis and Validation
Study” by R. M. Tamin (2011), Review of
Educational Research, Vol. 81, pages 4–28.
