What Does the Research Say?
RESEARCH WINDOWS
Clare Strawn
Clare Strawn, PhD,
is a senior research
associate with ISTE’s
Research and Evaluation Department (iste.
org/research). You
can contact her at
cstrawn@iste.org.
W hat does research say about the impact of technology on learning outcomes? It’s a question that ISTE’s Research
and Evaluation Department fields often, but
it’s difficult to answer. Unfortunately, you can’t
separate technology as a component of learning
and teaching from the many other contexts that
influence learning outcomes, such as how it is
used pedagogically in the classroom.
Think about how we usually measure learning
outcomes: with standardized test scores. Then
think about everything that influences how well
each student performs on the test, from the
home environment to individual psychosocial
factors (including learning abilities), teacher
preparation and skill, and classroom environment. In the big picture, how much difference do
we expect the use of technology to make?
Fortunately, thousands of authors of basic research studies and program evaluations have investigated the impact of many implementations
of technology on learning in many contexts. To
answer the question of how any one influence
affects learning outcomes, researchers enter
measures of what they expect will be important,
including an intervention such as technology
use, into a statistical model. By interpreting the
results, the researcher can estimate to what degree any item influences the outcome relative to
the other items in the model. The influence of
technology use would have to be big to outweigh
all of the other contextual and individual factors.
Studies can also report results as an effect size
(the difference between the mean for the treat-
ment group and the mean for the control group,
divided by the pooled standard deviation) that
compares and standardizes the average scores of
the two comparison groups. A meta-analysis can
derive a composite effect size across many stud-
ies to estimate the overall impact of technology
on learning outcomes in the big picture.
Tech Makes a Big Difference in K– 12
According to R. M. Tamin’s article “What 40
Years of Research Says about the Impact of
Technology on Learning: A Second-Order
Meta-Analysis and Validation Study,” published
earlier this year in the Review of Educational
Research, the use of technology in instruction
shows small to moderate gains in student learning over instruction that does not use technology. Technology used to support instruction had
slightly stronger effects than applications that
deliver direct instruction.
The study—which includes K– 12 and postsecondary levels, different subject matters, and many
kinds of technological interventions but excludes
100% online learning—supports findings from
one of the contributing meta-analyses that “
computer technology used as ‘support for cognition’
were significantly greater than those related to
computer use for ‘presentation of content.’ ” For
example, students learn more from teachers who
use technology as tools for learning than when
they learn directly from an educational website,
CD, or educational software program.
The other important distinction is that technology integration in K– 12 schools had a larger
effect than it did in postsecondary contexts.
Research does not explain most of the difference in student scores, but on average, a student
exposed to technology in K– 12 instruction will
perform 12 percentile points higher than a student without technology-enhanced instruction.
