Look Beyond Test Scores
to Win the Race to the Top
Dear Stuck,
Lately I’ve been keeping up with news about
the U.S. Department of Education’s (DoED)
new round of Race to the Top (Rtt T) funds,
worth $400 million, which can go directly to
districts and not just states. I was pleased but not
surprised to find out DoED was especially interested in projects targeting the effective use of
educational technology. An article ( tinyurl.com/
cg5k3dw) on the Southern California Public Radio
website featured quotes from Justin Hamilton, a
DoED spokesman, explaining how the department
will give preference to “bold plans that focus on the
use of technology and personalized learning.” He
continues: “We really want to help create 21st century classrooms to give our students 21st century
skills … [by] fundamentally changing the way you
deliver instruction. Yes, it’s about access to technology, but it’s also about how you use it.” As an advocate for the effective use of educational technology,
I am thrilled to hear that, and I bet you are too.
As an ISTE research associate, I spend a lot
of time working on program evaluations for
grant-funded programs in educational technology. Truth be told, I’m a bit particular about a
few things at work, one of which is the distinction between assessment and evaluation. Believe
it or not, that isn’t just some nomenclature nit-picking—it’s important.
A single program evaluation might target several
different goals and thus use different assessments.
Often, stakeholders (especially funders) are interested in test scores. But it usually takes a while for
teachers, students, and classrooms to shift enough
to see the effect of interventions on standardized
tests. That kind of change doesn’t happen overnight.
For a program like Rtt T, you’ll want to make sure
to assess several constructs as part of your program
evaluation. Start with how teachers and students
use technology. Take a look at how students exhibit
digital age skills because they are central to the
Rtt T program. They are also the focus of the ISTE
Classroom Observation Instrument (ICOT, iste.
org/icot).
We know that digital age learning environments
require students to use and build digital skills.
Understanding the relationship between those
skills and technology integration is fundamental to
evaluation for a program like Rtt T. You should use
different assessments to target different outcomes,
all of which are a part of a single program evaluation. For instance, you might use data from surveys
or interviews to shed light on what happens in the
classroom, then use data from classroom observations to help you understand patterns in test scores.
ISTE’s Technology Research Exchange ( trex.iste.
org) may have some reviews of publications related
to this topic, so take a look there if you’re hunting
for ideas.
While gains in test scores may be an important
outcome, don’t let them distract from other program goals and the interplay of factors that affects
those goals. A good program evaluator will help
you identify how to assess different goals and
model how different methods inform each
other to contribute to a solid evaluation.
ASK DR. EVAL
Dr. Eval is part of ISTE’s
Research & Evaluation
Department, which
provides needs assessment, grant development,
and program evaluation
services to school districts, universities, public
agencies, and private
organizations. Through
ISTE’s Technology
Research Exchange
(TREx) review site and
the Ask Dr. Eval column,
R&E helps ISTE members use research and
evaluation in their work.
