The Profits and Perils
of Classroom Observation
Brandon Olszewski is
a research associate in
You can email him at
T he programs that ISTE’s Research and Evaluation (R&E) Department evaluates often focus on the effective use of educational technology in K– 12 science, technology,
engineering, and mathematics (STEM) classrooms. We use all the usual methods for gathering data—surveys, interviews, document review,
statistical analyses, and the like—but classroom
observations hold a special place in our hearts.
While people report all sorts of useful things on
surveys and in interviews, we have found that it is
something else entirely to see learning in action.
Contrary to popular misconceptions, observations are not stern judgments that evaluators
confer upon classroom practitioners. They are
more about constructive feedback. We work
with a number of projects funded by grants from
the National Science Foundation and U.S. Department of Education, and in that work, we do
our best to help project staff and teachers understand what parts of their programs are working
well and how to improve those that aren’t.
Sure, compared with online surveys, observations are expensive. But well-designed observations are worth the time and money, especially
when we want to know about how teaching
changes over time.
What makes a well-built observation? Here’s
what the research tells us.
Sometimes, we all know what something is and
what it is called. For instance, if part of an obser-
vation plan is to assess instructional groupings,
it’s probably not difficult to distinguish among
individual, small-group, and whole-class activi-
ties. But what if we’re interested in assessing the
presence of project-based learning or authentic
instruction? Each of these concepts must come
with a set of indicators or aspects of the concept
that we can observe. Agreeing on these indicators
is sometimes difficult. We must establish con-
struct validity, which means that a community
(of researchers, evaluators, teachers, etc.) agrees
that the indicators adequately represent the
concept of interest. One way to do this is through
collaboration among key stakeholders, where the
group iteratively refines the concepts through dis-
cussion and research. But remember that in edu-
cation, a variety of people must be able to under-
stand what you’re talking about. So it’s imperative
that the concepts are accessible and “valid” for
others outside of your expert circle.