When it comes to evidence-based programs, nothing matches Direct Instruction
Direct Instruction has something most education programs don’t: decades of research demonstrating its effectiveness.
It’s standard language in the sales pitches educational publishers make to school districts: Our curriculum is rooted in “research,” “evidence,” or lately, “the science of reading.”
Federal law says programs don’t need to be tested to claim they’re “evidence-based.” They just need to be derived from earlier research findings, even if there’s no evidence the programs themselves improve student learning.
Direct Instruction is a rare exception. Its reading, math and other programs have been examined in hundreds of studies over more than half a century — a “mountain of data,” in the words of one research team. Taken together, the studies show a consistent pattern of positive results.
In a 2021 review of existing research, professors Lee Mason and Maria Otero concluded that Direct Instruction can effectively double the rate of learning, allowing students to make two years of academic progress in a single year.
Direct Instruction’s effectiveness, they wrote, “cannot be doubted.”
SOURCE: Stockard, et al., “The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half Century of Research” (2018)
Yet today, few schools use Direct Instruction, and it rarely appears on state lists of recommended reading curricula.
Direct Instruction has long struggled to distinguish itself from its less-tested competitors, to the frustration of its late founder, Siegfried “Zig” Engelmann.
In a 2004 column in Education Week, Engelmann criticized policymakers for focusing on individual features of research-tested programs, rather than the combination that made them successful. For example, research shows phonics is important. But not every program with phonics is necessarily effective. Policymakers were making a leap in logic, he wrote.
Engelmann had an analogy for this: Every Dalmatian has spots, but not every spotted dog is a Dalmatian. Policymakers needed to get away from that “spots-first logic,” he wrote.
“If the goal is to identify programs that are effective, why not take the most direct route?” he asked.
More effective ‘by a long shot’
Some of the earliest research establishing Direct Instruction’s effectiveness dates back half a century — to one of the largest educational studies ever conducted.
Project Follow Through was a massive research project run by the federal government from 1968 to 1977. It tested 22 educational models, focusing on kindergarten through third grade students, mostly from low-income households.
The research design was far from perfect, but in the end, Direct Instruction performed the best. It was the only model to have a positive effect, on average, in every category measured. And in several follow-up studies when the students were in middle and high school, the program continued to show evidence of long-term positive effects even years later.
In the decades since, researchers have studied Direct Instruction again and again. Meta-analyses — reviews that combine the results of many previous studies — have repeatedly found students learn more when they’re taught with Direct Instruction.
John Hattie, a prominent education researcher, found that the effects of Direct Instruction and similar direct teaching methods were more powerful than many other efforts to raise achievement, including improving classroom management or increasing parental involvement.
In 2020, sociologist Jean Stockard and her colleagues published the most extensive review of Direct Instruction to date. Their book, “All Students Can Succeed: A Half Century of Research on the Effectiveness of Direct Instruction,” was the culmination of more than a decade of work.
“We had hundreds of studies and thousands of effect sizes to study,” Stockard said. “Having a big pile of data for sociologists is like going into a candy store. It was really exciting.”
In the book’s preface, Stockard described the results across 50 years of research as “extraordinarily consistent.” Students did better with Direct Instruction regardless of when the study was conducted, how it was designed or which schools and students were studied.
“No matter how we sliced the data, we got the same result,” Stockard explained. “The program was incredibly effective — and more effective than other programs by a long shot.”
One notable example in the meta-analysis came from Florida. A 2009 study compared six different reading programs used with about 30,000 elementary school students — a large sample. The programs were all required to meet the standards of “scientifically based reading research.” The study found that students taught with the Direct Instruction reading program performed the best.
The sheer volume of research allowed Stockard and her colleagues to look at the conditions under which Direct Instruction was most effective. The program got the best results when teachers closely followed the developers’ guidelines and when students spent more time in Direct Instruction. Under those conditions, the gains were powerful enough to erase the achievement gap between children from low-income families and their higher-income peers.
Direct Instruction research today
For all the positive findings, some researchers still have questions about the strength of the evidence supporting Direct Instruction.
The What Works Clearinghouse, a U.S. Department of Education project that evaluates research, has determined that most studies of Direct Instruction are too old or don’t meet its standards for research design.
The federal clearinghouse prioritizes randomized experiments. Few such experiments have been done on Direct Instruction, and they are relatively rare in education research generally.
Conducting randomized experiments in schools is “really tough,” said Keith Smolkowski, a senior scientist at Oregon Research Institute who has studied Direct Instruction.
Smolkowski said Direct Instruction also faces resistance from some research funders. He recalled that federal grant reviewers rejected a proposal in the 2000s to study Direct Instruction with English language learners. “The committee was quite adamant that this was going to ruin kids’ lives,” he said. Despite its long track record of good results, the program is unpopular with the education establishment because it is viewed as rigid — and even authoritarian.
Another issue is the age of the evidence. Much of the research on Direct Instruction dates back decades, and the What Works Clearinghouse generally excludes studies more than 20 years old from its reviews.
Geoffrey Borman, a professor at Arizona State University, coauthored a 2003 meta-analysis that found Direct Instruction was the most widely studied and most effective program available at the time. But he said relatively little high-quality research on Direct Instruction has been done since.
New research is underway. Smolkowski and other researchers have a $4 million federal grant to test Direct Instruction with struggling first grade readers in eight states. And McGraw Hill, which publishes Direct Instruction programs, is funding a new evaluation.
Direct Instruction supporters argue that the older evidence still deserves serious consideration. “Penicillin was invented a long time ago, too,” said Cathy Watkins, a professor emerita at California State University, Stanislaus. “But it still works.”
