March 2015 – Core Knowledge Foundation

Reading Recovery is an intensive intervention for first graders who are struggling to learn to read. Although its research base is not huge, well-controlled studies have found it highly effective. Newly published research shows that Reading Recovery is remaining effective even as it scales up. This is great news—and could mean that Reading Recovery will be adopted by thousands more schools.

To reap Reading Recovery’s benefits for first graders without lowering achievement in upper elementary and beyond, schools will need to be very careful about when they use it. Reading Recovery is a pull-out program: Providing one-on-one instruction is Reading Recovery’s strength—but if students are pulled out of history, science, art, or music, their short-term gains in reading ability could come at the expense of their long-term comprehension ability.

Building young children’s knowledge of science, history, art, and music matters just as much early reading skills (image courtesy of Shutterstock).

Let’s take a quick look at what Reading Recovery does. According to a CPRE report that is an earlier version of the new, peer-reviewed study (and that was well-vetted by the What Works Clearinghouse):

Reading Recovery is an intensive intervention targeting the lowest-achieving 15-20 percent of 1st-grade readers. It takes as its underlying principle the idea that individualized, short-term, highly responsive instruction delivered by an expert can disrupt the trajectory of low literacy achievement, produce accelerated gains, and enable students to catch up to their peers and sustain achievement at grade level into the future. Reading Recovery attends to phonemic awareness, phonics, vocabulary, fluency, and comprehension–the critical elements of literacy and reading instruction identified by the National Reading Panel (2000).

In short, it has a strong research base. Even better, it has strong results. On the Iowa Test of Basic Skills, impact on reading ability was two-thirds of a standard deviation.

Reading Recovery does more for struggling first graders than many people believed possible. But it can’t do everything. As a short-term intervention, it can’t meaningfully increase students’ general knowledge. As a result, it can’t address a key factor for reading ability in later grades.

To be clear, I don’t think Reading Recovery should be responsible for increasing knowledge of the world. It’s a targeted program that’s getting a great deal from a relatively small amount of instructional time (about 30 minutes a day for 12–20 weeks). So my point is not that Reading Recovery should change—it’s that schools using Reading Recovery need to be very strategic about when to deliver the intervention. What will the student miss? Is there any way to not miss anything, to deliver Reading Recovery before or after school? Or perhaps during silent reading time, which these low-achieving first graders may only minimally benefit from?

It may seem that there’s nothing more important in first grade than developing basic reading skills. But in fact, research indicates that building general knowledge is just as important—possibly even more important. In a 2010 study by David Grissmer et al., general knowledge at kindergarten entry was a better predictor of fifth-grade reading ability than early reading skills. General knowledge also predicted later science and math achievement:

[The] general knowledge test measured the child’s early comprehension of physical and social science facts. Whereas the early math and reading tests focused mainly on procedural knowledge, the general knowledge test focused mainly on declarative knowledge (i.e., elementary knowledge or comprehension of the external world). General knowledge was the strongest predictor of later reading and science and, along with earlier math, was a strong predictor of later math…. Paradoxically, higher long-term achievement in math and reading may require reduced direct emphasis on math and reading and more time and stronger curricula outside math and reading.

This is a powerful finding: Kindergartners’ general knowledge is critical to their reading, science, and math achievement at the end of elementary school. So, building students’ knowledge—as much as possible and as early as possible—is critical too.

Educators do not have to choose between building children’s knowledge and skills. There is time for both, if everyone values both. Sadly, the importance of building knowledge in the early grades is still unrecognized by many schools. As Ruth Wattenberg has explained, “When elementary teachers were asked during what time period struggling students received extra instruction in ELA or math, 60 percent said that they were pulled from social studies class; 55 percent said from science class.”

Pull outs from science, social studies, art, and music must stop. Along with great literature, these subjects are what make up general knowledge. They are inherently interesting and absolutely essential. As Reading Recovery continues to spread, it would do well to help schools see that when they do their interventions matters just as much as which interventions they choose.

Slowly, steadily building the knowledge and capacity of administrators and teachers—that’s what’s working in school systems around the world. As explained in a report by Geoff Masters, a leading researcher in Australia, the educational systems that are improving have invested in educator capacity, while those that are not improving (or are declining) have been tinkering with accountability and incentives (as US policymakers have been, with poor results for reading).

Before those who oppose accountability and market-based reforms declare victory, they’ll need to take a careful look at what Masters means by building capacity. The sad fact is, very few school systems, and few very reformers, in the US are doing the things Masters identifies as essential to increasing student achievement. What he describes is not your typical professional development—it’s a more fundamental commitment to quality and equity:

In some countries, reform efforts tend to have been focused first on building the capacity of school leaders and classroom teachers to deliver high quality teaching and learning, and on ensuring that excellent teaching and leadership are distributed throughout the school system. In other countries, including a number of English-speaking countries, greater reliance has been placed on using systems of accountability and incentives to drive improvement….

Table 1 Two general approaches to school reform
[Reformatted for blog]
________________________________________

Belief
Improvement will occur if schools are given incentives to improve (rewards, sanctions, having to compete for students).
Strategies
– stronger performance cultures
– better measures of outcomes
– personal accountability for improvement
– performance pay linked to test scores
– greater public transparency
– financial rewards for school improvement
– sanctions for failure to improve
– increased competition for students
– greater autonomy to compete
– more parental choice

Belief
Improvement will occur by building the capacity of teachers and school leaders and by ensuring high quality practice throughout the system.
Strategies
– attract more able people into teaching
– train approximately the number of teachers required
– place a high priority on building teachers’ content and pedagogical content knowledge
– develop school leaders’ capacities to build and lead cultures of continual improvement in teaching and learning
– ensure that high-quality teaching and leadership are equitably distributed across all schools
________________________________________

There has been growing recognition that more effective than setting ambitious targets for improved student performance, or attaching money or other consequences to student test results, is to work directly on developing the teaching and leadership practices that result in improved student outcomes….

Systematic studies of what school leaders do to achieve whole-school improvement reveal a high degree of consistency in the priorities set by leaders of turn-around schools. These priorities are summarised in the National School Improvement Tool (Masters, 2012) and can be thought of as a set of micro-strategies for whole-school reform. They include:

– setting an explicit school improvement agenda;
– systematically monitoring progress in achieving desired improvements;
– establishing and sustaining a culture of support and high expectations;
– targeting the use of school resources to address student needs;
– encouraging teachers to work as a team to improve teaching and learning;
– establishing a coherent, sequenced, shared school curriculum;
– sustaining a strong focus on addressing individual learning needs;
– implementing effective pedagogical practices including diagnostic practices; and
– using local community resources to better meet student needs.

Reading down this list, these strategies seem like common sense—which makes it all the more frustrating that they are not common practices.

The only suggestion I’ll make is to put “establishing a coherent, sequenced, shared school curriculum” at the top of the list. Almost all of the other strategies require a shared curriculum as their foundation—you can’t set expectations, monitor progress, address needs, or work as a team without a grade-by-grade and subject-by-subject map of the specific content and skills students must master.

Without a detailed map of what students are supposed to learn, how can we know if they are ahead or behind, much less how to help them? (Image courtesy of Shutterstock.)

Thanks to Marc Tucker for drawing attention to this report. I admire Tucker—if we’d been listening to him for the past few decades, we’d have one of the world’s best, most equitable school systems. So, when Tucker wrote that Masters has “written a paper you need to read,” I started reading (you should too).

In recent months, Teach Plus had over 1,000 teachers review sample items from PARCC, one of the two testing consortia trying to create assessments aligned to the Common Core standards.

I say “trying” because in reading, the task is pretty much impossible. The standards specify things students should be able to do, but they contain almost no content. Thankfully, they do call for content-rich curriculum and explain that comprehension depends on broad knowledge, but they don’t provide the content-specificity needed to guide instruction or assessment.

Thousands of different curricula and assessments could be aligned to the standards, which would be fine if teachers were trusted to develop both. But teachers are not allowed to create the assessments—at least the ones that count. So it is entirely possible for a teacher to develop an “aligned” curriculum that does not prepare students for the content that shows up on the “aligned” assessment.

The result is an unfair assessment.

Test developers acknowledge as much, creating guidelines for item development that minimize knowledge as a source of “bias.”

Well, the 1,000 teachers who just reviewed PARCC think the stripping of knowledge did not go far enough:

Nearly all participants found that the PARCC passages were better quality than the passages in state tests, as they are previously published pieces (indicating that they are complex and demonstrate expertise in nonfiction). However, there was some concern students did not have “background knowledge, nor the vocabulary to understand” vocabulary within the texts. Their comments suggest that to assess students as accurately as possible, some portions may need to be edited for diverse learners, or those with limited background knowledge of certain content areas.

I understand why teachers would call for reducing the prior knowledge demands of the test—they are stuck in this crazy world of being measured with content that no one told them to teach. But let’s be honest: reducing the knowledge demand makes the test a little fairer; it does not make the education students are getting any better.

The knowledge bias can’t be avoided with tests that are not explicitly aligned to the curriculum. Without a curriculum that specifies what has been taught—and therefore what it is fair to expect students to know—test writers are reduced to a narrow band of banal topics (but even “Jenny goes to the market” demands some prior, unequally distributed knowledge).

The less the knowledge bias, the less the test reflects real-world comprehension. Outside testlandia, comprehension is not isolated from knowledge. An adult who can’t comprehend a newspaper is not considered literate. Broad knowledge is inherent in literacy. If we care about reading, as opposed to testing, we shouldn’t be creating tests that minimize knowledge demands. We should be developing a coherent instruction, assessment, and accountability system that builds broad knowledge and is fair because it tests what is taught.

Clearly, our nation’s policymakers need a crash course in reading. Once they understand that there is no such thing as general comprehension ability, maybe they’ll stop trying to hold schools accountable for developing it.

Fortunately, a great crash course is now available: Daniel Willingham’s latest book, Raising Kids Who Read: What Parents and Teachers Can Do. If policymakers read between the lines, they’ll see an awful lot they can do too.

As with Willingham’s previous books, this one is engaging, easy to read, and super informative. Here’s just a taste:

Most parents want their children to be solid general readers. They aren’t worried about their kids reading professional journals for butterfly collectors, but they expect their kids to be able to read the New York Times, National Geographic, or other materials written for the thoughtful layperson. A writer for the New York Times will not assume deep knowledge about postage stamps, or African geography, or Elizabethan playwrights— but she will assume some knowledge about each. To be a good general reader, your child needs knowledge of the world that’s a million miles wide and an inch deep—wide enough to recognize the titles The Jew of Malta and The Merchant of Venice, for example, but not that the former may have inspired the latter. Enough to know that rare stamps can be very valuable, but not the going price of the rare Inverted Jenny stamp of 1918.

If being a “good reader” actually means “knowing a little bit about a lot of stuff,” then reading tests don’t work quite the way most people think they do. Reading tests purport to measure a student’s ability to read, and “ability to read” sounds like a general skill. Once I know your ability to read, I ought to be able (roughly) to predict your comprehension of any text I hand you. But I’ve just said that reading comprehension depends heavily on how much you happen to know about the topic of the text , because that determines your ability to make up for the information the writer felt free to omit. Perhaps, then, reading comprehension tests are really knowledge tests in disguise.

There is reason to think that’s true. In one study, researchers measured the reading ability of eleventh graders with a standard reading test and also administered tests of what they called “cultural literacy”—students’ knowledge of mainstream culture. There were tests of the names of artists, entertainers, military leaders, musicians, philosophers, and scientists, as well as separate tests of factual knowledge of science, history, and literature. The researchers found robust correlations between scores on the reading test and scores on the various cultural literacy tests—correlations between 0.55 and 0.90.

If we are to increase reading ability, policymakers will have to accept that it takes many years to develop the breadth of knowledge needed for tests that are not based on a specific curriculum. We shouldn’t be stripping the knowledge demands out of our tests; we should be stripping the unreasonable mandates from our accountability policies. If we all focused on raising readers, we would spend far less time on testing and far more on building broad knowledge.

Young reader, building knowledge and comprehension, courtesy of Shutterstock.

Teach for America (TFA) aims to increase student achievement by increasing the quality of teaching. Concerned with the short-term commitment TFA asks its recruits to make, I’ve never been sure what to make of TFA. Seeking teachers who were themselves strong students and focusing on outcomes, it has the potential to elevate the teaching profession. But by recruiting for a two-year stint, it also questions teaching as a career.

With these grumblings in the back of my mind, I’m always interested in research on TFA. If there were clear evidence that it did, or did not, work, my dilemma would be resolved. Well, even with a very rigorous new study by Mathematica Policy Research, my dilemma lives on. Comparing 66 TFA teachers with 90 non-TFA (mostly traditionally prepared) teachers in 36 schools serving preschool through fifth grade, only one significant difference was found: TFA teachers in preschool through second grade were more effective in reading. They added about 1.3 months of learning.

That’s important—it indicates that the TFA teachers are doing a better job on foundational reading skills. But what’s also important is that there was no difference in third through fifth grade. Even more important, there was plenty of room for improvement: on average, these preschool through fifth grade students’ were at the 34^th percentile.

Why isn’t reading comprehension budging? The Mathematica study can’t answer, but readers of the Core Knowledge blog certainly can. Until elementary schools—and all types of teacher preparation programs—get serious about systematically building knowledge and vocabulary, reading comprehension will remain far too low.

Books on the brain courtesy of Shutterstock.

Fortunately, more and more educators, administrators, and professors are coming to understand the nature of comprehension. A recent paper by Donald L. Compton, Amanda C. Miller, Amy M. Elleman, and Laura M. Steacy makes me think knowledge is starting to get its due (thanks to Aaron Grossman for sending it to me).

With a strong-but-brief review of the research on comprehension, the paper is well worth reading. So, I’m just offering some highlights (and hoping TFA’s Wendy Kopp is reading):

Much of the instructional research on reading comprehension has focused on strategy instruction as a means to engage students with text and help them monitor their comprehension…. This focus is warranted as evidenced by the effectiveness of strategy instruction especially for struggling readers…. However, it is unclear whether increased comprehension can be attributed to learning specific strategies. In their review of strategy instruction, Rosenshine and Meister (1994) noted that it did not matter which strategies were combined; as long as multiple strategies were used, students’ comprehension increased. In fact, it may not be the strategies themselves that engender changes in comprehension, but possibly some other factors that strategy instruction fosters, such as deeper engagement with the text and awareness of the need to monitor comprehension.

Our intent here is not to argue against the positive role strategy instruction may play in increasing engagement with text but instead to highlight unforeseen consequences associated with this type of instruction. We propose that strategy instruction may result in low-level text representations that embody only what is explicitly expressed in a text…. Deep level understanding of a text, on the other hand, goes beyond the text in nontrivial ways, requiring the construction of meaning through inference making, not just passive absorption of information….

Reading comprehension occurs as the reader builds a mental representation of the text…. The majority of comprehension theorists suggest that there are at least two levels of representation: a text-based representation and a situation model…. The text representation conveys the underlying meaning of the text’s explicit information…. The situation model involves the intertwining of the reader’s background knowledge with the text-based representation to form a deep representation of the text. Thus, the situation model is a more meaningful representation that goes beyond the text-based information…. We maintain that failure to construct situation models during reading is an acute symptom associated with reading comprehension disability.

A number of studies have reported that individual differences in background knowledge significantly influence the building of a representative situation model…. Readers who possess high levels of knowledge consistently exhibit better comprehension and retention than readers with low levels of knowledge….

[One study] examined the contribution of knowledge to comprehension processes by asking good and poor fifth-grade readers to read or listen to passages and answer questions. Results indicated that having some knowledge about a passage’s topic, which poor readers had less of, was positively associated with the likelihood of correctly answering questions about that passage. In addition, general knowledge and vocabulary knowledge remained significantly associated with correct responses even while controlling for passage specific knowledge. (Again, poor readers possessed less general knowledge and vocabulary knowledge compared to good readers.) Finally, regardless of passage-specific background knowledge, questions about information stated literally in the text were easier to answer than questions that required inference. Results suggest that multiple forms of knowledge, both passage specific and general, are likely required to form coherent and high-quality representations of text.

In my last post, I described conversations with three teachers that revealed their different views about what teaching is.

The most persuasive was a teacher who focuses on retention—and thinks teachers are making a mistake when they change topics as soon as they see that students have comprehended the topic at hand.

As we spoke, I thought about what happens to me as I listen to NPR. Even when I find a story really interesting, I’m only able to remember it well if it is on a topic I already know well. Most of the time, the stories are on things I only know a bit about. If I try to retell them, the details are fuzzy; I mix up the key people and events and can’t convey much. It’s an odd feeling—I fully comprehended the story at the time, but I don’t realize how little of it I’ve retained until I try to tell a friend about it.

To really learn the story, I’d have to comprehend it, then study it—quiz myself, practice those details that make the story coherent, and quiz myself again. I’d also need to revisit the material periodically—hopefully adding to it, but at a minimum refreshing my memory. That’s the type of learning that would enable future learning, including deeper comprehension each time new details are added to the web of knowledge growing in my long-term memory.

The retention-focused teacher I spoke with was very intentional about her instructional time. She argued that if a topic was worthy of mentioning, it was worthy of fully teaching—teaching so students could confidently talk about their new knowledge. She saw the school year as far too short, and each class as a precious resource to be fiercely protected. She saw instruction aimed at coverage and even comprehension—anything less than retention—as a waste of time. And, she accepted that her approach meant that she taught fewer topics, and thus had to carefully decide which topics merited class time.

One great benefit of this careful weighing of topics was that she had gotten really thoughtful about embedding skill development in serious academic content. While some of her colleagues taught skills with “fun” content, she eschewed that as inefficient. For example, she taught grammar with sentences that refreshed students’ memories on key content they were learning in science and social studies—no grammar lessons with sentences about basketball or cartoons in her classes.

Reflecting on our conversation, my mind returned to Daniel Willingham’s article on familiarity vs. recollection. Along with that article, he has several useful tips for ensuring that students don’t mistake their familiarity for real learning. His tips focus on “jostling students away from a reliance on familiarity and partial access as indices of their knowledge, and encouraging (or requiring) them to test just how much knowledge they recall and understand.” He recommends, for example, that teachers “Make it clear to students that the standard of ‘knowing’ is the ‘ability to explain to others,’ not ‘understanding when explained by others.’”

This pretty well sums up what the retention-focused teacher I spoke with learned over many years of teaching. So it raises a question for another day: why didn’t she learn about the dangers of familiarity (or mere comprehension) and the necessity of recollection in her preparation program or in ongoing professional development?

Writing is a great way for students to explain, solidify, and gauge what they have learned. (Image courtesy of Shutterstock.)

Reading Recovery Works—Now Let’s Make It Even Better