Default image for pages

By Debbie Jenkins

Debbie Jenkins is the elementary curriculum and instruction supervisor of Bogalusa City Schools in Louisiana. This post originally appeared on Amplify’s Viewpoints. 

Learn more about Bogalusa City Schools’ use of Core Knowledge in this video.

There’s an old Barbara Mandrell song that goes, “I was country when country wasn’t cool.” Similarly, I like to say, “E. D. Hirsch was Common Core before Common Core was cool.”

For those who don’t know who E. D. Hirsch is, he is the chief architect of Core Knowledge Language Arts, the reading and language arts program for K-3 that we are using in our two elementary schools. The gains our students have made in just one year with CKLA are just beyond belief. It gives me goosebumps thinking about it.

At our two rural elementary schools, 93 percent of our students qualify for free and reduced lunch. Many of our kids don’t have much of a chance to leave our city of Bogalusa. Their parents would love to give them the opportunity to see more of the world, but it’s just not possible.

As a result, our students have had issues with comprehension because they don’t have a lot of background knowledge or world knowledge to help them. So as they get to the upper grades, they know how to read the words but they don’t understand their meaning.

I’ve been following the work of Hirsch for many, many years, and as he says, a comprehension problem is a knowledge problem. We needed a program to help build knowledge around topics, and so I took a leap of faith by bringing in CKLA. Other than a few charter schools, we were the only public school district in the state of Louisiana to use CKLA, so it was a risk. But it totally paid off.

With CKLA, our students are learning to decode words through the curriculum’s Skills strand, but they’re also learning about topics like the human digestive system and ancient civilizations, as early as kindergarten, through the Listening and Learning strand. Each year, the curriculum builds on what they learned the previous year. So we’re building a foundation of knowledge at the youngest age. You’d think kindergartners wouldn’t be interested in Mesopotamia, but they love it, all of it. They’re just like sponges, taking all of this information and absorbing it.

The progress our students have made in language arts is unbelievable. The year before we had CKLA, 88 to 89 percent of our students hit the reading benchmark. After CKLA, that number jumped to 95 percent. Our teachers had said our kids would never be able to read the readers that come with CKLA. But you see, it’s the Common Core State Standards and you need to raise the bar, and we did, and our kids rose to the occasion. They did read those readers by the end of the year.

Now the state of Louisiana has put CKLA on its “Tier 1” list of curricular resources for ELA and literacy. So we know we took the right leap of faith, and now other schools in our state will benefit from CKLA, too.

My hunch was right: You can’t go wrong with a curriculum that has E. D. Hirsch and his Core Knowledge Foundation behind it. I continue to be one of his biggest fans.

It used to be that advocating for building broad knowledge with a content-rich curriculum in the early grades was a lonely enterprise. No more! Whether it’s the focus on the early word gap or the Common Core’s explanation of literacy or the moral universe bending toward justice, knowledge is finally getting its due.

New reports from the National School Boards Association (NSBA), the Education Commission of the States (ECS), and the National Association of Elementary School Principals (NAESP) emphasize knowledge as a prerequisite to skills. In deference to the nature of the blogosphere, I’ve arranged them from shortest to longest.

In a new blog post and report, NSBA highlights the importance of nonfiction reading. The post takes on three widespread myths about the Common Core: that the standards push fictional literature out of the curriculum, that nonfiction doesn’t help prepare students for college, and that nonfiction is boring. Lovers of history, science, art, music, geography, civics, and Core Knowledge already know these claims are preposterous, but the post is worth a quick read. Here’s my favorite nugget: “Beth Deniell of Kennesaw State observed that the critics of informational reading ‘seem not to have considered that the contextual information students need in order to understand a literary work arrives in non-literary texts.’”

NSBA’s report takes a more data-oriented approach, showing that US students and adults lag behind in information reading ability. It will be eye-opening to anyone who thinks that life-long literacy—the type the enables prosperity and civic engagement—can be built on fiction alone.

For those new to building knowledge and literacy from preschool through third grade, ECS’s report is a great place to start. It moves rapidly through key points on everything from access to preschool and kindergarten to educational quality and continuity to financing and governance, and it offers snapshots of advances made by various states. With a state-level policymaker focus, the report only touches lightly on curriculum, but it does hit on the necessity of carefully sequencing learning experiences:

When children engage in a coherent set of high-quality P-3 learning experiences, the “fade out” effect (i.e., the notion that early gains in learning disappear later in school) is greatly diminished. Aligning standards, curricula and assessments ensures that young children engage in the right sequence of learning experiences at the right time. Alignment also ensures children are working toward building the set of skills and knowledge they will need as they move from a high-quality preschool to a high-quality full-day kindergarten and the early elementary grades. (p. 16)

shutterstock_200477420

Happy reader courtesy of Shutterstock.

NAESP’s report is both the longest and most informative. It’s a real gem for preschool directors and elementary principals. The first two sections—on preschool to third grade continuity, curriculum, and instruction—are especially strong. A few highlights:

Longitudinal studies have shown that an integrated learning continuum for children from age three to grade three contributes to sustaining achievement gains made in prekindergarten programs. (p. 11)

Alignment of standards, instruction, assessment and professional development ensures that students enter each successive grade having the foundation and skills needed to succeed there. Such alignment can reduce unnecessary repetition in instruction and allow for coverage of more instructional topics. A successful Pre-K-3 learning community aligns standards with a sequenced, coherent curriculum that describes what should be taught in each grade and in each subject and makes clear what mastery of each subject means and how it looks. (p. 21)

Learning is cumulative: Early learning facilitates later learning, and children who already know something about a particular topic often have an easier time learning more about it….

Effective instructional leaders support two specific early reading abilities: decoding and comprehension. Decoding is the ability to identify the words on a page; comprehension is the ability to understand what those words mean…. Instructional leaders support teaching that builds comprehension through read-alouds in prekindergarten, kindergarten and first grade, which help children to build knowledge and vocabulary….

Effective Pre-K-3 instructional leaders know that to be successful in a variety of subjects in middle and high school, students also need to build a basis of prior knowledge in science, history, civics, the arts, physical education and social-emotional learning. (p. 22)

E­ffective principals … know that student engagement is essential and that significant learning happens through exploration and play, particularly in prekindergarten and kindergarten. Strategies used to ensure understanding of key content and concepts will, however, change as children progress from grade to grade. For instance, once children enter first and second grade, effective principals know that these strategies shift to more direct instruction, integrated into engaging and dynamic learning opportunities. (p. 23)

To each of these very strong reports, the one thing I would add is domain-based instruction. As the research appendix to the Common Core ELA and literacy standards states, “Word acquisition occurs up to four times faster … when students have become familiar with the domain of the discourse and encounter the word in different contexts…. Vocabulary development … occurs most effectively [when] domains become familiar to the student over several days or weeks” (Appendix A, p. 33). In essence, most vocabulary is not learned through vocab lists, dictionaries, and weekly quizzes. Those things can be useful, but the vast majority of words are learned through multiple exposures in multiple contexts.

The difference between domain-based instruction and widely used theme-based units is focus. While a theme might be friendship and cover everything from family members to pets to pen pals, a domain is much narrower, such as the solar system or early Asian civilizations. The benefit of the domain is that vocabulary and concepts are repeated, deepened, and expanded with a carefully selected set of texts and supporting activities. While a theme might offer a great variety of words and ideas, little is repeated often enough to be learned. A focused domain provides a more genuine opportunity to learn; students get the multiple contexts they need and teachers have several opportunities to differentiate instruction, allowing everyone to master the core concepts and vocabulary of the domain.

Ideally, all children would learn from a content-specific, domain-based, cumulative curriculum that begins in preschool and extends through elementary school. When the preschool is located in the elementary school, collaboration on curriculum is feasible. But coordinating among a disparate set of child care settings, preschool centers, and elementary schools can be next to impossible. When planning together is unlikely, the next-best option is a preschool through fifth-grade program that ensures one grade builds on the next even without teachers interacting. A coherent program can provide continuity in developing language skills, vocabulary, and broad knowledge even as it shifts from a play-oriented approach in preschool to a more academic approach in the upper elementary grades. (Interested? Give Core Knowledge Language Arts a try. Preschool through third grade can be downloaded for free, and several units from grades 4 and 5 are also now freely available.)

Over the past two weeks, we’ve looked the ETS guidelines for fair assessments that PARCC adopted, as well as a sample item from PARCC. Now let’s turn to the “Bias and Sensitivity Guidelines” ETS developed for Smarter Balanced. While I can’t say that ETS’s guidelines for Smarter Balanced contradict those adopted by PARCC, they are different.

In the introduction, validity and fairness are equated: “if an item were intended to measure the ability to comprehend a reading passage in English, score differences between groups based on real differences in comprehension of English would be valid and, therefore, fair…. Fairness does not require that all groups have the same average scores. Fairness requires any existing differences in scores to be valid” (p. 6).

By this logic, since youth from higher-income homes, on average, have more academic and common knowledge than youth from lower-income homes, the test that conflates reading comprehension ability with opportunity to learn is perfectly fair. Valid I can agree with. Fair I cannot.

A couple pages later, further explanation is offered (p. 8):

Exposure to information

Stimuli for English language arts items have to be about some topic…. Which topics and contexts are fair to include in the Smarter Balanced assessments? One fairness concern is that students differ in exposure to information through their life experiences outside of school. For example, some students experience snow every winter, and some have never experienced snow. Some students swim in the ocean every summer, and some have never seen an ocean. Some students live in houses, some live in apartments, some live in mobile homes, and some are homeless.

Even though curricula differ, the concepts to which students are exposed in school tend to be much more similar than are their life experiences outside of school. If students have become familiar with concepts through exposure to them in the classroom, the use of those concepts as topics and contexts in test materials is fair, even if some students have not been exposed to the concepts through their life experiences. For example, a student in grade 4 should know what an ocean is through classroom exposure to the concept, even if he or she has never actually seen an ocean. A student does not have to live in a house to know what a house is, if there has been classroom exposure to the term. Similarly, a student does not have to be able to run in a race to know what a race is. Mention of snow does not make an item unacceptable for students living in warmer parts of the country if they have been exposed to the concept of snow in school.

Let’s pause here: “Even though curricula differ, the concepts to which students are exposed in school tend to be much more similar than are their life experiences outside of school.” Maybe. Maybe not.

It might be the case that all elementary schools teach snow, oceans, houses, races, and deserts. But does Smarter Balanced really test such banal topics? No. As far as I can tell from its sample items, practice tests, and activities for grades three to five, Smarter Balanced (like PARCC) tests a mix of common and not-so-common knowledge. Passages include Babe Ruth, recycling water in space, how gravity strengthens muscles, papermaking, the Tuskegee Airmen, tree frogs, murals, and much more.

The sample items strike me as comprehensible for third to fifth graders with broad knowledge, but I am highly skeptical that we can safely assume that children are acquiring such broad knowledge in their elementary schools.

As Ruth Wattenberg explained in “Complex Texts Require Complex Knowledge” (which was published in Fordham’s Knowledge at the Core: Don Hirsch, Core Knowledge, and the Future of the Common Core), students in the elementary grades have minimal opportunities to acquire knowledge in history and science. Reviews of basal readers in 1983 and 2003 revealed that they contained very little content. This would be a lost opportunity, not a serious problem, but for the fact that elementary schools tend to devote a substantial amounts of time to ELA instruction, and very little to social studies and science instruction. Wattenberg’s table (p. 35) should be shocking:

Grade and subject 1977 2000 2012
K–3 social studies 21 21 16
4–6 social studies 34 33 21
K–3 science 17 23 19
4–6 science 28 31 24

Even worse, Wattenberg found that “When elementary teachers were asked during what time period struggling students received extra instruction in ELA or math, 60 percent said that they were pulled from social studies class; 55 percent said from science class.”

In their home environments, the schools they attend, and the curriculum to which they are exposed, lower-income children do not have an equal opportunity to learn. As Smarter Balanced guidelines state, the assessment is fair “if students have become familiar with concepts through exposure to them in the classroom.” That’s a big if.

Making matters worse, Smarter Balanced (like PARCC) asserts that it’s just fine for some kids to have to learn during the test. Returning to the “Bias and Sensitivity Guidelines” (p. 8):

Information in the stimulus

A major purpose of reading is to learn about new things. Therefore, it is fair to include material that may be unfamiliar to students if the information necessary to answer the items is included in the tested material. For example, it is fair to test the ability of a student who has never been in a desert to comprehend an appropriate reading passage about a desert, as long as the information about deserts needed to respond to the items is found in the passage.

Last week, we explored how difficult it is to learn from one passage and how greatly such test items advantage students who already know the content that the passage is purportedly teaching. Smarter Balanced clearly disagrees with me. Here’s the introduction it its fourth grade Animal World activity:

The Classroom Activity introduces students to the context of a performance task, so they are not disadvantaged in demonstrating the skills the task intends to assess. Contextual elements include: an understanding of the setting or situation in which the task is placed, potentially unfamiliar concepts that are associated with the scenario; and key terms or vocabulary students will need to understand in order to meaningfully engage with and complete the performance task.

Please take a look at the activity—it assumes an enormous amount of knowledge. Even if it did not, the notion of learning and immediately demonstrating ability flies in the face of well-established research on human’s limited working memory capacity. There’s no getting around it: the students with relevant prior knowledge have a huge advantage.

One (sort of) positive note: I am cautiously optimistic that Smarter Balanced’s computer adaptive testing will help—a little. Here’s how it’s described:

Based on student responses, the computer program adjusts the difficulty of questions throughout the assessment. For example, a student who answers a question correctly will receive a more challenging item, while an incorrect answer generates an easier question. By adapting to the student as the assessment is taking place, these assessments present an individually tailored set of questions to each student and can quickly identify which skills students have mastered…. providing more accurate scores for all students across the full range of the achievement continuum.

In a hierarchical subject like math, the benefits of this adaptation are obvious. In reading, adaptation might help, but it might be misleading. Once a student has mastered decoding, what makes one passage “easier” to comprehend than another is driven primarily by the topic. If the student knows a lot about the topic, then factors like rare vocabulary (which isn’t rare to the reader with the relevant knowledge) and complex sentence structure are of little import. If a student does not know about the topic, then making the vocabulary and sentence structure easier will only help a little. The main way in which adaptive testing might be helpful is in varying the topics; “easier” passages would consist of more common topics, while more “challenging” passages would consist of less common, more academic topics. Then, if we examined the results carefully, we might see that a child lacks essential—teachable—academic knowledge.

Yet, I am only cautiously optimistic because the knowledge that drives reading comprehension is accumulated more haphazardly than hierarchically. One can have some academic knowledge while missing some common knowledge. A student whose grandparents lived most of their lives in Greece may know a great deal about ancient and modern Greece and be ready for a highly sophisticated passage comparing and contrasting ancient and modern Greece. That same student may have no knowledge of China, gravity, Harlem’s Jazz age, or other topics that might appear on the test. Without assessing topics that have been taught, I see no way to truly gauge a students’ comprehension ability (or what the teacher or school has added).

To reinforce the most important message—that comprehension depends on knowledge, and thus schools must systematically build knowledge—the tests need to be tied to the content taught or the high stakes need to be removed so schools will no longer take time out of regular instruction for test preparation.

Last week I explored the “ETS Guidelines for Fairness Review of Assessments.” These guidelines were adopted by PARCC, so I decided to take a look at PARCC’s sample items for English language arts. (PARCC is one of the two consortia of states with massive federal grants to create assessments aligned with the Common Core State Standards. Smarter Balanced is the other consortium; ETS developed somewhat different guidelines for it—I’ll take a look at those next week).

The knowledge demands in PARCC’s sample items are very broad, from cougars to Amelia Earhart to DNA testing. While I am happy to see some substantive questions—and hopeful that such test items will reinforce the standards’ call for systematically building knowledge with content-rich curriculum—I worry about the fairness of these assessments given how they are being used.

As I mentioned last week, it would be perfectly fair to have test passages on topics that had been taught. But, since the test developers do not know which topics are taught in each grade, they have to assess “common” knowledge. Due to well-documented differences in opportunities to learn at home and at school, some children know a good bit more common knowledge than others.

Let’s take a look at one of PARCC’s sample items for third grade. Three questions are asked based on the 631-word passage “How Animals Live.” There’s a typical main-idea question paired with a supporting-evidence question, and then a narrower question that assesses the “skills of rereading carefully to find specific information and of applying the understanding of a text.” Here’s the first section of the passage:

What All Animals Need

Almost all animals need water, food, oxygen, and shelter to live.

Animals get water from drinking or eating food. They get food by eating plants or other animals.

Animals get oxygen from air or water. Many land animals breathe with lungs. Many water animals breathe with gills.

Animals need shelter. Some animals find or build shelter. Other animals grow hard shells to protect themselves.

Many words here are undefined: oxygen, shelter, lungs, and gills. Are these words common to all third graders? Probably not, but much of the content is likely familiar to the vast majority of third graders—and perhaps enough content is familiar for most third graders to grasp the section (if not every word). Nonetheless, children who have learned about oxygen, shelter, lungs, and gills start out with a big advantage. They are reading and comprehending more quickly (which is extremely important in a timed test), and they are comfortable as they move into the more difficult content in the rest of the passage.

Here is the second section, and the beginning of the third:

Ways Of Grouping Animals

Animals can be grouped by their traits. A trait is the way an animal looks or acts. Animals get traits from their parents. Traits can be used to group animals.

Animals with Backbones

Animals with backbones belong to one group. A vertebrate is an animal with a backbone. Vertebrates’ backbones grow as they get older. Fish, snakes, and cats are all vertebrates. Vertebrates can look very different.

Let’s ignore the stiff, unengaging style. What really concerns me is the delusion that it is fair for content to be learned and applied during a high-stakes assessment. (As I noted last week, I do not dispute that the assessment is valid and reliable, so my concerns are with accountability policies, not really with this type of assessment.)

Since a definition of trait is given, it’s clear that some significant portion of third graders is not expected to know that word. Now imagine this is the first time you’ve encountered trait and examine the text:

A trait is the way an animal looks or acts…. Traits can be used to group animals…. Vertebrates can look very different.

What is a third grader to make of this? Clearly, vertebrates are not grouped by how they look.

A trait is the way an animal looks or acts…. Fish, snakes, and cats are all vertebrates.

Clearly, vertebrates are not grouped by how they act. How is the backbone (which is not defined) a trait, since it does not seem at all related to how all these animals look or act?

Making matters worse, understanding trait is essential to correctly answering the main-idea and supporting-evidence questions.

shutterstock_57834808

Do these vertebrates look or act alike?
(Image courtesy of Shutterstock.)

The fact is, vocabulary is not learned by being given a definition. Definitions can be helpful, but they are always incomplete. Words are learned through multiple exposures in multiple contexts. Even with simple words, multiple contexts are necessary: What, exactly, makes hoagies and gyros and PB&Js all sandwiches? I can’t even attempt a concise answer—I just know a sandwich when I see one.

Third graders who have had a unit on vertebrates and invertebrates will breeze through this passage; its inadequate definition of trait won’t matter. But students relying on this definition will surely be at least a little confused, possibly totally lost. The assessment will accurately tell us that children without knowledge of traits have limited comprehension of this passage—but it will not accurately tell us anything about their teachers or schools, for no one alerted the educators that the test would measure knowledge of traits.

You might expect to see a headline like this in the Onion, but you won’t. The Onion can’t run it because it isn’t just ironic—it’s 100% true.

A few years ago, a researcher at one of the big testing companies told me that when developing a reading comprehension test, knowledge is a source of bias. He did not mean the obvious stuff like knowledge of a yacht’s anemometer. He meant typical K–12 subject matter.

Since reading comprehension depends chiefly on knowledge of the topic (including the vocabulary) in the passage, the student with that knowledge has a large advantage over the student without it. And since there have always been great educational inequities in the United States, students’ knowledge—acquired both at home and at school—is very strongly correlated with socioeconomic status.

A logical solution would be to test reading comprehension using only those topics that students have been taught. Teachers can do this, but testing companies can’t—how would they have any idea what topics have been taught in each grade? It’s rare for districts, much less states, to indicate what or when specific books, people, ideas, and events should be taught.

Without a curriculum on which to base their assessments, testing companies have devised their own logic—which is sound given the bind they’re in. They distinguish between common and specialized knowledge, and then they select or write test passages that only have common knowledge. In essence, they’ve defined “reading comprehension skill” as including broad common knowledge. This is perfectly reasonable. When educators, parents, etc. think about reading comprehension ability, they do not think of the ability to read about trains or dolphins or lightning. They expect the ability to read about pretty much anything one encounters in daily life (including the news).

I already had this basic understanding, but still I found the “ETS Guidelines for Fairness Review of Assessments” eye opening. Guideline 1 is to “avoid cognitive sources of construct-irrelevant variance…. If construct-irrelevant knowledge or skill is required to answer an item and the knowledge or skill is not equally distributed across groups, then the fairness of the item is diminished” (p. 8). It continues, growing murkier:

Avoid unnecessarily difficult language. Use the most accessible level of language that is consistent with valid measurement…. Difficult words and language structures may be used if they are important for validity. For example, difficult words may be appropriate if the purpose of the test is to measure depth of general vocabulary or specialized terminology within a subject-matter area. It may be appropriate to use a difficult word if the word is defined in the test or its meaning is made clear by context. Complicated language structures may be appropriate if the purpose of the test is to measure the ability to read challenging material.

Avoid unnecessarily specialized vocabulary unless such vocabulary is important to the construct being assessed. What is considered unnecessarily specialized requires judgment. Take into account the maturity and educational level of the test takers in deciding which words are too specialized.

On page 10, it offers this handy table that “provides examples of common words that are generally acceptable and examples of specialized words that should be avoided…. The words are within several content areas known to be likely sources of construct-irrelevant knowledge”:

ETS table 1

Since having good reading comprehension means being able to read about a wide variety of common topics, table 1 seems just fine. But testing companies’ silence about what their reading comprehension tests actually measure is not. They say they are measuring “reading comprehension skill,” but their guidelines show that they are measuring a vaguely defined body of “common knowledge.”

Common words are not common to all. Even “common” knowledge is knowledge that must be taught, and right now—at home and at school—far too many children from low-income homes don’t have an opportunity to learn that knowledge (which is common to youth from middle-class and wealthy homes). That’s why reading comprehension scores are so strongly and stubbornly correlated with socioeconomic status.

These tests of “common” knowledge are accurate assessments and predictors of reading comprehension ability; but they are not fair or productive tests for holding children (and their teachers) accountable before an opportunity to learn has been provided.

If all testing companies would clearly explain that their reading comprehension tests are tests of knowledge, and if they would explain—as the ACT’s Chrys Dougherty does—that the only way to prepare for them is to build broad knowledge, then we could begin to create a fair and productive assessment and accountability system. Before the end of high school, all students should have broad enough knowledge to perform well on a reading comprehension test. But what about in third, fourth, or even seventh grade? In the early and middle grades, is a test drawn only from topics that have been taught in school the only fair way to test reading comprehension? How many years of systematically teaching “common” knowledge are needed before a reading comprehension test that is not tied to the curriculum is fair, especially for a student whose opportunities to learn outside of school are minimal?

The answer depends not so much on the test as on what is done with the scores. If we accepted the fact that reading comprehension depends on broad knowledge, we would radically alter our accountability policies. Scores on “common knowledge” reading comprehension tests would be recognized as useful indicators of where students are in their journey toward broad knowledge—they would not be mistaken for indicators of teaching quality or children’s capacity. Instead of holding schools accountable for scores on tests with content that is not tied to the curriculum, we would hold them accountable for creating a content-rich, comprehensive, well-sequenced curriculum and delivering it in a manner that ensures equal opportunity to learn. To narrow the inevitable gaps caused by differences in out-of-school experiences, we would dramatically increase free weekend and summer enrichment opportunities (for toddlers to teenagers) in lower-income neighborhoods. (We would also address a range of health-related disparities, but that’s a topic for another day.)

In sum, reading comprehension really does rely on having a great deal of common knowledge, so our current reading comprehension tests really are valid and reliable. To make them fair and productive, children from lower-income families must be given an equal opportunity to learn the knowledge that is “common” to children from higher-income homes.

shutterstock_200841857

Reading is always a test of knowledge (image courtesy of Shutterstock).