Before the meeting the team ensured that all student answer documents had been scanned into Mastery Connect so that we would be working with the most complete data set for the 8th grade team as possible. Our simulated exam targeted many Common Core Standards under the umbrella of Reading Literature and Informational texts. In order to create a manageable data set the team narrowed its focus to the first 23 questions, which were based on 4 passages (3 nonfiction, 1 fiction).
Our expressed purpose in conducting the item analysis was to surface and trends or patterns in our students reading skills that would allow us to strategically address those learning gaps or build on the places of relative strength.
Our meeting began by orienting ourselves to the data in a low inference way, wherein team members (with the Mastery Connect generated report in front of them) shared initial observations and noticings.
A few of those initial observations were:
- There seems to be a big difference between literature and informational (see questions 1 and 3 and 20)
- Question 6, 17
- Question 5 has an interesting pattern
We transitioned from our low inference-inspired observations to a more critical and analytical mode, aided by the "Multiple Choice Item Deconstruction Chart". We followed the protocol and soon discovered that students struggled, universally, on questions that were tied to RL/RI 8.4. These questions are about defining unfamiliar words using context; something that we surfaced as an issue in September after our Baseline exam. Confused, because we triaged that particular issue early (and often), we dug deeper.
Looking at the particular items in question another interesting pattern came to light. Questions that asked students to define unfamiliar words were answered at a fairly high rate. However, also bundled in the 8.4 standards is a focus on figurative language. Soon we realized that the questions that students were struggling with (in all three classrooms) were those that asked students to engage with figurative language, and in some cases using figurative language to identify unfamiliar words (a double whammy).
Given that this pattern extended across the whole grade, we as a team, developed a response that took advantage of a common grade structure that holds one period a week in reserve for independent reading. During this class we decided not only to employ a bit of item analysis with our students, but to also address the way to analyze figurative language. Furthermore, we identified several key lessons in the upcoming unit wherein we can check in on the progress of student's skill in analyzing figurative language. We plan to include several short responses to Laura Hillenbrand's use of figurative language, organize at least one accountable talk about the how figurative language is used to develop the characters in the narrative, and asking students to generate their own figurative descriptions of what they are reading. In this way we will be addressing this instructional need in a myriad of ways and modalities.
Our final response to this discovery was to manipulate the standard itself. Instead of having RL.8.4 include unfamiliar words, figurative and connotative language, etc... we are going to create sub-standards to allow for a more granular collection of data. We have done this in the past with success, and look forward to the ways in which more discrete data will enable more targeted instruction in the weeks to come.