Saturday, February 13, 2016

What if Multiple choice questions allowed teachers to SEE their students' thinking instead of just the purported result of that thinking?

An interesting thought.

A couple disclaimers:

  1. It is late at night/early in the morning, and I am a bit fogged because of some cold-symptoms...the chance of this post containing non sequiturs and general rambling is high
  2. I do not have an answer to this question, so the following post is more of a thought experiment. If YOU have the answer, share it by commenting on this post.
Here is what made me think about this question in the first place.  I like the Mastery Connect tool, but I know that many are still warming up to it.  Part of my role this year is to check in with teachers about their use of assessment data to drive their instructional choices, and in doing so I get to hear some interesting feedback.  

During one conversation a math teacher bemoaned (rightfully so) the seeming dependence on multiple choice questions as the method of assessment.  Yes, there are other types of questions, as you all know by now, but when it comes down to a quick turn around a solid multiple choice question is hard to beat.  What I believe this teacher was responding to, was the way in which a multiple choice question's response is so easily divorced from the the thinking that ostensibly lead to it.  Was it a guess? Was it a happy accident?  If wrong, why?

At present (so far as I know) Mastery Connect does not allow you to capture student's work alongside their multiple choice responses.  You may be thinking, "so what? You can always collect their 'scrap' paper or test booklet/sheet, and review that for a more robust picture of their current state of mastery."  You are certainly right, but what if you did not have to, or more to the point...did not have time to?

Ok, I am going to jump to an idea I have been playing with in my head for a little bit.  It is important that you do not shuffle the story you just read off to your long or short term memory, but that you keep it in your active working memory so that you can see the connection (that I hope I am able to make).

I like small groups, and I think most do.  As a teacher you can really be efficient with you instruction, targeting precisely what is needed and by whom.  It seems to be a win win.  Lately, I have been playing with mid-class checks for understanding: little mini-quizzes that might seem to have a more likely home at the end of class in the form of an exit ticket, a mid-ticket, if you will.

They are uniformly short, so that the teachers can quickly score and sort them into groups based on the results.  To do this requires very precise questioning so that you can be sure that when a student misses question 1-3 they need the type of intervention being offered by small group A, whereas students missing questions 4 or 5 need what the other small group is offering.

This works well when there are multiple teachers in the room, and allows misconceptions to be cleared up right away so that students do not lose too much ground while they struggle until that difficulty is noticed.

What if I did not have a co-teacher?  Sad, yes, but not the end of the mid-ticket, for I can use Mastery Connect to deliver the assessment digitally.  That, in turn, allows students to see their level of mastery immediately upon submission.  If I set some criteria for 1-2 small group stations in the room (ex. re-teaching, partnered, and independent) students can self-sort on the basis of their own data. 

Now for the acrobatics.  Put the two stories together; that of the teacher wanting to see student thinking, and of the teacher trying to create data-based groupings on the fly.  

What if Multiple choice questions allowed teachers to SEE their students' thinking instead of just the purported result of that thinking?

Suppose you created a short assessment wherein each multiple choice question surfaced a student's knowledge of the process. The question is not about the final answer, but about a step that would need to be taken in pursuit of that answer.  In short, their thinking. Here I am envisioning the exploded views brought to us by the world of technical drawing.

Think of a multi-step problem.  Each step can be demanded in the form of a multiple choice question, and can be done in a few ways.
For example:
  1. Take one problem and explode its steps into the appropriate number of questions and:
    • ask students to choose the response that best shows what the proper step/work looks like
    • ask students to identify the statement that best describes the step necessary to solve the problem 

Directions: Your answers to the following questions will demonstrate your knowledge. Each one represents the completion of a step in the process needed to solve the equation below:

4x - 12 = 12

Question #1:
After completing the FIRST step in solving the equation you will be left with which of the following:
a.) 4x + 12 =  0
b.) 4x = 24
c.) x = 6
d.) -12 = 12 - 4x

Question #2:
After completing the SECOND  step in solving the equation you will be left with which of the following:
a.) 4x = 24
b.) 0
c.) x =6
d.) 4 = 24

Question #3:
After completing the THIRD  step in solving the equation you will be left with which of the following:
a.) 4x = 24
b.) x= 6
c.) 24/x = 4
d.) x = 3


Directions: Your answers to the following questions will demonstrate your knowledge. Each question will ask what is needed next to solve the equation below:

4x - 12 = 12

Question #1:
What is the FIRST step that you should take?
a.) Isolate the term 4x by adding 12 to each side of the equation.
b.) Divide each side by x  remove the variable from the left side of the equation
c.) Subtract 12 from each side so that you have zero on the right side
d.) Multiply both sides of the equation by 4 to get rid of the variable

Question #2: 
What is the SECOND step that you should take?
a.) Multiply both side by 4
b.) Divide each side by 4
c.) Divide by zero
d.) there is no next step

Question# 3:
Finally, what is the correct answer?
a.) x = -12
b.) x = -6
c.) x = 6
d.) x = 24

I am certain that there are many more ways to ask students to demonstrate their knowledge of processes and, correspondingly, their thinking.   A math teacher could differentiate the same assessment by making it visual and actually providing students with the "notebook" page that SHOWS a problem in various states of completion, and thereby eliminate the classic barrier to math knowledge demonstration that literacy often presents.

This approach of taking an exploded view of a mental process is immediately applicable in science curriculums wherein students must follow procedures, or conduct observations and experiments in a specific sequence, but how does it translate to the more nebulous world of humanities?

Again, I am not sure that I have an answer, but I think that in classrooms all over the country ELA teachers are instructing students to analyze text in specific ways, and the same is true for writing. After all, much of teaching ELA is about slowing down the mental processes our adult brains go through as we analyze text and modeling that for students in the form of discrete steps.  Therefore analyzing the development of Jean Louise Finch is as much a product of systematic thinking as the graphing of linear equations.

There is simply no way that this is the first time something of this kind has been written, it is even more unlikely that I am the first to have thought about this, so please help push my thinking further and add your comments to this post.

One last disclaimer:

3. Somehow I have been listening to Conway Twitty the entire time I was writing this...especially odd, since I have only three songs of his mighty catalogue.  Doubly odd that I only recognized this here at the end. I don't know what happened.

What's that Darlin'
How am I doin'
Guess I'm doin' alright
Except I cant sleep
And I've blogged all night 'til dawn.


  1. I'm going to have to re-read this when my brain has fully rebooted, but I think this is a great idea. I'm disappointed in myself for not having thought of it. Good job
    Mr. Kehoe! I'm trying this as soon as I get back. I do wonder though, if perhaps this sort of choose- your-path form of assessment can lead a student. Regardless, it will reveal a lot more than the standard multiple choice assessment. Another way in which this model may be interesting is to see how students use scores as feedback and as a way to go back and "debug" their work. If they can see the kinks in their process, the exact moment that he or she went wrong, that definitely will help them to be more metacognitive in looking at personal work. We need to set it up so that students are able to log in to mastery connect and view assessment results. Can we make that happen soon? Thanks Paul. -d

  2. Well, just when I say I'm not sure I believe there is a step by step process to text analysis I recognize I am always telling the kids, "Here's what I ask myself to answer this question?" Still, would like to play with creating some of these questions. The most shocking thing I've just read, however, is you listen to country music!

  3. By a step by step I meant "one step by step"

  4. I think this is a great idea! I'm going to try this in math with some of my students that feel paralyzed by the problems. It may give them some confidence in knowing that they can start a problem, but can get a little confused at a specific part. Then, knowing what they need to ask for specific help on. Instead of saying, "I don't know what to do." They can say , " I understand how to get to step two but after that I'm not sure what to do." This type of one to one conference would be more helpful to everyone in the room because it could be a quick fix or you could direct a tablemate to assist, or gather a small group of other students struggling with the same part. Thanks Paul for sharing your thoughts.