Motivating with Data

Here is a chart that shows how Michelle's students are progressing on their math facts.  The data points represent the total number of items answered correctly by the class on that day's timed multiplication test.  One thing to note is that each student's test is not the same--they're working on different levels of multiplication, and once they pass one level, they move on to the next.  So improving the total number of facts correct is actually quite challenging, and may get harder as the kids get better.  Lee Jenkins argues that this type of a run chart works to motivate 70% to 90% of students, while only 50% of students are motivated by grades.  I'm curious to see how it works out!  Stay tuned for more examples of data in action!


Evan

Fwd: Methodology Momentum

Happy weekend, everyone.

I'm forwarding an interesting exchange I had recently with Lee Jenkins, another Deming/Baldrige advocate.  He has some intriguing ideas about motivation, goals, and grades.  If you have a few moments this weekend, I invite you to read over our email messages (below), and take a look at the images he attached.  Here are the questions I'm mulling over after reading his work:

1) what is the most motivating way of using data?

2) what's wrong with using traditional grades to motivate students?

3) should students be asked to set their own goals?

I welcome your own thoughts on these matters, and I look forward to continuing the conversation about best practice in pursuit of Baldrige-based Quality.

:)

Evan

Begin forwarded message:

From: "Lee Jenkins" <Lee@LtoJConsulting.com>
Date: August 22, 2011 6:41:42 AM HST
To: "'Evan Anderson'" <evananderson2@gmail.com>, "'Amy Vorderbruegge'" <amyvord@gmail.com>
Cc: "'Margaret Byrnes'" <margbyrnes@aol.com>, "'Sue Deuber'" <sdeubie@aol.com>
Subject: RE: Methodology Momentum

Evan,

1.        Of course you can call me Lee.

2.       I do believe you understand the writing on goals.  Pulling numbers out of the air (the normal way of setting goals) is a huge problem.  The only real number is how we did in the past.  Attached is a picture of a school run chart in their second year of what I call LtoJ.  The goal is to outperform last year.  Note they did this on 6 occasions. Every time you see a star, this is an all-time-best, meaning the students answered at least one more question correctly than any prior week in the year.  Kids get this goal, even at a young age.  Students, do not need to set their own goals.  They have their own run chart and have a desire to improve by having their own personal all-time-best.  Another picture is attached which shows a collection of school starts.  Every time an individual student, a class, or a grade level has an ATB, it is recorded on a star, announced over the PA and posted on the wall.  The teachers say this is one of the very best things they have ever done for special education as these students rarely meet goals plucked out of the air, but often can and do improve, and then improve again.

3.       I am not sure you will want to measure everything.  Since the book was published more work was done, however, with dichotomous rubrics which are more precise than the traditional ones.  I've attached one for you to see.  We've created 4, 6, 8 and even 10 point dichotomous rubrics.

4.       In the book Wad-Ja-Get, the authors quote research stating that grades motivate 50% of the students.  This matches information gathered in my seminars.  Further, teachers tell me that the LtoJ quizzes that are not graded motivate 70-95% of the students.  If they are ever graded, motivation goes down.  (We have figured out a way to fairly grade spelling tests so that teachers are not giving two tests per week.  Write if interested)

Best to you,

Lee Jenkins

Lee@LtoJConsulting.com

480-221-7603 (cell)

From: Evan Anderson [mailto:evananderson2@gmail.com]
Sent: Sunday, August 21, 2011 7:05 PM
To: Amy Vorderbruegge
Cc: Margaret Byrnes; lee@ltojconsulting.com; Sue Deuber
Subject: Methodology Momentum

Aloha Amy, Margaret, and Lee-

I've spent my weekend as I spent most days this summer--reading up on TQL/BBQ, to get focused on what really makes a difference for students. Thank you, Margaret, for the opportunity to preview the new edition of your book.  I've also found Lee Jenkins's book Improving Student Learning incredibly instructive, and I'd like to start a conversation with him and Margaret about the topic of goals.  In Jenkins's work, he states that focusing on the goals too much can be a distraction, since what we're really always going for as our aim is getting better, all the time (increase the positives and decrease the negatives).  Target-type goals, then, tend to fall short because they are very likely to overshoot, leading to disappointment, or undershoot, leading to results that are lower than what could have been.  

If the focus is not on goals, then what? Well, Jenkins presents the three cornerstone charts of continuous improvement:  the class line chart, the student line chart, and the scatter chart.  All of these tools are meant to measure improvement. If the tools are used consistently throughout the year to measure class and individual progress toward the mastery of meaningful content, then quality can be built into the process. And celebrations of success can occur every time that line chart moves up, rather than just when (and IF) the class meets its class goal.  I don't believe he's saying "don't have class goals", but I do believe he's saying "don't make class goals the be-all, end-all of your quality processes".  Intriguing, especially considering my spotted history of setting and striving for appropriate class goals in the K/1 classroom.  I also have a hard time holding on simultaneously to the two Baldrige decrees: know exactly what your students should know and be able to do at the end of the year and give students the power to set their own goals.  Again, it may be my K/1 bias creeping in here, but it always seemed insincere to ask the students to set their own class goal when I already know what it is they need to achieve.

A few useful guidelines from Jenkins (may I call you Lee?):

For a year-long course of study (Singapore Math, for example), give a preview test of all content to be mastered this year.  A sample of questions from the end-of-year test is fine--he suggests using a number of items equal to the square root of the total items to be mastered.  For Singapore Math Level 4, for example, the test has 38 total questions (A+B), so the sample would be about six questions.

Then, quiz the class weekly on a randomly selected set of six of these items.  Plot the total number of items correct on the class run chart, and have individual students plot their own run chart in their data folder. Scatter charts can show the distribution of scores and may help in adjusting instruction.

Finally--here's the kicker--DON'T MAKE THE QUIZZES PART OF THE GRADE.  This way, the message is clear: we are all response-able for the success of the class, and we are using the data as guiding feedback for instruction group improvement.  Improvement is grounds for celebration, but lower or higher performance is neither punished nor rewarded. He presents some alternatives for giving grades, though we at Voyager don't necessarily need grades, now, do we? :)  Our narrative can include specifics about areas of strong understanding and opportunities for development, all of which the student will have a firm grasp on, anyway.  Hmmm . . .maybe the student should write the narrative?

One question I'd like to bring up in this esteemed company concerns the "21st Century Skills".  It is easier, it seems, to track progress as a class in an area like high-frequency word reading, or math facts.  However, these things are not necessarily what we think are most important.  The "4 Cs" of Collaboration, Communication, Creativity and Critical Thinking are more slippery, and require more qualitative data.  Jenkins gives an example of an art rubric score run chart that reflects some of these skills, but such rubrics are more subjective, I would guess.  The big question for me about TQL/BBQ education is "how can we use it to measure what really matters?"

If we can answer that question, I think we will be on our way from "Good to Great" in a hurry.

I look forward to continuing our conversation together, as we move Voyager forward. 

Onward Always, 

Evan Anderson

Training and Methodology Coordinator

Voyager Public Charter School

Honolulu, Hawaii

808-389-0270

Twitter: @UnclePapa