Isobel Stevenson

Thinking about Teaching and Learning

The Classroom Experiment

March 30, 2013 by · No Comments · Student Achievement

Dylan Wiliam, the guru of formative assessment, has a new book out called Embedded Formative Assessment.  Additionally, and more entertaining I think, he collaborated with the BBC to produce a documentary in two parts called The Classroom Experiment.  This was shown in Britain quite a while ago, but I just found that the two episodes are available on YouTube at the moment.  I don’t imagine this is legal, but I’m not complaining.  I think it’s great stuff.

Part 1   http://www.youtube.com/watch?v=J25d9aC1GZA

Part 2    http://www.youtube.com/watch?v=1iD6Zadhg4M

The documentary shows Professor Wiliam going in to a comprehensive school in Hertfordshire to work with the teachers there on the implementation of strategies for improving student achievement.  Featured are: the use of randomizing techniques for questioning students, daily exercise, constant monitoring of student understanding, and feedback instead of grades.

There are several things I really like about The Classroom Experiment.

  1. It is really gripping.  I initially was thinking of using clips from it to illustrate a couple of different projects I’m working on, but it really has most impact as a story, an unfolding of events.  I got really caught up in the drama of how the various characters (teachers and students!) deal with the disruption to their routines, and more significantly, to their understanding of what school is about and what roles they play in that.
  2. It shows the teachers really struggling with implementation.  This strikes me as much more real than most of what we generally get to hear or see about innovation, which always sounds so easy and straightforward (yes, all you have to do to meet the needs of all students is differentiate, or of course, you just need to teach using cooperative learning).  These teachers would, I am sure, have given up their attempts to implement if Professor Wiliam had not been able to step in and tell them why they were struggling and suggest what to try next.  The math teacher is particularly hard to watch.  Her first attempt at using the colored cups is a disaster, as the kids are trying desperately to tell her that they have no clue what is going on in her classroom, and she gives up in disgust.  I always think that our usual pathetic attempts at professional development for teachers are like giving them one skiing lesson and then asking them to ski down Everest.
  3. It shows the impact of changing classroom practices on the students.  I always think of classroom formative assessment as benefiting the lowest achieving students most, but this documentary also spends quite a lot of time with the higher achieving students, and their reaction to the change is fascinating.  Some of the interviews with the students show them being incredibly insightful about how classrooms and education in general work and what impact it has on them.

Please watch!  Let me know what you think.

Educators and Research

February 18, 2013 by · No Comments · Student Achievement

The number of educators who work in pre-K-12 schools who read research articles is infinitesimal.  The number of educators who read books about education, or journals aimed at a practitioner audience is much larger.  Even among this audience, they subject what they read to a credibility test based on their own experience of what makes sense, what is likely to work, and what is simply not reasonable.  The prevailing attitude is that they read to keep up, to know what is current; and at the same time, while they are reading they are gauging the plausibility of what the author has to say.  If they come to the conclusion that the author “has no idea what schools are really like” or is (the worst insult) “clueless”, then they pay no more mind.

There does not appear to be much doubt that there is a gap between research in education and the practice of educators.  This topic has, in fact, been the subject of many books and articles.  There is also widespread consensus, at least among educators, as to why this divide exists.  Here is a summary, taken from my own experience but also including what others have written about the topic.

They don’t have time

Occasionally I read our local newspaper online, and an article about education, particularly when it comes to contract negotiations and ipso facto, money, is liable to spawn a colony of comments about how much teachers get paid considering how much time they get off in the summer.  I would like to see students in school for more days during the year, and I would also like to see teachers be paid more, but both of those opinions are beside the point that during the school year, teachers are pressed for time.

When I first considered moving into a job that was not a classroom teacher, the position I was contemplating was grant-funded, and therefore I faced the prospect of giving up my contract with my school district and the security that represented.  I remember going for a walk with my husband and our dogs and trying to lay out the pros and cons for him.  All he would say was, “yes, but will you have to grade?”.  I don’t think it was until that walk that I fully understood the extent to which being a classroom teacher consumed my life outside school, and I would freely admit that all the jobs I have held since leaving the classroom have all been easier than being a classroom teacher.

Nobody should be surprised that, given the myriad demands on their time, educators do not make reading research a priority.  As a principal once said to me, “I’m too busy doing it to read about it.”

There is too much research to keep track of

Even if teachers did have time to read the technical literature, how would they decide what to read?  Mountains of articles are published every month.  The American Educational Research Association alone publishes three journals, and then there are the more specialized publications in literacy, mathematics, and so on.  The Social Science Citation Index (SSCI) catalogues over 150 education journals.  A hundred and fifty!  You couldn’t possibly keep track of all those, even if that was your full time job.

Researchers in academic positions have the luxury of a considerable amount of specialization; a historian will have a specialty like the Civil War or Ancient Egypt, and a geographer may be an expert on arctic biomes or how tragedy affects a place.  The same is true in education, where a researcher in mathematics education probably knows only a limited amount about second language acquisition.  A fourth grade teacher, on the other hand, has to know something about everything.

The problem of volume leads teachers to a couple of practical solutions.  First, there are publications that feature articles similar to news stories: they are current, they focus on personal experience, and they are written for a general audience.  Educational Leadership is a great example of this kind of publication, and with a circulation of over 150,000 (not to mention those copies that are passed from teacher to administrator to teacher, or read by many teachers in the teachers’ lounge), it is obviously a very important source of information for educators.  Second, there are books that take a very practical approach to making research accessible to educators; for example, (Hattie, 2009; Marzano, Pickering, & Pollock, 2001).  These researchers have perfected the art of the meta-analysis: taking vast numbers of research studies and drawing conclusions about what each individual mote of research contributes to what we know about good practice.

Previous experience with the uselessness of research and theory

When I talk to principals about what they are looking for in terms of instructional practices when they visit classrooms, and how they know what to look for, they almost invariably cite their own experiences as teachers and as children as the reason.  Equally, they are apt to talk about the lack of utility of teacher training and principal preparation programs as being overly focused on theory and insufficiently practical. One principal described his teacher training as being like learning to swim by being told what the back stroke is and when you would use it, without ever getting into the pool.  Then when he got a job teaching in inner city Chicago, he jumped into the pool and, in his words, sank like a stone.

I know from personal experience that some teacher training programs are better than others at weaving together research and practice, for I have gone through student teaching twice.  I imagine having readers whose first thought is that I flunked out the first time, but happily, this was not the case.  In fact, when I moved to the USA, my teaching license from England did not transfer, as I had had too little classroom experience as a full-time teacher.  I recall that I needed two years of experience, and I had only one.  I was required, therefore, to take certain classes and to repeat student teaching.

Although I didn’t know it at the time, my first teacher training experience was in a program that had been explicitly and thoughtfully designed to encourage reflective practice in its participants.  It was a rather odd experience to read about my experience in quite clinical terms in a book about reflective practice a quarter century later (McIntyre, 1993).  In contrast, my second teacher training experience was perfectly traditional: theory followed by application.  That methodology was behind the times 25 years ago, yet is still the experience of many aspiring teachers.

The fact that educators do not associate research with practice is not the fault of the researchers.  Nevertheless, the lack of connection seems to have made them think of research as existing almost in opposition to practice, and this of course is unfortunate for the profession as a whole.

Hattie, J. (2009). Visible learning: Routledge.

Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2001). Classroom instruction that works: Research-based strategies for increasing student achievement. Alexandria, VA: ASCD.

McIntyre, D. (1993). Theory, theorizing and reflection in intial teacher education. In J. Calderhead & P. Gates (Eds.), Conceptualizing reflection in teacher development. London: The Falmer Press.

 

 

How to give feedback

November 8, 2012 by · No Comments · Student Achievement

Here’s a nice little story from Tim Harford’s blog.  He’s worth reading, not least because he writes a lot about learning from failure.

I recently spoke at Wired 2012 and I felt it went well (video to follow, when they put it up).

Afterwards, people came up, shook my hand, patted me on the back and told me I did a great job. That felt nice, but it won’t help me to do a better job next time.

Elsewhere in the building, other people gathered in corners and grumbled about all the things I did or got wrong. (I don’t know if this happened. I assume it did. You can’t please everyone.) That didn’t help me to do a better job next time, either.

But someone did something helpful. Bruno Giussani of TED, seeing someone praise me for speaking without slides,  immediately got to the point. “You talked about the Spitfire,” he said, “But this is an international audience. Many people won’t know what you’re talking about. You should have shown just one slide: a photograph of a Spitfire. Then everyone would have understood.”

Next time I give a similar speech, I’ll be showing one slide: a photograph of a Spitfire.

It isn’t easy to get straight to the point and offer a single, focused suggestion for improvement. And the truth is, we rarely seek that kind of feedback. When we ask “what do you think?”, we’re usually looking for those confidence-boosting pats on the back. But giving such feedback – and seeking it out – is hugely important.

And here is a photograph of a Spitfire.

 

A brief thought about mission statements

August 22, 2012 by · No Comments · Student Achievement

The mission of the school is the shorthand for the meaning that the people in the school attach to their work.  Several writers have made the point that there is a difference between a mission statement and a mission.  During a fabulous summer camping trip, I was in one of the gift shops in Mesa Verde, and one of the little cards in a display case described the significance of a fetish, and how they are accorded a power that the physical object itself does not possess.  If you don’t believe in the potency of a fetish, then the object is more accurately described as a carving.  I think this is a great metaphor for a mission statement.  If you don’t believe in the power of the mission that the statement represents, then the statement is a poster, and not a mission.

RTI

July 31, 2012 by · No Comments · Student Achievement

I keep, and look at from time to time, a memo I wrote more than ten years ago to the higher-ups at the district where I was at that time a principal.  I wrote it because I and the special education teachers with whom I worked knew that the special education system was not working the way we thought it should.  The memo asked for FTE to support students who needed extra help in reading without qualifying them for special education services.

We wanted to do this for two main reasons.  First, many of the students who needed help were what we called “gray area” kids.  They scored below average on IQ tests, and their achievement was also low, and therefore the magical one standard deviation discrepancy was not in evidence.  Second, we didn’t like the idea of labeling students as broken even if they did have scores that met the magic formula.  Ascribing that discrepancy to a disability seemed an inferential leap of daunting proportions that had implications for the student well beyond the initial staffing.

The meeting to talk about the memo didn’t last very long.  We were told that that wasn’t the way the funding worked, and that there wasn’t anything to be done.  I look back at that now and I’m a little disgusted with myself.  I know so much more now about, well, everything.  How special education functions as a safety valve in an unforgiving system.  How to teach adolescents to read—we really didn’t do a good job with that 10 years ago, and we do a much better job now although we still have a long way to go.  How deeply detached most regular education teachers are from special education, and vice versa.  I should have tried harder, but I didn’t know what to do.

Times have changed.  The discrepancy formula is on its way out and my state, Colorado, is ahead of the curve in requiring that all districts have a plan that makes them ready to do away with the discrepancy formula altogether and replace it with RTI (I know that the American educators reading this will all know what I mean, but if not, see http://www.rti4success.org/.  So are we in good shape?  Not necessarily.

While working on RTI with a group of educators, all from the same district, I asked them to bring in RTI materials from their respective schools—these materials could have been a formal plan for how the process operates in their school, forms to be filled out, lists of assessments to be given or interventions to try.  What I asked them to do was to sit in groups, review the materials brought by everyone in the group, and to rank order the schools according to the most successful RTI process in place.

Each group went through the materials from each school.  From looking at the materials, what they knew to be good practice in RTI, and from their own experience, they created attribute charts of what an ideal RTI process would look like.  From that, they were able to evaluate each school’s RTI plan, and produce a kind of league table of schools according to where their processes fell on the attribute chart.

When they were done, I asked them whether they were able to produce a ranking, which they did.  My follow up question was: “So, let me get this straight.  The school that you’ve ranked number one has the process that creates the best outcomes for kids, right?”  Silence.

The educators, being a thoughtful and experienced group, saw immediately that they had done what is often done in education, but that we are not supposed to do: use process as a proxy for outcome.  I worry that, in setting up elaborate RTI models, we will miss bigger questions, like, are we doing this to fulfill a mandate, or will kids actually be better off?  Will regular education teachers (and maybe also special education teachers) see RTI as simply the new process for getting kids labeled as special ed, or even the same process with a different name, instead of a chance to think differently about how we address differences in achievement among our students?

So while this is a story about RTI, it is also a story about other things as well, including: we frequently do not hold ourselves accountable for tying our policies, procedures, and practices back to better outcomes for students.  But more importantly, we are often not in a position to see the systems of which we are a part, just like the fish who have never realized that the medium in which they swim is water.

Tags: ··

Reflection, part one

July 29, 2012 by · No Comments · Student Achievement

I am very interested in the idea of reflection, and my conversations with principals lately have prodded me to try and think a little more deeply about what we mean by it, exactly.

First, it is hard for me to tell, when principals talk about reflection, whether they are talking about anything other than running through a predictable set of questions: what went well? what didn’t go well? what will you do differently?  These questions are fairly ubiquitous in education, and are the same as are asked on evaluation forms for professional development for educators and in other similar situations.

It’s not that I object to these questions.  I just don’t think they go deep enough.  How do we promote the idea that reflection also entails asking why?  And actually, it may entail asking why several times.  For example, here is an abbreviated and slightly fictionalized account of a post-observation conference:

What went well?The questioning went well.  All the kids I asked knew the right answer.Was that your goal?

Well, no, the goal was to introduce chapter two of the book [we will leave aside for the moment the question as to why that was the goal of the lesson].

So why is it a good thing that the kids already knew the answers?

Well, it meant that the lesson went smoothly.

Was that your goal?

Well, yeah, actually, it is one of my goals that the lesson go smoothly.  Of course.

Why is that?

Are you serious?  It’s less stress, I can get through what I need to get through, all those things.  Plus, if my principal walks in, I will get an attaboy if all the kids are answering the questions right.

If you didn’t have to worry about behavior, would it still be your goal that the kids knew the answers? [tempted to add, before you taught it, but decide that's too leading...]

Long pause.  No, not actually.  I would love to be in a situation where the kids could get a question wrong and they wouldn’t get made fun of and I would know what to do.  But that’s not how things work in my classroom.

Would you like to talk more about how to make that happen?

 

OK, there are lots of interesting things about this little vignette.  One is that it is a conversation—perhaps with a coach—and not what we generally think of as reflection, which is an internal mental process.  But there is no reason why that same conversation couldn’t have been an internal one, and I think that the goal of coaching is to model the asking of reflective questions, so that the person can internalize the thinking and prodding behind those questions.  In other words, the goal of coaching is to scaffold reflection with the aim of teaching self-regulation in the practitioner.

Second is the asking of why questions.  There are really good reasons for not asking why questions:  see this article, called Ask What Not Why.  The basic idea here is that being asked why tends to put a person in a position of rationalizing or justifying.  In other words, a person is prompted to defend a position, an action, or a belief by being asked why.  The preferable alternative is that a person is prompted to think deeply about that position, action, or belief, thereby becoming more self-aware and more able to self-critique in the future.  So I could rewrite the questions so that the word why is never used.  For example, instead of asking “Why is that?” I could have asked “What about that is so important?” or “What led you to pick that as a goal?”  The underlying question is the same, but I think a lot of how you ask the questions depends on who you’re working with and what kind of relationship you have. At the same time, the act of asking why questions should not be construed as a negative judgment on the part of the person asking the questions.  I left the whys in this vignette to highlight the times when there is another layer of cause and effect being explored, even though in real life (whatever that is) I may have asked a slightly different question with the same end in mind.

Third, and I think most interesting, is that this conversation, and the potential for future action, came from a response to a question about what went well? rather than what would you do differently? I think that we often take the what went well at face value, and don’t investigate any deeper.  I think this comes from a particular interpretation of the term reflection.  We frequently take it to mean that we will perform a post-performance review, in search of lessons learned.  And it’s not that that’s not important; it’s just that that’s not all there is.

Instead, we should take the opportunity to look for the theory of action (I realize I need a blog post devoted to theory of action) behind our action, and look at the assumptions on which the theory of action rests.  If you look back at the vignette above, it is replete with assumptions on the part of the teacher about the role of the teacher, the administrator, and the student, and indeed the nature of learning.

At the same time, the teacher being coached was quick to catch on to the idea that if he (although it could just as easily be a she) were not concerned about keeping the students under control, he would think differently about how to teach them, and therefore he is perfectly capable of re-conceptualizing what it means to teach.  He can ask and answer the questions:

  • If I do this, what do I anticipate will happen?
  • Why do I think this (in other words, what is my theory of action)?
  • What assumptions do I have that create that belief?

The whole idea of theory is not something that we talk about much in education, because educators see themselves primarily as practitioners.  There are very good reasons for this, mostly having to do with the unfortunate reality that their experience with theory has been detached from the actual practice of educating students.  But I think that the true meaning of reflection is to attach theory to practice, to generate explanations for why we behave the way we do.  This gives us the occasion to examine our thinking, to elaborate on our theories, and most importantly, to test them.

Tags: ·

Mystery v. Puzzle

July 29, 2012 by · No Comments · Student Achievement

I’ve been reading a collection of Malcolm Gladwell’s essays that I’ve owned for a while but just got around to looking at.  He has lots of clever ideas, but the one that has caught my attention so far is the distinction between a mystery and a puzzle–an idea drawn to my attention by my friend Patty Kipp, so it was fun to read Gladwell on the notion.

A puzzle is something that can be known with enough information.  A mystery is not like that: the solution is not easily predictable.  Gladwell’s example to show the difference is about Osama Bin Laden, written of course before Bin Laden was located and killed a few months ago.  Where Bin Laden was hiding was a puzzle.  We did not know where he was because we did not have the information.  But what the world will be like after his death is a mystery.  There is no crucial datum that will provide us with that answer.

There are some other ideas that go along with the mystery/puzzle distinction.  A puzzle has to do with deduction, and a mystery with inference.  A puzzle is about how much information is made available, and a mystery is about what we do with that information.  Gladwell makes the point that we often do not make the distinction that we should between the two, and are less productive as a result.

This distinction seems to me to be useful in many educational settings, particularly when it comes to teacher evaluation.  We think that the mystery of what makes a good teacher has to do with getting more information, so we go out and collect more data about teachers and teaching.  But this is the easy way out.  What we should be doing is making more out of the information that we do have, which we don’t do remotely well.

Looking at Student Work

July 12, 2012 by · No Comments · Student Achievement

I was in a workshop once and the presenter said (I’m paraphrasing), “Why are we even observing in classrooms any more?  The indicator of whether effective teaching is going on is not what the teacher is doing, it is what the student is learning, and that should be visible in the work the student does.  Instead of watching teachers teach, we should be spending that time looking at student work.”

The idea of looking at student work is the same as Rick DuFour’s questions, what do we want them to know, etc.  Looking at student work tells us whether they got it or not, and then we figure out what WE have to do differently according to what answers we get from our investigation.

I agree totally that the measure of effective teaching is in what the student learns, and not on what the teacher is doing.  As we used to say, if they didn’t learn it, you didn’t teach it.  The learning is the measure.

At the same time, I go back to the idea that people are doing the best they know how to do, and this particular model of looking at student work that negates the value of classroom observation makes the assumption that teachers will know what to do differently as a result of looking at student work, and often they just don’t.  Maybe they’re brand new teachers, and my conversations with principals and teachers over the years tell me that teachers enter the profession with very little knowledge of some critical aspects of teaching and learning, and very limited experience of how to make things happen in the classroom.  They have had little opportunity to put into practice the theories they learned in teacher training.  Maybe they are being asked to teach a subject for which they have had little preparation; there is a big difference between being highly qualified in a subject and knowing how to teach it well.

Looking at student work as a substitute for observing in classrooms is inadequate because it means that, first, the diagnosis of what is going on is delayed until there is student work to look at.  It’s like waiting to go to the doctor until you’re sick, rather than going in for your regular preventive check-ups.  Second, the diagnosis is limited to self-report, like not letting the doctor take your temperature or blood pressure, let alone examine you.

Dear Jeff

October 28, 2011 by · 1 Comment · Student Achievement

Dear Jeff,

It was a pleasure to meet you and get the chance to talk education.  Your question about the absolute best two or three books in education that I’ve ever read has really got me thinking.  It is a difficult question because the books that had the greatest impact on me are now out of date, I think, so I could not recommend them now for someone else to read.  Just for the sake of completeness, I would include in that category:

  • the Horace books, by Ted Sizer (I know, that makes them sound a bit like Harry Potter, but for me they re-defined what student engagement might look like).
  • The World We Created at Hamilton High.
  • The Unschooled Mind, by Howard Gardner. I am not an uncritical Howard Gardner fan, but this book really made a big impact on me because it challenged my idea of what it meant to really understand something.  I heard Gardner speak at a conference in Boston around the time this book was published, and he talked about the interviews that were taped of Harvard students on their graduation day, during which they were often spectacularly wrong about why the Earth has seasons.  He made the point that we often ask students a question that in effect asks them to parrot back to us what we have told them, and we think that this denotes understanding.  If we change the wording so that they have to draw on understanding of a concept rather than what we told them, it is a better indicator.  I was teaching seasons to 9th graders at the time, so I changed the wording on the test.  And guess what?  You guessed it.  That experience changed the way I taught and was, looking back, my first foray into formative assessment.  If you want to see the Harvard grads on YouTube, click here.

Then there are the books I read when I was making the move into administration.  One of those books made a big impact on me, which I mentioned yesterday, and which is very much still worth reading:

  • The Fifth Discipline, by Peter Senge.

Now that I have read more in the field of organizational learning, I would recommend other books that preceded Senge’s work, particularly the work of Chris Argyris and Donald Schön (sometimes together, sometimes individually, and sometimes with other authors).  This goes back to what you and I talked about: that sometimes just because a book is recent doesn’t mean it’s best, and often it pays to trace back intellectual ideas to their source.  In that vein, I would recommend:

  • Overcoming Organizational Defenses
  • The Reflective Practitioner
  • Action Science
  • Theory in Practice: Increasing Professional Effectiveness (this is the book I find most challenging; it challenges common notions of what it means to act with integrity, and it makes me think harder about my own personal and professional practice than any other book I own).

I think you were particularly interested in books about instruction, and what I realized was that there are no books I would recommend on instruction.  But there are articles, and particularly, videos and podcasts.  My own strong conviction is that the research on formative assessment and self-regulation has the most to offer for increasing student achievement.  We often talk about wanting students who are self-directed learners, but our classroom practices often do not actually support the creation of self-directed learners.  So here is my list of resources to consult on this subject.  First, the articles that I consider foundational, cleverly linked to PDFs in my Dropbox:

But honestly, some of the best resources are available on YouTube.  In particular, I would recommend the snippets of Dylan Wiliam from a BBC series on improving instruction in an English middle school.  They are not very polished clips, but they will show you what is wrong with grades and asking for volunteers, and show you how teaching techniques could be better.  Check out:

I think Dylan Wiliam is fabulous.  A clear thinker, speaker, and writer.  I also recommend a chapter he wrote.  In fact, if you read nothing else, read this:

And if you really want books on what good instruction looks like, I would recommend:

  • Teach Like a Champion
  • Visible Learning
  • The Seven Strategies of Assessment for Learning

I look forward to our next conversation!

Yours, Isobel

Tags: ·

The way you think changes the way you act

October 6, 2011 by · No Comments · Student Achievement

In class tonight, we played a game.  Great exercise, and I think everyone involved learned a lot.  My favorite part was when someone pointed out to me that you could think of the game as a game, or you could think of it as a simulation, and it would change the way you played it.  Game implies competition, comparison to others, numerical targets, and so on.  Simulation has a different set of connotations, including strategy, and analysis.  I immediately made the connection to what I know about performance goals and learning goals.

You tell people that the goal is to meet a target and they don’t know everything they need to know to meet the goal, and they scrabble around taking shots in the dark to see what works.  You tell them instead that their goal is to learn what strategies are most effective in whatever the purpose of the simulation is, and they are more deliberate, they take their time, they evaluate what strategies work and why they work, and they learn how to be effective.  And of course the irony is that they typically reach a higher performance than the people who were focused on the target.

So one lesson from the game is that learning goals are more powerful than performance goals when you don’t already know how to reach a target.  And then there’s the bigger picture lesson, that they way we conceptualize a situation dictates to a large extent how we behave in that situation.  And we get to choose that.  We get to choose the theory that gives us the most control.  I love it.