On outcomes

It’s arguably the most boring part of any course syllabus: outcomes. It’s also one of the most controversial; lots of us, I know, don’t want to be hamstrung by committee-sourced course or program objectives, in part because they seem so broad and vague as to do almost no work whatsoever (“to learn to think critically”; “to learn to write effectively”), and in part because a large part of academic freedom is the freedom to determine the course of a class’s journey on our own. That’s a huge responsibility, but it’s also a core part of what it means to teach at university level. No two classes, even those with the same title, ever look the same. The instructor’s idiosyncrasies, along with the strengths, weaknesses, energy, and willingness of the students, make a university classroom experience what it is.

It sounds idyllic – and at its best it is. But when it’s not at its best, well, it can be terrible. For every professor that shapes a student’s future with an inspiring syllabus and a dynamic personality, there’s a professor who takes the scattershot approach, lectures veering onto wild tangents, no course objectives to be found as tethers to student needs or experience. And then there’s the part where students don’t always know what’s expected of them, even in the best of teaching circumstances, other than the non-negotiable: to show up and look like they’re doing something valuable…

lect2

I know that one of the reasons course objectives are controversial for my peers in the arts and humanities is because the requirement to have them is typically imposed from the top down. Governments tell university administrators, who tell faculty, that we need some centralised measures to ensure we’re on track with broader learning goals. Those goals often feed strategic plans, and those plans lie at the heart of the neoliberal university – where some faculties are typically “winners” (typically not A&H…) while others are not.

Objectives and outcomes, in other words, are not politically neutral things: they form one core part of measurement-based education policy, in which academic labour becomes less and less about engaging in creative research and teaching, and more and more about demonstrating the “impact” of research and teaching in order to justify the “handout” of government dollars for higher education / in the name of what used to be understood as a core public good. UGH.

And yet, from a pedagogical perspective, they make lots of sense.

Objectives and outcomes keep university teachers accountable: not (just) to administrators or governments, but more importantly to our students and ourselves. For those of us lucky enough to be empowered to make our own objectives and outcomes, course by course and program by program, they are exceptional planning tools. We get to think deeply about what it is we actually want our students to do in our courses, and we get to then think about how different lessons and assignments might link up with these stated plans.

scrappy-quilts2

I’ve made a point of foregrounding outcomes (what I hope students will end up with) as well as objectives (things we’ll do together to try to get to the outcomes) on my course outlines for a few years now. I learned their value – as I learned the value of a number of things I previously believed both hegemonic and overly centralising – while teaching in England, where the expectation that everyone will offer clear course outcomes has been moot for some time now. I take students through my outcomes and objectives at the start of every term; I highlight a crucial caveat – you can only expect to attain these outcomes if you “take our course seriously” – and then I invite them each to create an outcome (what I call a learning goal) for themselves and add it to their copy of the syllabus.

I try to keep my outcomes front of mind as I plan assignments and even class lessons. But I have to be honest; once I’ve ticked the box of making my lists of objectives and outcomes I often pat myself on the back, and then sort of conveniently forget about them. I trust that I’ve got such good and clear intentions for each class, of course my assignments and lectures and discussion plans will feed constantly into them.

But do they?

Last December I decided to test my capacity to teach to my own stated goals by asking the students in my fall term performance studies class to feed back on how well they felt they had met the course’s outcomes. I did not do this in a survey, or in class; rather, I created a final exam question about it.

That meant the students were required to think fulsomely about both the class’s outcomes and the means by which we tried to get there; they were also asked to consider both when we had and when we had not reached outcomes, and to reflect critically on outcomes-based learning as a process through which they, as students, had traveled.

Here’s the question I posed:

At the outset of our course, Kim offered the following potential “outcomes”:

Students who take our course seriously and commit to our shared labour can expect:

  • To be introduced to a host of contemporary performance theories and practices;
  • To develop the capacity to critique a piece of non-scripted, non-traditional performance;
  • To learn the value and power of collaborative teaching and learning;
  • To practice critical thinking using written text, video, and audio tools;
  • To continue to improve their research, writing, and editing skills;
  • To practice, develop, and improve public presentation skills;
  • To experiment with independent and/or team performance-making;
  • To take some risks, make some mistakes, and have fun!

Did you achieve them? Some more than others? Did you not achieve some? Using “thick description” of key moments in or outside class, talk about how a selection of these outcomes contributed, or not, to your learning in TS2202. You need not talk about all outcomes. You need not be positive about all outcomes! Nuanced, honest self-analysis is welcome.

Seven out of 20 students (a statistically impressive 35%) chose to write on this question. Grades ranged from 36/50 (for a thoughtful reply, but one missing a clear structure or detailed descriptions of learning events), to 48/50 (for a reply that was well structured and well detailed, and full of careful self-reflection). Students were not judged on whether or not they deemed outcomes to have been met or not; I was far more interested in hearing them talk about how, and why, either result may have obtained.

Several students talked about the value of learning about non-traditional forms of performance; one made the point of saying his directorial practice was shifting as a result of our class’s exposure to work far outside the Western dramatic canon. Another noted that non-traditional performance forms required us to explore non-traditional ways of talking about those things, and then commented on the fear, but also the excitement, of engaging in that kind of exploration.

Most students mentioned the power of taking risks and making mistakes (likely because I mess up a lot in class, and never hide it, my students tend to get comfortable with error). One student described a moment early in the semester when they had shared an intimate, taboo piece of personal history, and the positive impact they experienced when I did not judge, but turned that sharing into a teachable moment. Another talked about learning that their mistakes in class could all be “manageable” (probably the most important outcome any university student can take from any class, anywhere!). Still another offered this helpful reflection on the first day of class:

On the very first day when we were asked to act out the syllabus I made a decision to let myself take risks and be silly. I decided to really try to turn off that voice that says ‘oh don’t do that, you’ll look foolish’. … I went away with that quiet voice telling me I was ridiculous but I didn’t listen, and I looked forward to every class that followed.

In general each student selected a range of outcomes to talk about, with some outcomes getting more attention than others across all seven papers. Every single student, however, wrote about the “collaborative teaching and learning” outcome. Some expressed continued anxiety about group work, but also took the time, in the spirit of the question, to think about the positive (if still difficult) experiences of shared labour they’d had – learning to account for others’ perspectives and personalities, learning to deal with clashes of opinion, and learning that sharing and negotiating ideas does not require consensus or group-think to emerge.

My favourite reflection on our collaborative classroom practice was this one:

What was very evident throughout the year was the collaboration between teacher and students. I am currently taking an educational psychology course, and there were a lot of tasks we did throughout the course that are akin to optimal teaching. For example, the first day of class we partnered up to discuss any questions we may have had about the syllabus, known as reciprocal questioning, which encourages a deeper understanding of the material being discussed. This goes for many of our group interactions throughout the semester. You also relinquished some control in the course content by allowing us, in groups, to pick some of the readings. This elevated sense of control, or human agency, in our learning increases motivation and self-efficacy.

The student who wrote this response did something very special for me. They connected my classroom labour to the prevailing pedagogical research, and noted how the collaborative environment I create for my students is geared directly toward an outcome I’ve not yet identified: providing students with the opportunity to build agency, and take ownership over a lifelong learning process. I will be adding that outcome to future syllabi, you can be sure – and crediting the student (whose name I know) in the process.

I’ll be putting an outcomes question on the final exam again; I learned a great deal from it about where my students see the connections between my stated goals and our classroom labours. These connections are sometimes where and what I expect them to be, and sometimes not – which means these answers offer me very useful fodder for future classroom planning. I think I’ll tweak the question next time around, though, to encourage balance: I’d like to hear a) where students met an outcome, and how; b) where they did not, and why; and c) what else we might have done to meet an important potential outcome, stated or not.

Now, I’d love to hear about YOUR outcome labours. What do you do to set objectives and outcomes effectively? How do you test their efficacy? Please leave comments! I also want to thank all of the students in Theatre Studies 2202F (2016) for inspiring me to think more, and more carefully, about how I remain accountable to them, to their peers, and to myself in our shared learning environments.

Kim

Three morsels and a question

Yesterday, some words of wisdom from teaching and learning scholar Phyllis Blumberg’s latest book crossed my virtual desk. They do that thing I really love the stuff that spams my inbox to do: they say a bunch of should-be-obvious things in ways that make me think about why they should be obvious, and why nevertheless they often are not, for many of us anyway. Whether you teach at the secondary school or college level, in private or public institutions, I suspect they will resonate. Here are my favourite three morsels, followed by a question for all of you related to the issues they raise.

1118275489

First, Blumberg talks about learning “objectives” vs learning “outcomes”. I’m betting many of you, like me, have learned to tune these phrases out, rolling your eyes along the way. Yes, yes, we all have objectives and stuff! We know what we want the kids to learn already! As for these terms, they are just university biz-speak; they mean the same thing. Right?

Actually, no. Here’s Blumberg, helpfully, on the difference between “objectives” and “outcomes”:

Learning outcomes are the big picture, often complex goals that instructors expect students to achieve or learn by the end of the course.  They should be stated in terms of student performance, not what the instructor hopes to achieve, such as what content will be covered.  Learning outcomes are also called instructional goals (Diamond, 2008; Nilson, 2003).

Learning objectives are smaller units of learning that flow directly from the learning outcomes.  While a course may have about five larger learning outcomes, each learning outcome may have a few learning objectives associated with it.  For example, learning objectives may describe what students will learn from the discussion in a specific class. The literature often interchanges learning outcomes and learning objectives. In this book, I am referring to the larger learning outcomes.

I’ve realized, as I’ve thought about this over the last 24 hours, that while I have plenty of (let’s face it, stock) “outcomes” on my course outlines, I almost never take the time carefully to think them through, much less to link objectives to outcomes. Students will learn to write proper critical essays; students will write some critical essays in order to do that. Except –

Most concepts and tasks in higher education are complex, involving different component skills, cognitive processes, and many different facts.  To help students learn, instructors need to break down these complex concepts or tasks into their component parts, provide students opportunities to perform these skills or cognitive processes separately, and then allow them to practice the integrated tasks before assessing them. Instructors can point out the key aspects of the task so students know where to concentrate their efforts (Ambrose et al., 2010).

Jeez – duh! Except for the part where I’d never actually thought consciously about this cause-effect framework before. Thinking back to my course this past term on Shakespeare’s afterlives (QM Drama’s DRA 316: “Shakespeare After Shakespeare”), I considered the work we did in the weeks after spring break on building individual essay topics, then on creating essay-related presentations to solicit peer feedback, and then on the essays proper; I realised that the outcome I was aiming for was not “will learn to write proper critical essays” but rather “will learn to work methodically through all of the steps required in order to build independent areas of research interest, and will then apply that learning in the creation of a personal-interest-driven critical essay modelled on the kind of independent critical exploration characteristic of graduate school study”. I also realise that the objectives I employed were multifaceted: 1) will study and practice the process of developing independent research questions; 2) will learn how to shape research questions into essay topics; 3) will test the viability of research questions via peer feedback; 4) will learn to re-shape (sometimes reformulate completely) essay questions based on that feedback without panicking unduly; etc.

Why didn’t I think of this stuff, in this way, before? I think I know the reason (one of them, anyway).

Preparing course outlines during summer research time is (one) onerous, and (two) also works frequently only at the level of theory (because, let’s face it – it’s summer). I undertake this work mostly under duress, try to get it done pretty quickly, and rely on what I think will be “really cool” rather than (in some cases anyway) on what has worked well before. I tend to over-tweak existing formulas based on the elusive/imagined “really cool” factor, rather than letting the formula rest for another year or simply refining the outcomes/objectives a bit with the past year’s evidence fully in mind. Note that I always want to do the latter – I keep and annotate all my course evaluations for this purpose. I just don’t give it enough time or care, I think, because I don’t take the pedagogical logic behind the language of “outcomes/objectives” seriously enough when I’m also kind of hurrying through the task so I can get back to the book/article/conference paper/edition/whatever. In fact, as the above example from DRA 316 demonstrates, “outcomes/objectives” are absolutely central to the most interesting, and the most fun, work I do with students. If I forget about them come course-prep time, I risk losing all that productive fun, the legacy of previous work and the power of framing its value clearly for students in future.

And on that note, Blumberg offers me one more crucial morsel I want to share here.

The relationship between the difficulty of a course and student learning is curvilinear.  The best learning occurs when the course is perceived as difficult enough to be challenging, but still seen as achievable.  Under these circumstances, students are motivated to try.  If a course is too easy, students do not put forth any effort.  If the course is perceived as too difficult, students are not motivated to try because they think there is no way they will succeed (McKeachie, 2007).

I’m pretty sure we all aim for this, in theory, all the time. But many of us often don’t succeed. Why not? I’m going to go out on a limb and say it’s because we haven’t built the logic of this statement into the “objectives/outcomes” framework that (if you’re like me) you’ve been busy dissing/ignoring/putting up with rather than taking fully on board as a pedagogical tool.

What, then, if every course you taught next semester featured as a stated outcome:

  • “Students will learn that being intellectually challenged can be a source of pleasure, a source of community, and a site of genuine personal growth!”

…followed by a bunch of complementary objectives? And what might some of those objectives be?

That’s the question. Please weigh in!

Invigorated,

Kim