Testing, Assessment, and Mastery Learning

Fair Juliet claimed that which we call a rose by any other name would smell as sweet, but Juliet never had to assess learning and certainly could never have imagined the tools we now have to do so. Most of which we just don't use much or in very inventive ways.

Use of technology that allows a learner to take more ownership of learning, subject mastery and self-evaluation has helped in understanding more applied formative assessment practices. A few that we tossed about in the latest College of Public Programs TechByte lunch topic on assessment were the rich, often hidden features within Blackboard tests/quizzes.

Here's bottom line. We want learners to learn. Do we really care that it took them more time/effort than another learner, or that they didn't understand a concept first time around? Why not use testing as a way for them to explore meaning, work through issues they don't understand, concentrate harder on the material they didn't master, and give them a tool and an option to rethink / redo / revise? Isn't that how learning happens best?

Here's a few practices shared in putting our heads together. Add any you've come up with and perhaps our students will actually value, instead of dread, our assessment practices.

Blackboard (at ASU; other places, other CMS...same principles):

• Set test for multiple attempts. You can choose number of attempts (unlimited, 3, etc) and time frame (Monday from 9am-noon, 50 minutes per attempt) , but why are we asking that they get it right first time? What if it's the way you phrased question that stumped them? Where's the harm in going back to the book, rethinking, trying again, going back to the source? It may be the only time they do!
  
• Randomize questions for each redo, forcing students to concentrate on questions each time, not just fill in madly to get to the missed questions. (also hinders sharing)
  
• Put questions in a pool, allowing students to pull different questions each attempt (see bullet above on sharing). This is also a great way to offer a final, randomly pulling questions from previous quizzes.
  
• Set exam for storing best attempt, not last attempt. Research shows that students are more likely to try again if not afraid of inability to top last score. (BB instructors: this feature is oddly hidden in Grade Center/Modify Column, rather than via deployment - which is why so few of us use it and leave last attempt as default)
  
• Give feedback immediately for each of the missed questions. Don't provide answer, but perhaps the page in text where concept is found or the reason they may have missed question, etc. One instructor (Hey, Kelly!) tells us that she has been consistently receiving grateful feedback from learners as she improves feedback on missed (and correct!) responses in her quizzes.

Tom D'Angelo and Patricia Cross suggest that unless feedback is very immediate or absolutely needed to progress (eg "follow my advice on your rewrite of this paper, or else"), students don't review or follow up on feedback. Giving them auto-feedback, in a low-risk environment where they know you don't see their first effort, is a great use of technology to invite time-on-task and mastery learning.

Other ideas for more formative self-assessment practices using the CMS testing options? Send them my way and I'll incorporate.

Best Practices in Online Teaching, Part II

Long ago and faraway, I wrote a post about the workshop I do on e-Teaching. The point of the workshop is to remind all of us, again and again, that technology can be a time-sucking black hole and that online instructors often burn out by not setting boundaries or using known "best practices" in e-Teaching. I listed a few easy things you can do to create very engaging learning experiences while still protecting your time and sanity. I promised that I'd collect more ideas from the workshop offerings and post them and like so many good and well-meant plans, they got sucked into the time-sucking black hole of technology, e-learning and life.

So, meeting with the community that calls itself "instructional designers of ASU" and talking about these ideas & asking for more/their favorite/new e-teaching practices that save instructor sanity brought a few reminders and new tools to my kit. Here, many months later, are Tips, Part II:

• Use an FAQ. Gather your questions, especially the technical or how-to kind, into an FAQ. You never know when/where/why students will mysteriously forget how to do the simplest things and if they don't find the answer where they're looking, they'll send email (forbidden, as you know if you read Part I) or ask in the discussion board. Teach them to look before asking by putting general questions in one helpful FAQ location.


• Don't use dates in your content. Don't put hard dates in any of your material. If you work with good designers, you'll know they repeat this till they're blue in the face, and there's a reason. You'll just have to rip them out when you offer the course again, and dates are hard to find. Label your modules "Week X" or "Module X: The Role of Z in Y" or whatever you like, but NOT "Week X: April 3-10". Instead, post a course calendar or schedule in a prominent place. Post current week/module/topic in the announcements. Send them email. Just don't build dates in your content material.


• Plan Ahead. Duh! And still we don't because life happens. But unlike the F2F mode, you can't scramble online. Mistakes are made and you can't see the puzzled faces at the end of the "series of tubes" that deliver your course. It's twice as much work if you scramble to pull together material and assignments and outcomes and assessments online. Like the novice carpenter that measures once, cuts twice and wastes a lot of wood, many of us learn the hard way that the best online material is the kind that's fully developed on the first day of class.


• Summarize & summarize again. Yes, something is lost when learners can't see your face or gestures. Mastery of the medium means we use technology to create other ways of enforcing meaning. An excellent suggestion at the ID meeting came from an instructor's practice of "Summary Monday and Surprise Thursday" posts. Every Monday morning, the instructor recapped in a post all the questions/comments/ideas that had come up that deepened understanding. On Thursday, she posted summaries of little problems, glitches, misunderstandings, etc that she had encountered or heard about. It is a great way of creating community, deepening understanding, encouraging more time on task, and sharing solutions to problems that might occur again.
• Highlight the process. One designer claimed that the most problems disappear if you send the student back to the instructions, which they didn't read. Most likely true, but turning the lens back on ourselves, perhaps the message is don't embed important instructions in line after line of dense text. Use "microposts" for important info and dates. Try providing text and an audio file. Consider using the free software Jing to record a short animation capturing onscreen action when you're asking them to do a computer-related task (like submitting an assignment or searching library resources). Maybe the learner is making mistakes because we're making it so easy for them to get lost.

One designer talked about a course that uses Adaptive Release to motivate learners to stay up to speed. It was a great example of new approaches to teaching-as-coaching, but that's a whole new Blog post. Meanwhile, if you have a favorite teaching/tech practice, please share!
Happy trails, happy tech innovation, happy teaching.