Q& A with Convention Speaker Sandy Anderson - Developing Effective Exams

Examinations are an important way to test your students’ critical thinking skills. Get tips for developing reliable and effective exams by reading this Q&A with AMTA Teachers Track speaker Sandra Anderson. Her session on "Developing Critical Thinkers Through Exams" will change the way you think about testing your students. 

How do you define “critical thinking?”

In a word, critical thinking is analysis. It requires multi-logical thinking in that students need to consider multiple pieces of information in order to make an informed decision. In the massage and bodywork profession, critical thinking means that students need to be able to integrate pieces of information they have learned and come up with a conclusion about a particular client or situation. They should be able to correctly justify the next right action, explain it clearly, and then confidently and competently carry it out.

What are some of the components of reliable and relevant examinations?

  • A template that is based on the learning objectives for the information being tested.
  • A good balance among recall, comprehension, application, and analysis questions. These are the levels of Bloom’s Taxonomy. Recall is just what it sounds like: simply recalling learned information. Comprehension involves paraphrasing, matching, changing, confirming, expressing, illustrating, restating or transforming the concept being tested. Application involves choosing, classifying, or interpreting the concept or construct. This level places students in the clinical setting and asks them to make the same choices they would make when assessing  a client and determining a treatment plan.The cognitive domain, analysis, is one of the hardest areas to write questions for because it requires multi-logical thinking. Students need to consider multiple pieces of information to make an informed decision. In other words, analysis is critical thinking.
  • Fair, not “tricky” questions.
  • The most effective types of questions. Typically, multiple choice questions are the most effective, and true/false are the least effective. True/false questions tend to be confusing or make students overthink them. Matching and short answer fall somewhere in between.
  • Questions that the most accomplished students answer correctly and that uninformed students cannot answer correctly simply by guessing.

What’s one thing a lot of teachers are doing wrong when it comes to examinations that could be easily corrected?

One particular challenge that many instructors face is writing an effective critical thinking question. Often, these questions are written as “tricky” instead of analytical. These questions may have extraneous words and information, or are simply not written clearly at all. Sometimes these questions do not even tell students what information is being asked. This can be especially daunting for certain students such as those with dyslexia or test anxiety.

An example of a tricky question is:

"A 49 year old client who lives with her mother was a cheerleader in high school. She sprained her knee several times during this time, and now has limited movement in her right knee. She arrives for her massage appointment wearing two sweaters and is smoking a cigarette. She says she’d like to regain the mobility of her right knee and wants to know how massage can help her do that. What should the massage therapist do?"

In this example, there is a lot of extraneous information – the client lives with her mother, she is wearing two sweaters, and she is a smoker. Additionally, there is no clear indication what information is being asked. There are many things the massage therapist should do – assess the mobility of the client’s knee; determine a treatment plan; discuss how massage can help increase the mobility of the client’s knee; discuss what massage is unable to do to help increase the mobility of the client’s knee; discuss self-care the client can do; and so forth. 

This question can be rewritten as:
"A 49 year old client has limited mobility in the right knee from sprains that occurred during cheerleading over 30 years ago. Which action will help the massage therapist determine the appropriate treatment plan for this client?"
With the question rewritten this way, the student is given all relevant information to make an informed decision, and knows in what direction to focus to come up with the answer.

What process do you use when developing examinations? Are there set steps that you follow?

  • Understand the purpose of the exam.
  • Follow the course objectives; make sure all course objectives are represented on the exam.
  • Develop a test blueprint.
  • Determine the best type of questions to evaluate the students’ ability to meet the objectives
  • Depending on the purpose of the exam, use COMTA competencies as guidelines.
  • Depending on the purpose of the exam, use NCETM, NCETMB, and MBLEx Content Outlines as guidelines.
  • Make sure there is a good balance among recall, comprehension, application, and analysis questions.

Does the type of examination you implement depend on the class and the students you are teaching?

Yes, definitely. Many, if not most, massage and bodywork students are kinesthetic learners. They learn by doing. Therefore, it is important that hands-on skills be assessed through hands-on practical exams.
Communication, ethics, and business knowledge is best assessed through short answer and essay. These types of exams also serve to assess students’ ability to express themselves through the written word. Some may think that in this age of technology writing skills are not that important. However, writing skills are needed for brochures, websites, newsletters and so forth.

Many types of information such as effects of techniques, how techniques are performed, anatomy, physiology, kinesiology, communication, ethics, business and clinical applications can be tested effectively through multiple choice questions. It is for this reason that the NCETM, NCETMB and MBLEx are constructed entirely of multiple choice questions.

» Learn more about how to develop relevant and accurate examinations at Sandra's in-depth session Developing Critical Thinkers Through Examsat the Teachers Track at the AMTA 2013 National Convention.


"AMTA is an awesome organization. I can’t imagine any massage therapist not joining."

Karen T., AMTA member since 2002