Ask, Not Tell

Following my learning from the William Rothwell’s session, I now incorporate a course-end quiz into most of my new courses.   There are all different test formats – essay, demonstration, fill-in-the-blank, multiple-choice…   Amongall, I prefer the multiple-choice question since it takes minimal time to check the answers.

I am drafting questions for a Train-The-Trainer course.   I want to test the learners on Adult Learning Principle.   I am thinking…   my question can be:

Option A – Which of the following is NOT an Adult Learning Principle?

1. Problem-Centered- Adults are motivated to learn so they can perform a task or solve a problem.
2. Association – Adults need to build on experiences.
3. Color – Adults learn more effectively when their workbooks contain more than 5 different colors. (Answer)
4. Variety – Adults learn more rapidly from a variety of instructional methods.

Or the question can be:

Option B - Which ‘Adult Learning Principle’ is NOT applied in the following learning experience?

‘The trainer broke the 1-day system training into 4 sessions covering the 4 major components of the new core banking system.  In each session, he first lectured for 45 minutes and then ran an activity.   In the 1st session, the learners were asked to present back the content in small group.  In the 2nd one about the enquiry function, the learners performed a task to find out information from the system as requested.  In the 3rd one about the transaction function, the learners were asked to construct wrong transactions on purpose and passed to the another small group for checking.  And in the final one, the learners challenges each other in small groups by composing their own questions on the content.’

1. Problem-Centered- Adults are motivated to learn so they can perform a task or solve a problem.
2. Association – Adults need to build on experiences. (Answer)
3. Self-Concept- Adults need to be autonomous and self-directing.
4. Variety – Adults learn more rapidly from a variety of instructional methods.

Which option is better?   Quite obviously option B is better because it pushes the learners to apply.   So, questions can  be asked differently to test different level of understanding.    In fact, I found something interesting from the internet – there are 6 levels of questions:

• Knowledge - Remembering previously learned material, e.g., definitions, concepts, principles, formulas
• Comprehension - Understanding the meaning of remembered material, usually demonstrated by explaining in one’s own words or citing examples
• Application - Using information in a new context to solve a problem, to answer a question, or to perform another task. The information used may be rules, principles, formulas, theories, concepts, or procedures
• Analysis - Breaking a piece of material into its parts and explaining the relationship between the parts
• Synthesis - Putting parts together to form a new whole, pattern or structure
• Evaluation - Using a set of criteria, established by the student or specified by the instructor, to arrive at a reasoned judgment

It gets more challenging to the learners as we go down the list.   Looking back at my example, option A is probably ‘Knowledge’ whilst option b is ‘Analysis’.   For ‘Comprehension’, I probably should ask them to define ‘Self-Concept’.   For ‘Application’, I would ask them to describe what to add in a particular training (e.g. one of their own) in order to make it ‘Variety’.   For ‘Synthesis’, I should ask them to construct a case where all 5 principles are present.   For ‘Evaluation’, I should show them a training video clip and ask them to criticise against the principle.

These 6 levels are very helpful for us to make the test more effective.   Well, it may sound quite academic.   And it would also be challenging to the trainers to make every question of the ‘Evaluation’ level.    I guess for the very least this model reminds us not to ask just ‘Knowledge’ level of question!!

An email came to my mailbox yesterday saying that there is an event by the Hong Kong International Coaching Community called ‘Ask Not Tell’.   It is about changing organisation by coaching.   What a coincidence!

I facilitated a team building activity earlier for a group of 90 people in a business conference in Shanghai.    Right after the event, I feel quite unhappy with the result.   I should have done better but I did not.

The activity went well – clear instruction, smooth process and lot of energy in the room.   The problem was the debrief session.     I found it too short and not deep enough.  Basically, I had a few questions for them to discuss in small groups and invited them to share in the big group.  There were only a few responses during the big group discussion.   After attempting a few more questions with lukewarm response, I could only end the session.  Of course, there were inherent challenges e.g. a big group and crowded environment.    Still, I blamed myself for under-estimating the challenges.  I under-estimated how outspoken the participants would be.

I was angry about myself for such negligence!

Done is done.  Time to move on.  I reflected on what happened and here are my learning:

Assign Responsibility – I would have more sharing from the floor if I have instructed each small team to select 1 person to speak up.   Nobody will do anything if everybody is asked to do it.

Idenify the ‘Pioneer’ – I should scout for those who are more willing to speak up before the debrief sessions starts.   I should walk around, observe and invite them personally to speak up afterward.

Ask Specific Questions – I should avoid asking questions like ‘How is your insight from the activity similar to challenges you met at work?’     Questions like this are too broad (or scary) for the participants to answer.     I should avoid this especially in the beginning.  I should have asked easier questions e.g. closed end questions in the beginning to kick start the participation.   Examples of more appropriate questions are:

‘Mr XXX, is the result in your team as what you all expected?   If not, what would be the reason behind?’

‘Did you have a leader in your team?   If yes, how did he / she establish to be a leader?’

In short, I got to prepare.  I need to prepare a lot of questions so that I will run out of them on stage.     I should think through the process.  Preparation is key, always!

This is my new ‘toy’ – the logitech presenter.  I lost my old one (Kensington) at work.  See my previous post on this.

This device is more advanced than my Kensington one without scarificing too much the simplicity in design.   There is an interesting function – the device can count down time and vibrate when the time is up.   It helps the facilitator control time in a subtle way.

I spend my Chinese New Year holiday in Shanghai this year.  As usual, I feel like being in the war zone here during the Chinese New Year – the firework is everywhere.  The atmosphere is the strongest on the new year eve and the day 4 of the Chinese New Year.  (On day 4, people are supposed to receive the god of fortune by playing firework.   As such, the businessmen generally spend a lot on day 4 on firework.)   Anyway, back to the topic….

Before the Chinese New Year holiday, a few colleagues and I reviewed the learning from Dr Rothwell’s classes.  We talked about course evaluation.  Normally when we talk about evaluation, it is about end of class evaluation – e.g. the famous Kirtpatrick 4 levels of evaluation.    However, some raised a question on that day ”How to get feedback on your class before it finishes?’    This is a valid question especially for courses of longer duration e.g. over 1 day.  It makes sense to check how the learners think and – the course as much as possible.

Here are some interesting thoughts we came up:

Just ask them - It is always an option.  In fact, we should not forget this option.  But sometimes we do commit this mistake.  We continue to talk, talk and talk without at least asking the learners how they feel.

Ask them tactfully - If we ask for feedback openly, the feedback is likely to be a polite (and thus not informative) one.  This is especially the case in the Asian environment.  We got to do it tactfully.  For example, I learn a method from Sharon Bowman – Ask the learners to position their hands in front of their chests.   If they are very satisfied with [say, speed of delivery], raise 5 figures.  If they are very unsatisfied, raise only 1 figure.   They will probably be more frank since they are expressing privately.   With that, you can then get a quick feedback with whatever question you ask.

Keep a feedback wall - Another way is to post a flipchart where the learners can write their feedback on.  You can keep it blank or post a few questions for them to answer.  Preferably, the flipchart should be posted at place where the learners can write without anyone seeing it.   For example, you can post it on the side of a whiteboard which faces the wall.

Talk to a few – Identify 2-3 whom you believe to be expressive and can represent the others (or pick randomly).   Invite them to observe and stay behind on say day 1 to give you feedback.   Most will be more comfortable to tell you more in a small group setting.  And they will feel honoured for your invitation.

Hide yourself in the toilet – It is the most extreme measure!   Some said that the comment exchanged among the learners during the first tea break is key to the success of a course.   So, once you announce the tea break, you should run to the washroom and hide yourself there, you will hear the most frank (and brutal) comment from the learners!!

If you try out some, let me know how it go!!