Sometimes it is not just how to do things RIGHT, it’s about how not to do things WRONG. Reflection on both is important.
Let me outline a recent bad experience I had with a set of online exams.
There were 3 fortnightly exams. Each with about 10-20 questions relating to the previous two weeks of study.
First Exam: You could do it as many times as you liked- you just needed to get 80% (eventually) to pass. Once you had done the exam, it showed you which questions you got right, and which ones you got wrong. But not the answers.
Of course, I just guessed the first lot, then printed it out. The questions I got right, I kept the same, and the questions I got wrong I just changed to a new answer. I did it about 3 times to get my 80%. But it was worth it. I actually learnt a lot- the old theory of repetition.
Second Exam: You could again do it as many times as you liked- you just needed to get 80% to pass. This time it didn’t show you which ones you got right, and which ones you got wrong. Bummer. I did the same thing though- just had to write out all the questions, and then I developed a code so that I could indicate if f I was 100% sure I was right (i.e. I looked it up and found the answer), or if I was half sure, or not sure at all. It took me about 5 or 6 tries, but I got my 80%.
The positive aspect of this exam was that I actually had to look it up. The con? I still to this day don’t know which of my answers were right, and which were wrong. Which sucks. I am now no better off for my in class exam in two weeks, or my assignment due next month.
Third Exam: You could only do it once, and you needed 80% to pass. I went to my friend’s house, and we spent a whole heap of time studying each question, and then researching the answers. We were 100% sure we were 100% right. Because we had found each answer… somewhere. It turned out that we only got 75%, which isn’t a pass. This is frustrating, because we honestly believed that we were right.
The justification? The answers had to be the answers that the Uni wanted, and they had to be the answers from the syllabus, and related EQ documents. Not from any educational source- like our text book… *rolls eyes. Ah huh…
Again, I still have no idea which questions I got right, and which ones I got wrong. So there is no learning in this final exam- just pain and stress.
So lets have a look at how this whole online exam situation could have been better.
I understand that they don’t want you to do what i did in exam 1, so that’s why they don’t show you what you get right or wrong in the subsequent exams. BUT it would have been better if after I got 80%, it locked me out so I couldn’t do the exam again, and then showed me what I got right and wrong. That way I could use it as study material, and see where I went wrong. As it stands I could be completely off track on this subject and I would have no way of knowing. I could go into my exam thinking that I know about investigations, and not really have a clue.
It would have also helped to have a more user friendly platform. Let me tell you now, I do NOT find blackboard a user friendly method for doing online exams, nor for making them, and I have had experience with both. Online exams are one of those difficult things, that in all the different platforms I’ve used, I haven’t seen one which does it completely adequately.
One of the things I found really frustrating with these exams is that there were spelling mistakes, grammar mistakes, and inconsistencies throughout a lot of the questions. As a learning professional it is VITAL that in online exams (or any exams) you take the time to check everything over. I am not saying that I do this all the time (even though I know I should), but it is really something that makes your students feel like they are participating in a quality exercise.
Adequate teaching on the subject is also really important. Anything you are planning on putting in your test should have been TAUGHT previously, not just left for the student to research. This is important because these are your key concepts, and the most important parts of the subject. If your students don’t know about these things, and don’t have accurate information on them, then you haven’t done your job.
So hopefully by looking at this example of how an online exam was done poorly, we can gain some understanding of how it can be done with excellence.
If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!
May 10th, 2008 at 3:02 am
Interesting experiences and observations Talia. It seems that most of what you said is not specific to an online experience though - any exam must cover the content actually taught, be proof read etc and I think they should continue the learning process, not just be an evaluation tool. I think you can make those sorts of conections about so many elearning situations - sometimes we get hung up on the technology component, but if we just apply good sound education theory, the impact of the actual technology to how things should be done is minimal.
May 14th, 2008 at 10:07 pm
Hi Suzanne,
I guess what I was trying to point out in this particular post was the importance of doing things like teaching the content, and proof reading. I wasn’t trying to say that educators don’t do that. I just felt like it was a poor effort from a University, where I know that the lecturer has taught this subject for quite a few years now.
I completely agree with you about it continuing the learning process, and not just evaluating. I think that a lot of people miss this. And you’re right about sound education theory. A lot of people think they have to do things to differently when it comes to eLearning, and sure, there certainly are things that need to be ’shuffled’ to make eLearnign effective, but the basic principles and theories are the same!
Thanks so much for your comment, and for stopping by my blog, I really appreciate it.
Talia
July 13th, 2008 at 4:37 pm
Hi Talia. I’ve been enjoying browsing through your site. I was pleasantly surprised to see myself quoted in your section on the Circle of Courage.
I’m preparing to speak on the model, and in looking for helpful information, stumbled on your site. You do a nice job of interpreting the model and providing details/plans for how teachers can incorporate the model into their classrooms.
I also read your piece about online exams. I set up the system to tell students which answers were correct and incorrect. They can then take the exam again, but will receive many new questions from the large pool of questions, and the answers will be randomized too. If students disagree with the “correct” answers, they print out the screen and talk with me via e-mail, phone, or office visit to defend their answers. A thoughtful response (even if different from the text) gets full credit for the question.
Best regards,
Dr. Mac
July 30th, 2008 at 12:52 am
That’s a really great idea! I’ve been experimenting with something similar actually!
Glad you like my circle of courage thoughts. It really is quite a good system!
August 12th, 2008 at 3:32 am
Wow! That\’s something to ponder… Thanks for the informative post.. and thanks for adding our comment to the blog. I am subscribing to your feed so I don?t miss the next post!