Best practices for college exam design?
January 24, 2012 1:45 PM Subscribe
How many questions should a one-hour quiz contain?
I want to test college students at a commuter university on 2 chapters of a constitutional law course. The test will constitute 20 percent of their grade. The test will be online, so it's going to be de facto open book (I think it would be pointless to make it a closed-book question, given that cheating would be easy).
My thought is to provide enough test questions that time would be at a premium, and so that the people who actually know their stuff would do better than the people who will just try to look stuff up. About how many questions would a one-hour test include? I was going to confine the test largely to T/F, multiple-choice, and fill-in-the-blank questions, with perhaps a few short-answer questions at the end. Any suggestions on the number of questions?
I want to test college students at a commuter university on 2 chapters of a constitutional law course. The test will constitute 20 percent of their grade. The test will be online, so it's going to be de facto open book (I think it would be pointless to make it a closed-book question, given that cheating would be easy).
My thought is to provide enough test questions that time would be at a premium, and so that the people who actually know their stuff would do better than the people who will just try to look stuff up. About how many questions would a one-hour test include? I was going to confine the test largely to T/F, multiple-choice, and fill-in-the-blank questions, with perhaps a few short-answer questions at the end. Any suggestions on the number of questions?
I think there's too many variables. Especially the reading speed of the student, even if they know the content.
Since the test constitutes so much of the grade, that you should really focus on making sure the content of the questions you have covers the important information in those two chapters. Not the amount. Inflating the test to combat cheating seems a little underhanded.. and unfair to the students who are in the middle of passing and failing.
If you're worried about cheating: Does your university have some sort of testing center? A place that provides proctored testing? We have one of those here at my Community College, and the tests are locked out with a password only the proctor can give them.
posted by royalsong at 2:01 PM on January 24, 2012
Since the test constitutes so much of the grade, that you should really focus on making sure the content of the questions you have covers the important information in those two chapters. Not the amount. Inflating the test to combat cheating seems a little underhanded.. and unfair to the students who are in the middle of passing and failing.
If you're worried about cheating: Does your university have some sort of testing center? A place that provides proctored testing? We have one of those here at my Community College, and the tests are locked out with a password only the proctor can give them.
posted by royalsong at 2:01 PM on January 24, 2012
If you were going to comprise the test solely of multiple choice and/or T/F questions, I'd say 30-35 questions for a 60 minute test. Adding fill-in-the-blank changes things a bit, but I'd still aim kind of high for the number of questions. If they are truly fill-in-the-blank (and not quasi-essay questions), I think you could get 10 FITB and 20 T/F or multiple choice. The ratio of 2 minutes:1 question favors those who know the material (or at least know where to look to confirm their answer) and punishes those who are less familiar with the material.
(I am currently enrolled in a [graduate-level, degree-seeking] distance learning program and take quite a few online quizzes and tests.)
On preview: I think my numbers may be a touch too high.
posted by heathergirl at 2:04 PM on January 24, 2012
(I am currently enrolled in a [graduate-level, degree-seeking] distance learning program and take quite a few online quizzes and tests.)
On preview: I think my numbers may be a touch too high.
posted by heathergirl at 2:04 PM on January 24, 2012
I don't think it's ethical for us to answer this question in addition to just plain not being able to. Aren't there past tests taught by the previous teachers that can enlighten you? Or staff that can help you?
posted by OnTheLastCastle at 2:07 PM on January 24, 2012
posted by OnTheLastCastle at 2:07 PM on January 24, 2012
A decent rule of thumb I've heard passed around among high school teachers is that however long it takes you (the teacher) to take the test, it will take your students twice as long. So time yourself taking the test, see how quickly you finish, and if the allotted time is twice what it took you, then it's about right. For added challenge, since this is at the university level, you can add a few extra questions beyond that.
posted by danceswithlight at 2:28 PM on January 24, 2012
posted by danceswithlight at 2:28 PM on January 24, 2012
Can't you just put in way more questions than anyone could answer, then scale it based on how everyone does? That accomplishes two things: it solves the length problem and rewards people who know the answers from memory, since a memory lookup will be exponentially faster than a book/internet lookup.
posted by supercres at 2:56 PM on January 24, 2012 [1 favorite]
posted by supercres at 2:56 PM on January 24, 2012 [1 favorite]
You're not teaching reading speed and comprehension. So why would you test it? Reading speed varies incredibly among people. I once did a 2 hour multiple choice final exam (100% pass/fail) in under 7 minutes. A strategy of loads of multiple choice questions just gives you three groups of people: good scores for those who read quickly and know the material, bad scores for those who read slowly and don't know the material, and the bulk of the people in the middle group either read quickly and had to look things up, or read slowly but knew everything by heart. This would especially be a problem if you were using e-texts or there were searchable references -- the test shows who's smart enough to type "Brown Vs. Board".
Long answer questions do a much better job of showcasing ability. Or could you ask students to do the exam closed book? Wouldn't law students have an understanding of the ethics involved?
posted by Homeboy Trouble at 3:51 PM on January 24, 2012
Long answer questions do a much better job of showcasing ability. Or could you ask students to do the exam closed book? Wouldn't law students have an understanding of the ethics involved?
posted by Homeboy Trouble at 3:51 PM on January 24, 2012
I was a chemistry TA, ages ago.
Our rule of thumb was that, for an hour-long exam, the TAs should be able to complete the exam in 20 minutes, and the professor in 15.
Of course, chemistry tends to involve a lot of calculation, and I presume constitutional law does not, so this may be overly restrictive for a ConLaw exam, but I offer it as a data point.
posted by DevilsAdvocate at 6:50 AM on January 25, 2012
Our rule of thumb was that, for an hour-long exam, the TAs should be able to complete the exam in 20 minutes, and the professor in 15.
Of course, chemistry tends to involve a lot of calculation, and I presume constitutional law does not, so this may be overly restrictive for a ConLaw exam, but I offer it as a data point.
posted by DevilsAdvocate at 6:50 AM on January 25, 2012
This thread is closed to new comments.
I've also taken tests where I really have to think about the multiple choice questions and work out the answer by logic - I once had a 10-question, 75 minute multiple choice test that was difficult to finish in the allotted time.
posted by insectosaurus at 1:53 PM on January 24, 2012