Yelpiversity
December 19, 2020 2:02 PM Subscribe
I was surprised to hear some of my colleagues (we are all tenured faculty members at an R1) are giving students extra credit for completing course evaluations.
It is better for our own promotion files to have lots of course evaluations, mostly because if you only have a few, it looks strange, & they tend to be from students who had an extreme experience.
We used to take 15 minutes in class and hand out a paper evaluation and leave the room. A student would bring the paper evaluations to the department office. This practice has gone the way of the aromatic purple mimeograph.
Now with online classes, and even with in-person classes, the students have to check their email for questionnaires and don't feel motivated to complete them. It's just another consumer survey in a way, now....the same thing I get after a doctor's visit.
But I am old school and feel a bit odd giving them any kind of incentive to do the evaluation. Another thing my colleagues do: get the students to write what they liked most about the syllabus as a for-credit exercise just before evaluations are to be distributed, hoping this will help remind the students of the positive aspects of the class.
I don't think this is exactly forbidden but it seems...maybe not completely right? I just ignore the whole thing and let the students fill out the evaluations if they choose. I might give the same level of reminder as I would have in the past with a paper evaluation.
Yet people are very cheerful about pushing it now.
College students and teachers of all levels, and anyone else, what do you think of this practice?
It is better for our own promotion files to have lots of course evaluations, mostly because if you only have a few, it looks strange, & they tend to be from students who had an extreme experience.
We used to take 15 minutes in class and hand out a paper evaluation and leave the room. A student would bring the paper evaluations to the department office. This practice has gone the way of the aromatic purple mimeograph.
Now with online classes, and even with in-person classes, the students have to check their email for questionnaires and don't feel motivated to complete them. It's just another consumer survey in a way, now....the same thing I get after a doctor's visit.
But I am old school and feel a bit odd giving them any kind of incentive to do the evaluation. Another thing my colleagues do: get the students to write what they liked most about the syllabus as a for-credit exercise just before evaluations are to be distributed, hoping this will help remind the students of the positive aspects of the class.
I don't think this is exactly forbidden but it seems...maybe not completely right? I just ignore the whole thing and let the students fill out the evaluations if they choose. I might give the same level of reminder as I would have in the past with a paper evaluation.
Yet people are very cheerful about pushing it now.
College students and teachers of all levels, and anyone else, what do you think of this practice?
My opinion that standard course evaluations are bullshit is probably influencing my opinion here.
The problem is that some instructors will be doing these things and some won't. I don't see an inherent problem with offering extra credit, but it becomes a problem with some instructors aren't offering extra credit and their lower number of responses is seen as a negative. You probably shouldn't do it unless it's standard practice in your department/institution.
I do think that priming students to think of only the positives is sketchy. That is clearly trying to influence the results. As an instructor I would never do it and as a former student I would feel like I was being manipulated.
But as the recipient of some pretty awful and probably sexist evaluations (including some containing outright lies), I think the whole system needs to be changed.
posted by Kutsuwamushi at 2:17 PM on December 19, 2020 [8 favorites]
The problem is that some instructors will be doing these things and some won't. I don't see an inherent problem with offering extra credit, but it becomes a problem with some instructors aren't offering extra credit and their lower number of responses is seen as a negative. You probably shouldn't do it unless it's standard practice in your department/institution.
I do think that priming students to think of only the positives is sketchy. That is clearly trying to influence the results. As an instructor I would never do it and as a former student I would feel like I was being manipulated.
But as the recipient of some pretty awful and probably sexist evaluations (including some containing outright lies), I think the whole system needs to be changed.
posted by Kutsuwamushi at 2:17 PM on December 19, 2020 [8 favorites]
Giving a bit of extra credit for completing course evals has been a common practice at three universities i've taught at. When we get penalized for low response rates, what else do you expect? The accuracy and fairness of student evals is a joke anyway, as many studies have shown. There needs to be a different system.
posted by demonic winged headgear at 2:27 PM on December 19, 2020 [2 favorites]
posted by demonic winged headgear at 2:27 PM on December 19, 2020 [2 favorites]
ChuraChura has a good practice! Looping students in on how evals are used and what they mean for us and future students is really good. No reason not to be fully honest here. I have a colleague who straight up explained how important evals are for tenure.
I do something similar where I spend time on the last day, pulling out the syllabus and talking about what my goals for students were, and the various ways we approached those over the semester. It's a good learning recall opportunity, for me to cement what they should be taking home.
It's worth considering that while you and I may be tenured, folks coming up for tenure still rely on evals (as flawed as they are, and hopefully they are read in that context), so you and I are in a good position to develop a departmental standard around encouraging students to do the evals, without resorting to reward tactics. Which are awful, IMHO.
posted by Dashy at 2:48 PM on December 19, 2020 [2 favorites]
I do something similar where I spend time on the last day, pulling out the syllabus and talking about what my goals for students were, and the various ways we approached those over the semester. It's a good learning recall opportunity, for me to cement what they should be taking home.
It's worth considering that while you and I may be tenured, folks coming up for tenure still rely on evals (as flawed as they are, and hopefully they are read in that context), so you and I are in a good position to develop a departmental standard around encouraging students to do the evals, without resorting to reward tactics. Which are awful, IMHO.
posted by Dashy at 2:48 PM on December 19, 2020 [2 favorites]
Former college student here. I don't see anything particularly wrong with offering students extra credit to complete the course evaluations.
Your colleagues' strategy of having students reflect on their favorite parts of the course just before completing the application does seem devious and a little untoward to me... but I don't think it's worth losing sleep over.
posted by mekily at 3:20 PM on December 19, 2020 [1 favorite]
Your colleagues' strategy of having students reflect on their favorite parts of the course just before completing the application does seem devious and a little untoward to me... but I don't think it's worth losing sleep over.
posted by mekily at 3:20 PM on December 19, 2020 [1 favorite]
It's definitely kind of shady and you know it. But what you might not see is the whole system is a bit suspect, and for your pre-tenure colleagues—and especially for non-tenure instructors—it's just a crappy part of a crappy game and I don't really begrudge anyone from doing what they have to. It seems more ethically dubious the more senior you get.
Honestly I think some of these gambits may backfire in some circumstances, but hey that's fine too.
posted by SaltySalticid at 3:31 PM on December 19, 2020 [4 favorites]
Honestly I think some of these gambits may backfire in some circumstances, but hey that's fine too.
posted by SaltySalticid at 3:31 PM on December 19, 2020 [4 favorites]
As an undergrad student I did occasionally (it did not appear to be part of an institutional or even department-wide practice) some extra credit for doing course evaluations and when teaching I was strongly encouraged to incentivize this process -- but wasn't explicitly told to give extra credit.
I, too, find the whole thing to be a mess and it's one of the many reasons I left academia.
posted by sm1tten at 3:41 PM on December 19, 2020
I, too, find the whole thing to be a mess and it's one of the many reasons I left academia.
posted by sm1tten at 3:41 PM on December 19, 2020
I ran my college's independent course review publication. So none of these were fed up the departmental chain for any instructors, but there couldn't be any extra credit either. I hardly remember any other course evaluations, though it's very likely that that was a quirk of the departments I took courses in.
We used to take shifts to babysit two printers ('click' and 'clack', who took commands from the computer 'tappet') as they faithfully printed course evaluations, stuff them into envelopes, get writers to run them all over campus, cajole instructors into actually handing them out, hope that intra-campus mail would get the responses back to us, then distribute them to writers to tally and summarize.
I think it's all digital now, of course.
posted by batter_my_heart at 4:31 PM on December 19, 2020 [1 favorite]
We used to take shifts to babysit two printers ('click' and 'clack', who took commands from the computer 'tappet') as they faithfully printed course evaluations, stuff them into envelopes, get writers to run them all over campus, cajole instructors into actually handing them out, hope that intra-campus mail would get the responses back to us, then distribute them to writers to tally and summarize.
I think it's all digital now, of course.
posted by batter_my_heart at 4:31 PM on December 19, 2020 [1 favorite]
There's a ton of research showing that course evaluations are racist, sexist, and bad measures of student learning. (They are good measures of what grades students think they are getting.)
On the other hand, there's some actual pedagogical value in getting students to articulate what they've learned. So at least when your colleagues ask their students to write about the benefits of the class, somebody learns something. It's both more beneficial and safer under pandemic conditions than giving out candy, which is another strategy proven to improve evaluation scores!
posted by yarntheory at 4:38 PM on December 19, 2020 [5 favorites]
On the other hand, there's some actual pedagogical value in getting students to articulate what they've learned. So at least when your colleagues ask their students to write about the benefits of the class, somebody learns something. It's both more beneficial and safer under pandemic conditions than giving out candy, which is another strategy proven to improve evaluation scores!
posted by yarntheory at 4:38 PM on December 19, 2020 [5 favorites]
I am a prof, and I think student evals are bullshit. Still, I encourage students to fill them out, even in hybrid classes. (I leave the Zoom room for 15 minutes and encourage them to do so. For all-online classes, all bets are off: all you can do, really, is encourage them via email or LMS announcement, which, yeah, many of them ignore. Whaddya gonna do.)
I think it's unethical to give extra credit to students for filling these things out. And I think students know very well that it's unethical for profs to do this.
But I don't think it's unethical to, for instance, make the publication of their grades contingent on their filling out the evals. Or something like that. I don't do that, but I think it's decidedly not ok to reward people for this reason, but that it is ok to delay the receipt of something they'd receive, anyway.
On edit:
I also think it's wise to explain to students the value of the evals: why they matter for employment, raises, tenure, institutional research, allocation of funds, etc. My students seem to appreciate this, and I think it allows them to see these things as something other than the "consumer surveys" that they sort of are.
posted by Dr. Wu at 5:24 PM on December 19, 2020 [2 favorites]
I think it's unethical to give extra credit to students for filling these things out. And I think students know very well that it's unethical for profs to do this.
But I don't think it's unethical to, for instance, make the publication of their grades contingent on their filling out the evals. Or something like that. I don't do that, but I think it's decidedly not ok to reward people for this reason, but that it is ok to delay the receipt of something they'd receive, anyway.
On edit:
I also think it's wise to explain to students the value of the evals: why they matter for employment, raises, tenure, institutional research, allocation of funds, etc. My students seem to appreciate this, and I think it allows them to see these things as something other than the "consumer surveys" that they sort of are.
posted by Dr. Wu at 5:24 PM on December 19, 2020 [2 favorites]
There's a ton of research showing that course evaluations are racist, sexist, and bad measures of student learning.
Indeed! So really all the tenured faculty should boycott. They have that power and privilege. But in the mean time, pre- and non-tenure women and POC instructors need to stay afloat and keep their academic jobs at R1 institutions, and that can depend upon participation and 'success' in these (racist, sexist) methods. So I think they should absolutely do whatever the tenured white dudes are doing if they think that will help.
posted by SaltySalticid at 5:29 PM on December 19, 2020 [2 favorites]
Indeed! So really all the tenured faculty should boycott. They have that power and privilege. But in the mean time, pre- and non-tenure women and POC instructors need to stay afloat and keep their academic jobs at R1 institutions, and that can depend upon participation and 'success' in these (racist, sexist) methods. So I think they should absolutely do whatever the tenured white dudes are doing if they think that will help.
posted by SaltySalticid at 5:29 PM on December 19, 2020 [2 favorites]
I can't tell in advance which students have filled out course evals and which have not, just an overall response rate. To get the response rate higher, I have offered full-class incentives if we meet a certain threshold.
The one I like best is letting students vote a topic off the final exam (from a list I provide, so they can't say, vote "derivatives" out of calc 1). In a way, this changes very little -- I can't give a final long enough to cover literally every type of problem we've covered, so obviously something has to be skipped. Letting the students have a say in it costs me basically nothing and gives them a little sense of control. It also has the benefit of putting them in a better mood for filling out evaluations, which is slightly suspect, but since student evaluations are so flawed in the first place, I do not feel even a little bad about this.
posted by ktkt at 5:46 PM on December 19, 2020 [10 favorites]
The one I like best is letting students vote a topic off the final exam (from a list I provide, so they can't say, vote "derivatives" out of calc 1). In a way, this changes very little -- I can't give a final long enough to cover literally every type of problem we've covered, so obviously something has to be skipped. Letting the students have a say in it costs me basically nothing and gives them a little sense of control. It also has the benefit of putting them in a better mood for filling out evaluations, which is slightly suspect, but since student evaluations are so flawed in the first place, I do not feel even a little bad about this.
posted by ktkt at 5:46 PM on December 19, 2020 [10 favorites]
As a student, I don't recall ever being willing to fill out a teacher evaluation. I never saw the value. To me, it was mental masturbation.
What I would do is write a note to the teacher thanking them for their efforts and note how much I learned. If the class sucked, I would not write a note. Average classes I would thank the teacher for any extra help they gave me.
I think giving extra credit is a bad practice. What does extra credit even mean? Can filling out an evaluation change a grade from say a B+ to an A-? Is the class graded on a curve? If so, you are penalizing those who do not fill out the evaluation. I also do not see the value in an evaluation that is incented to be filled out.
posted by AugustWest at 6:08 PM on December 19, 2020
What I would do is write a note to the teacher thanking them for their efforts and note how much I learned. If the class sucked, I would not write a note. Average classes I would thank the teacher for any extra help they gave me.
I think giving extra credit is a bad practice. What does extra credit even mean? Can filling out an evaluation change a grade from say a B+ to an A-? Is the class graded on a curve? If so, you are penalizing those who do not fill out the evaluation. I also do not see the value in an evaluation that is incented to be filled out.
posted by AugustWest at 6:08 PM on December 19, 2020
As a student, I remember a professor asking us to fill out evaluations because his tenure depended on it, and I really resented it. If his tenure depends on it, that means you're under pressure to help him or her get that job, which, in turn, means your evaluation has to be good, which is an unethical thing to ask: Students shouldn't be guilt-tripped when they're given an evaluation to fill out.
When I taught, I played around with making my own evaluation forms because the official ones mostly asked inane questions. I used to give my version out maybe a class or so before the official one because it helped me collect information on course content I couldn't get any other way.
posted by Violet Blue at 8:53 PM on December 19, 2020 [1 favorite]
When I taught, I played around with making my own evaluation forms because the official ones mostly asked inane questions. I used to give my version out maybe a class or so before the official one because it helped me collect information on course content I couldn't get any other way.
posted by Violet Blue at 8:53 PM on December 19, 2020 [1 favorite]
I, too, devote some class time to how the evaluation response rate matters effects my promotion, how the model is part of the corporatization of the university that I also don't like, but that it genuinely can help improve my teaching for the next batch. I dont know if it increases my response rate or ratings but at least I feel I am giving them a bigger view of what they are being asked to do and why.
I also have started making the last seminar a "what we learned" discussion (SLAC vibes)... "Your favorite thing on the syllabus" reads like "now kiss my ass please" but I have always found these conversations really insightful, honest and productive. I also share what I learned over the course of the semester, which is always something and sometimes even "this experimental assignment totally failed, never doing that again." Honestly I think students need to be given time to pause and reflect on what they learned and accomplished in a class, in a way that exams theoretically test but rarely actually do. If anything I'd like to do MORE of this reflective writing and self-assessment.
posted by athirstforsalt at 10:13 PM on December 19, 2020
I also have started making the last seminar a "what we learned" discussion (SLAC vibes)... "Your favorite thing on the syllabus" reads like "now kiss my ass please" but I have always found these conversations really insightful, honest and productive. I also share what I learned over the course of the semester, which is always something and sometimes even "this experimental assignment totally failed, never doing that again." Honestly I think students need to be given time to pause and reflect on what they learned and accomplished in a class, in a way that exams theoretically test but rarely actually do. If anything I'd like to do MORE of this reflective writing and self-assessment.
posted by athirstforsalt at 10:13 PM on December 19, 2020
Somewhat related: as a school teacher in South Africa I have to take part in a yearly professional self evaluation. It's a very generalised and simplistic evaluation, but takes significant time to do, paper copies of forms in triplicate etc, and it's scorned by many as a waste of time, and yet if you don't do it, you lose a small yearly but incrementally significant salary increase, so most do it. Each year I consider just saying scr&w it, instead I just do it as efficiently as possible.
How we entrench pointless institutional exercises.
posted by BrStekker at 11:15 PM on December 19, 2020
How we entrench pointless institutional exercises.
posted by BrStekker at 11:15 PM on December 19, 2020
My university has been in the weird situation for the past couple years where we're not legally allowed to require student surveys for hiring, tenure, or promotion decisions nor to read anything into an absence of student surveys, since a Canadian court handed down a legal ruling recognizing the various biases they exhibit against members of protected classes (as defined in Canadian human rights law); but at the same time, my university has still required every instructor except full professors to hand out student surveys for each course (the idea behind the exception being that full professors don't have any higher level to be promoted to, so no longer have to do annual career development meetings with their department heads/chairs, which is where the student surveys would typically be used/discussed, despite the fact that annual career development letters are a significant factor in internal tenure and promotion decisions and so that makes tenure and promotion decisions rely on student surveys, albeit second-hand, seemingly in contravention of the court ruling...).
Anyway, the student surveys are collected and processed by our Registrar's office, and they've been pushing folks to go to online surveys because they are significantly less work to process. But of course the completion rate for online surveys tends to be abysmal, and you get extra bad data because you mostly only get responses from students with strong feelings, especially strongly negative feelings. Add on to this the fact that the questions and format of our particular survey include questions and phrasing that have been shown in studies to elicit greater bias in responses, plus that no one has any evidence-based training in how to interpret our student survey results (lots of folks have general statistics and research methods knowledge, just not the domain knowledge around how biases show up on student surveys and degree of effect - our periodically required 1.5-hour every three years equity training workshops only point out that there are biases, they don't really have time to go into any details).
Our anonymity requirement means that we're not supposed to know which specific students completed a survey or not. But it is becoming increasingly common to give across-the-board rewards to classes for higher participation rates. This year, student surveys are not required of anyone, since folks recognize that the pandemic is going to increase both student and instructor stress and anxiety overall (and anxious students give more negative evaluations - it's almost as if they are still on their learning journey and not yet trained in separating their personal experience from the aspects of a class that the instructor has control over; gosh what a surprise) and switching teaching models to one that students are generally less excited about than regular fully in-person instruction and that instructors have far less experience with will naturally lead to lower student survey numbers, which would unfairly penalize junior colleagues.
So for the multi-section first year course I was involved with this fall, we took the opportunity to come up with our own alternate student survey (which likely still has issues, since this still isn't the area of expertise of anyone on the instructional team for the course, although I guess I've been following the literature a little more closely than most). We also conducted it online. To ensure sufficient participation for the results to be at least somewhat meaningful, we had a system of increasing class-wide bonuses for increasing levels of overall class participation, ranging from 70% to 90% participation. With closer to 90% participation, the bonus is such that it could bump some students a percentage point or so, potentially shifting their letter grade up to the next highest category. So it was a legit, worthwhile incentive from the student perspective. But the likelihood of that participation rate was low, and the noise/uncertainty in assigning marks for different levels of student performance in the class likely high enough this semester, that as instructors we weren't worried that this would lead to students receiving "inaccurate" marks for the course.
Years ago I used to be young and naive and think that doing anything that might influence student responses on the surveys felt dishonest. Now I realize that it's no worse than, say, wearing a suit to your job interview when you don't typically wear a suit to work (or ever). The bias and lack of knowledgeable interpretation of survey results effects at least cancel out, possibly dwarf the positive biases I might be able to introduce through incentives provided to students. 'Course, my hard work or attempts at positive influence of student survey results are statistically likely to have a smaller effects than the same efforts by a white, male, cis-gendered, native English speaking colleague. I certainly appreciate when colleagues in that position worry about introducing unfair biases in their student survey results, since that helps counteract all of the other unfair biases in the whole process, doing at least a little bit to help level the playing field.
posted by eviemath at 8:34 AM on December 20, 2020
Anyway, the student surveys are collected and processed by our Registrar's office, and they've been pushing folks to go to online surveys because they are significantly less work to process. But of course the completion rate for online surveys tends to be abysmal, and you get extra bad data because you mostly only get responses from students with strong feelings, especially strongly negative feelings. Add on to this the fact that the questions and format of our particular survey include questions and phrasing that have been shown in studies to elicit greater bias in responses, plus that no one has any evidence-based training in how to interpret our student survey results (lots of folks have general statistics and research methods knowledge, just not the domain knowledge around how biases show up on student surveys and degree of effect - our periodically required 1.5-hour every three years equity training workshops only point out that there are biases, they don't really have time to go into any details).
Our anonymity requirement means that we're not supposed to know which specific students completed a survey or not. But it is becoming increasingly common to give across-the-board rewards to classes for higher participation rates. This year, student surveys are not required of anyone, since folks recognize that the pandemic is going to increase both student and instructor stress and anxiety overall (and anxious students give more negative evaluations - it's almost as if they are still on their learning journey and not yet trained in separating their personal experience from the aspects of a class that the instructor has control over; gosh what a surprise) and switching teaching models to one that students are generally less excited about than regular fully in-person instruction and that instructors have far less experience with will naturally lead to lower student survey numbers, which would unfairly penalize junior colleagues.
So for the multi-section first year course I was involved with this fall, we took the opportunity to come up with our own alternate student survey (which likely still has issues, since this still isn't the area of expertise of anyone on the instructional team for the course, although I guess I've been following the literature a little more closely than most). We also conducted it online. To ensure sufficient participation for the results to be at least somewhat meaningful, we had a system of increasing class-wide bonuses for increasing levels of overall class participation, ranging from 70% to 90% participation. With closer to 90% participation, the bonus is such that it could bump some students a percentage point or so, potentially shifting their letter grade up to the next highest category. So it was a legit, worthwhile incentive from the student perspective. But the likelihood of that participation rate was low, and the noise/uncertainty in assigning marks for different levels of student performance in the class likely high enough this semester, that as instructors we weren't worried that this would lead to students receiving "inaccurate" marks for the course.
Years ago I used to be young and naive and think that doing anything that might influence student responses on the surveys felt dishonest. Now I realize that it's no worse than, say, wearing a suit to your job interview when you don't typically wear a suit to work (or ever). The bias and lack of knowledgeable interpretation of survey results effects at least cancel out, possibly dwarf the positive biases I might be able to introduce through incentives provided to students. 'Course, my hard work or attempts at positive influence of student survey results are statistically likely to have a smaller effects than the same efforts by a white, male, cis-gendered, native English speaking colleague. I certainly appreciate when colleagues in that position worry about introducing unfair biases in their student survey results, since that helps counteract all of the other unfair biases in the whole process, doing at least a little bit to help level the playing field.
posted by eviemath at 8:34 AM on December 20, 2020
I taught college back in the "I'm going to leave the room now, so fill out the form and put it in this envelope when you're done, and I need a volunteer to drop them off at the department office" days. To me, the most questionable part of all this isn't something the profs are doing, but something the school is doing. Specifically, it seems really odd to me that professors are being given information about which individual students have filled out evaluations, and especially that they're being given that information before grades are submitted. Within that framework, giving extra credit for completing them seems fine to me.
posted by Ragged Richard at 9:29 AM on December 21, 2020 [1 favorite]
posted by Ragged Richard at 9:29 AM on December 21, 2020 [1 favorite]
Without going into the long, dark rabbit hole of the value and efficacy of these end-of-course surveys, I don't think it's necessarily wrong to have a very small percentage of the course grade used to encourage quality, useful feedback about the course. It shouldn't be enough to make a meaningful difference for most students and outweigh the demonstrations of learning that have occurred throughout the course but it's reasonable to create an atmosphere that encourages and rewards meaningful and constructive feedback as a professional and scholarly norm. That, of course, depends on you not viewing the course grade solely as an objective measurement only of student learning; I think that most of us don't hold that view and have at least resigned ourselves to the idea that it's okay or necessary to also use grades as motivation and to try to inculcate appropriate (professional, scholarly, ethical, etc.) behaviors.
posted by ElKevbo at 6:08 PM on December 21, 2020
posted by ElKevbo at 6:08 PM on December 21, 2020
« Older Article about female Jewish roles | quarantine hair dye - covering grey in asian hair Newer »
This thread is closed to new comments.
posted by ChuraChura at 2:09 PM on December 19, 2020 [7 favorites]