Help me help the next generation of coders!

December 28, 2009 10:22 AM

What are your best methods, tips and tricks for grading student code?

I need to develop a better method of grading my students' work in programming classes.

For example, how much should simple homework projects, due once a week, be worth compared to more difficult assignments, exams or a larger final project?

What is good feedback? I honestly wonder if any of my students even look at the comments I make, since they seem to repeat the same errors over and over. What should I do instead? Should I attempt to fix their program, explaining what I did so they can see the difference?

How can I reduce or eliminate cheating? I allowed students to discuss their work with each other in lab, thinking that it would foster a sense of teamwork. I was unprepared for how many students felt that was an invitation to simply copy their classmate's files and submit them.

So, any ideas? If it matters, I'll be teaching a C# class this semester, and regular old C in the summer.

posted by SuperSquirrel to education (17 answers total)

If you can, structure your assignments into smaller modular components so that you can apply automated unit tests to each component.

Of course, if the students are supposed to figure out the modular design on their own, this won't work.

Will you have TAs?
posted by tss at 10:38 AM on December 28, 2009

Are these college level courses? If so, I think you should make the simple homework assignments worth few points, and the exams or final projects much more.

Think of the homework assignments as ways for your students to learn the material, fix their mistakes, improve, ask for help etc.

Think of the tests and exams as ways to measure how much they've learned.

If I were you, I wouldn't stop allowing the kids to discuss their work. If all they're doing is copying files they're only hurting themselves, and it'll be evident on their tests and other projects.

(Note: I have no programming experience)
posted by kylej at 10:41 AM on December 28, 2009

Sorry, should have mentioned that in my question. These are introductory college-level courses, and generally the students have little or no programming experience coming in. No TAs, but they're small classes.
posted by SuperSquirrel at 10:45 AM on December 28, 2009

Re: cheating
You can have the same overall theme of the assignment, but adjust a portion of it per student. Though this will depend on the size of the class.

Re: teaching
On thing I would say really held me back when I was learning CS in college, was the lack of knowledge of how to start the program. Setting up the environment and class structure and all that. If that's well explained, with well commented examples, it should help set up a framework for them.

The project I learned the most in was in DBMS class. We worked in groups, but each group had to come up with their own "software", I chose an auto dealership, that was then approved by the teacher/TA. Once it was approved, we had to code it on our own, with periodic check ins. That kind of structured approached helped.

hope this helps.
posted by pyro979 at 10:46 AM on December 28, 2009

A lot of your questions are about basic course design and have nothing to do with programming (except in application). It sounds like in general you'd benefit from looking into that topic in general (for example).

I've generally found students to be respectful of anti-cheating rules so long as you articulate them clearly. "You can discuss the project among yourselves" contained (for you) implicit assumptions about not literally sharing code. You need to spell these out explicitly, in writing.

I honestly wonder if any of my students even look at the comments I make, since they seem to repeat the same errors over and over.

While this could be partly the result of poor feedback (again, I vote for short, clear, hyper-explicit) you also need to remember that students go through developmental stages where it's going to feel like you're banging your head (or maybe theirs) against a brick wall. Between about 16-21, almost all students go through a period where they seem impervious to many kinds of feedback, only to see good behavior somehow emerge from them months or years later. Most of us don't remember this stage of our own development because it's totally invisible to us, while our teachers likely went home and drank heavily.

I am continually amazed by my own students -- I will sometimes hear them repeat, verbatim, something I told them years ago and would have sworn bounced right off their forehead and onto the ground. Usually these statements are delivered as if they have just discovered this amazing new principle entirely on their own, which I suppose in a way they did. The joys of teaching.
posted by range at 10:59 AM on December 28, 2009

In all my programming classes so far in college we weren't allowed to even discuss our work on the current project with one another, only "the project requirements, the C language, what was discussed in lecture and discussion section, and general debugging or syntax errors" (from syllabus). If you make stuff like this clear, kids will (generally) feel like you take this stuff seriously and will back off.
posted by azarbayejani at 11:10 AM on December 28, 2009

I used to teach university students (intro level --> grad classes) in a different topic (biology), but I think this would/could still apply.

Homework was worth little to nothing - the main point was to learn it by a certain point (exam, quiz). These are adults, they can use the feedback, learn by a certain time point, it didn't matter to me how they go there but that was my own philosophy.

This is what I did to automate the grading yet give them feedback. Although it took some work up front, I used tools such as blackboard or the online support you can use for your class. I put questions that were similar to what they would see on a test or quiz. Each answer gave the student feedback (so if they misunderstood a concept, there was feedback) - you do have to know where a student will go wrong, though. The student could take the online assignment as many times as he/she wanted. Anywho, I had an instant record of who did the assignment, and the students had feedback.

If I saw a consistent error on assignments (online or graded by hand), I went over it in class - I agree some students will not look at the feedback.

Just some ideas on group work/learning from peers:

Group work yet not encouraging slackers. I thought about this a great deal. I did occasionally assign group projects. However, I set it up so that the student also graded one another. It worked pretty well (if a student or students did all the work, they would point it out when they graded their peers -- in fact, the entire team did this/I was shocked at how well this worked). You may want to see what they do at actual work places - I have current friends that do coding and there is a practice at some workplaces of one person coding, the other watching over/double checking, then they switch places. If you approach it as a skill that they may do at a workplace, perhaps you can attempt it.

Cheating (or whatever I considered cheating) = 1) I put it on the syllabus 2) discussed it on the first day of class 3) put a quiz question about the cheating policy. Some idiot still attempted obvious cheating
posted by Wolfster at 11:37 AM on December 28, 2009

One simple thing: return the graded assignments as quickly as possible. If it takes any longer than a week for assignments to be returned, then I'd bet a good fraction are never reading your comments, or at least aren't spending more than half of a second thinking about them. This is especially true of code, where it takes some real effort to think about one's mistakes when you're looking at something you wrote three weeks ago.
posted by kiltedtaco at 11:46 AM on December 28, 2009

You could develop rubrics for your various assignments. Students are used to working with them and it will tell them exactly where you are taking away points and what you are looking for. For example, a project might be worth 50 points:
10 points - creativity
20 points - demonstration of learned concept
10 points - overall project
10 points - spelling, punctuation, grammar

Or something like that (I know nothing about computer programming)

Also, I allow my students to make corrections for half credit. This is a really great way for them to fix the mistakes they made and hopefully learn from them. So if they got a 40/50 on a test, if they make the corrections and they're right, they get 5 points, for a score of 45/50.
posted by NoraCharles at 11:57 AM on December 28, 2009

If this is really base level stuff for people with no experience (which it seems to be) what my professor did when I was in a simialr class was make us submit a paper where we had to write out why we used each process that we did. This helped with understanding for the students and the teachers were able to see why a student would make an error that seemed stupid. Also, it made each individual do at least some original work when all the student's code was really similar, if not the same, due to limited vocabulary and simple homework where there seemed to only be one obvious way to do it using what had been learned.
posted by WeekendJen at 12:08 PM on December 28, 2009

I took a high school Java course last year. I'm not sure how well this would transfer to a college class, but this is what my teacher did, FWIW.

We got regular programming assignments after each new concept in class. We could go home, work on it, and discuss with each other. Then, the next day, the teacher would have the same assignment on the board, and we'd have to write down a segment of code. He'd then grade it as a daily assessment. This method allowed us to collaborate, but ensured that we understood the concepts and weren't just copying.

As to how you could grade this, you could make each daily grade worth 5 or 10 points. Add them to a running total of the semester. At the end, find the percentage of points they got and make it equal to a test grade.
posted by estlin at 12:35 PM on December 28, 2009

There is an enormous body of evidence that students learn better from well-implemented group work, like Asera & Treisman (1990). I've never personally seen any an analysis of poorly implemented group work, as education publications almost never apply the scientific method.

I always encourage group work myself based upon this evidence, but (a) ask questions that are sufficiently hard that students cannot solve them without teamwork, and (b) demand that written solutions be "in their own words". Btw, if I had control over group size, then I'd require 2-3 only per group.

I believe returning grades quickly is far more important than writing comments. I shoot for returning grades via an automatic emailer script within three hours of the assignment being submitted. So student know the grade well before they've dumped that assignment from short term memory, even if they never collect the assignment.

I think computer science classes usually require electronic homework submission so that preliminary grading may be done using unit tests while possible cheating is flagged using systems like moss.

I'm sure students will take plagiarism seriously once you submit several for academic disciplinary proceedings, which will become extremely easy once you start using moss.
posted by jeffburdges at 12:44 PM on December 28, 2009

For cheating, strip out comments, and whitespace (all contiguous whitespace goes to a single space), change all identifier names to the same thing, and then diff the files. Or use Moss, but that way works remarkably well and shouldn't take very long (presumably C# has a parser available that lets you apply a transform to all identifiers?).

One very effective way of making sure they read the comments is grading harshly and offering partial points back for a corrected assignment, i.e. you make comments, they read the comments and fix what was broken, and you give them 2/3 credit back or such.

Please, please don't make group work for an intro CS class, people are at very different stages and group work is incredibly unfair to those people who know what's going on, and even those who know what's going on probably don't know it well enough to teach it to other students - you'll end up with a few students doing the others' work. Group work is excellent when all the students have a similar base, but not before.

One CS class I took had the cartoon rule for collaboration: after k minutes of discussing a problem together, you erase the board, trash all the notes, and then do k/2 minutes of something utterly mindless (e.g., cartoons). I don't think anyone did it, but it got the point across.. you shouldn't be going from discussion right to a writeup, because its still not your own work at that point.

Making them submit justification for decisions sounds like a good idea - if you can figure out a decent way to get them to do it, it might help quite a bit. That way, even if they're all working on it in a big group, they'll at least have to write down the reason someone else came up with, and hopefully learn it then.
posted by devilsbrigade at 1:10 PM on December 28, 2009

When I took a bunch of programming classes in high school, we had the benefit of about 30% teaching and 70% lab time in-class. This was very helpful for those that weren't the programming-type and didn't just get it.

In college, we basically had lectures and no structured lab-time for most of my Computer Science classes, so all work was done on our own. On the other hand, we typically saw each other regularly in the campus computer lab doing our homework, so it ended up being pseudo-structured lab-time, anyway. Then again, this was back in the 90's when our development was done on unix and x-windows stuff for graphics, so we didn't have the luxury of Java and C# where we could do it on our personal computers. Kids these days...they have it so easy...

In my intro-type classes, a portion of lecture time was focused on going over homework for the week so that was definitely helpful for those who struggled. This wasn't the case in the 300, 400, etc level courses since by that point the students should be competent to do the material. Also, after being out in the real world writing code, I don't think it would be unreasonable to identify your "good programmers" of the class and task them with mentoring, or have them lead any group work. This mirrors a typical team structure in the office where you have a tree of architect, senior programmer, junior programmer and the mentoring that goes on. So I guess if you do include group work, don't let all the techies group together and knock their project out in a couple days while the other groups struggle to finish the big project. Injecting this type of business-world type team model into college I think will give your students a lot of soft skills once they get out there.

As far as the programs I had to write back in those intro days for the weekly-homework type stuff, they were VERY simple. They basically just demonstrated your mastery of the current lecture topic at hand...for example, the age-old "Write a factorial method to demonstrate recursion." Make sure the students either write their own test harness, subscribe to your unit-testing tool interface, or even provide them with the framework code. This way it will be a very simple program to grade/debug the code visually along with being very simple to run for the professor to test.

In fact...I rather like the idea of providing the framework program for each assignment (at least early on in the semester while they come up to speed). Frankly, if you have a lot of non-CS people or utter beginners to programming, chances are they'll spin their wheels trying to just get a simple program to compile/run and waste tons of time instead of really writing code to demonstrate the concept they are learning. Then, they tend to never learn any true programming concepts - instead, if you give them the stubbed out source code with 90% of the grunt-work written for them, and they just need fill in assignment-specific code, then they focus more on the true concepts. Plus, you can control the code for any test-harness you want to plug their stuff into in order to quickly verify their work. I'll take a co-worker that understands basic programming concepts any day who struggles to setup a jar file, classpaths, compiling, etc over a co-worker that can compile their code but has zero understanding of simple concepts like not writing the same chunk of code multiple times without creating a utility method, or just can't seem to grasp iterating over a collection, or who recognizes when their code suddenly turned into an O(n^n) algorithm because of a bad looping decision. This again models the real-world in my eyes with team dynamics - the junior programmer isn't writing all code from scratch, rather, they're handed work from their superiors and it's more like "write just this method to do X, Y, Z." I'm guessing you'll find this will REALLY help your beginners, and for those students who easily understand the material it will frustrate them less with the "busy work" of all the code surrounding the simple assignments leaving them more time to "play" with their new knowledge.

Finally, I don't think anything is wrong with peer-collaboration, but I would suggest being very harsh on straight up copying. I straight-up-copied code from a peer once in my whole programming education. It was that last assignment of my last semester in college, and I was just tired of the busy work when trying to prepare for graduation. Luckily, I had the foresight to change the code with my own comments, alter variable names, and some simple re-ordering of logic...but others in the class who also copied did not do this and the hammer came down hard. Many 0% grades on that dark day...but I bet they didn't do that same thing again. On the other hand, if it were me I'd allow them to redo it themselves and present their own work for half-credit. [In my own defense, my copied-code was for an Assembly-programming class on embedded processors, so I had enough of going to the damn lab to program my 68HC11/EEPROM to truly test my code...so I said screw it and just scrapped my own work that I didn't want to test and copied a friend's known-working code]

Finally, drill into your students heads that comments are very important. Make sure they comment their code so they not only help you understand their thought process when you have to debug their work, but also so they get into the good habit of writing code that can be handed to another person to work on. In my early classes we spent a lot of time in the beginning learning to stub out our code with comments - so we basically put in comments of pseudo-code first of what we want to do and then compile it. Once it compiled, we'd turn this in as Part A of the assignment...then we'd fill in the real code between the comments, and the working code would become the Part B turn-in. This was done for the first month or so to get used to the compiler, writing comments, organizing the algorithm from head-to-code, etc. It also let our teacher correct logic mid-process with that Part A turn-in before things got too far gone. We also spent a lot of time learning about and commenting loop invariants and assertions in our code to make sure we are keeping in mind the state changes during the looping. I think this really helped the casual programmer visualize what was happening from the software's point of view on each iteration.
posted by JibberJabber at 2:04 PM on December 28, 2009

I TA'd several of these courses (which included teaching the lab section).

1) Cheating is painfully obvious. The only people cheating are the people who have no idea what they're doing. As a result, they'll be completely incapable of obfuscating their dishonesty.

I always allowed people to discuss, but not copy work.

2) Homework should, in total, be about 10% of the course grade. But, be generous on grading, and make lots of assignments (weekly at least), so that good students aren't screwed out of an A by one mistake.

3) I never found a good way to grade programming assignments, other than running the program, then reading the code. Automated testing doesn't help you catch huge semantic mistakes, like naming every variable barf, barf1, barf2, berf, borf, borf4, etc. (true story).
posted by Netzapper at 2:47 PM on December 28, 2009 [1 favorite]

Ask around at your school for automated suites they use for testing computer science projects. There should be a standard. Has a package been site licensed to your school? You should be able to just run a suite after the unit tests to look for cheating.

Seconding Netzapper, as a former TA I can also attest that it is really easy to spot the cheats if you read the code for style. I can't remember the precise breakdown, but a good 25-40% of the grade was on coding style (proper encapsulation, good naming, good comments, asserts, etc). Each code style convention has a number of points allocated to it on every assignment, and you deduct points (with short comment) if conventions are not followed. If students pay a grade penalty for not following the conventions, they tend to change their behaviour over the course of the term. Our style marks went up rapidly as the term progressed.

I would not bother trying to fix any given program. However, it would be helpful if you were to circle the line in red where things started going south for any program that failed unit testing.
posted by crazycanuck at 9:49 PM on December 28, 2009 [1 favorite]

First of all:
There is an enormous body of evidence that students learn better from well-implemented group work, like Asera & Treisman (1990).

Please, if you cite something, go ahead and include a full reference, otherwise it's a bit frustrating, honestly. Not everyone has heard of the authors/article you're talking about.

I took a CS course in college and the professor included stubbed out code with a test harness. This is incredibly helpful. It gets the toughest yet least relevant parts out of the way and also is a helpful nudge in the right direction. Also, if I was required to type out all of the beginning code myself, I might be tempted to just copy and paste a basic stub myself.

Obviously, at a bare minimum the code should run/compile, so I assume the errors you're talking about are errors in which the problem is in what happens during the program run.

To reduce those errors, I suggest upping the ante. Instead of making the program abstract, make all the programs run off of a theme if possible -- say, the operation of a nuclear power plant or a shuttle launch. That way, instead of just writing "You forgot to check for a null value" or something, you get to write "Your test failed to check for null values, so when the heat sensor went offline for .2 seconds, the program entered an infinite loop, causing the main reactor to overheat and eventually explode."

Similarly, make the unit testing interesting. Say you have a program that is supposed to read in data. The test script could say something like

Engineer is entering data...
.
.
.
The engineer suffers a heart attack while entering data! EOF! EOF!
EOF safely handled by your code!

The truth is, if most of these students do end up with programming jobs, most of their work is not going to be life-or-death, so this is an opportunity to make it a bit more exciting for them now.
posted by Deathalicious at 9:12 PM on December 29, 2009