a remote exam under exam conditions
April 28, 2011 8:01 PM   Subscribe

How to record something digitally in a way that verifies it has not been 'shopped

This is an experiment in remote education.

I want to film myself taking an exam as a way of verifying that the exam was done under exam conditions, without needing someone physically there to.. what's the word I'm after... invigilate the exam.

There are many problems to solve here and I am happy to hear your suggestions regarding any of the problems you imagine and possible solutions to them, but to maintain a bit of focus i want to concentrate on verifying the authenticity of the recording.

I have been imagining a version of the classic fitenss magazine before and after photos.

(where someone stands holding a copy of last months generic fitness magazine and their distended bare belly and then another photo the following month of them holding this months issue and their now rock hard abs after just one month of generic fitness magazine's one-month-rock-hard-abs-program)

So here's my current imagining:

The exam will be made available to me at a specific time*

The I will complete the exam so the work is done in view of the camera.

In the field of view of the recording there is a screen which is showing a random sequence of QR codes.

These QR codes are generated by a server which is recording the time each code was served.

Each frame of the video is then sent back to the server that makes a hash from the frame and the time using a secret key.

There will be a time lag between when the QR code was served, and when the frame containing the image of that code was hashed but it will be short.

If the video is edited, the correspondence between the frames, the times, and the QR codes will be broken.

So any trickery would have to be happening real time or it would be revealed.

In addition there will need to be a similar mechanism to verify the audio recording.

When I am finished the exam I will show all the work to the camera and place it in a provided envelope and close the security seals all in front of the camera.

The recording, the qr codes, the times and the hashes will be permanently stored on the server.

Does a system - or parts of a system - like this already exist?
If not, do you have any implementation suggestions?

Ok here's the extra peripheral stuff I don't want to be the focus of the question because otherwise the question just gets to big

I'm not sure how to deliver the exam, probably via the screen showing qr codes, where gestures in front of the camera provide a simple interface for navigating the text so that the I do not need to be provided with any kind of controller. Ideally I would do all the work on a whiteboard so the work is visible as it is done, but I don't know if this is a practical condition to impose on students.

Perhaps a convex mirror would need to be at the front of the room, but then what about the roof? What about messages sent by an accomplice using lasers shining on the floor? the list of potential exploits is long. Once the system is set up I'm going to try cheat it, and put the videos online and see if the public can accurately detect the cheats, and if not revise the set up.
posted by compound eye to Computers & Internet (17 answers total) 2 users marked this as a favorite
Why not just have CNN playing in the background while you're taking the exam? You might argue that you could edit the CNN feed into the video after the fact, but if so couldn't you also just edit the whole series of QR codes into place after the fact?
posted by alms at 8:08 PM on April 28, 2011

In security, we would ask: what's your threat model? Does the cost of preventing an attack equal the probability of a successful attack times the cost of that attack?

The easiest way to do what you want would be to just have a human being watch the video feed and occasionally ask the test-taker to raise his hand (to verify that it's real-time). Oh, and we need a really high quality video feed so you could see eg. a scrap of paper or a Bluetooth headset. Let's call that Plan A.

This prevents one attack: a student taking the exam early and then pretending to take it for the camera (Attack 1). It does not prevent an accomplice giving answers to the student off camera (Attack 2).

The system with the QR codes is an elaboration of Plan A, so let's call it Plan B. Plan B would be incredibly expensive to develop, and it would be vulnerable to a new attack: an ingenious student using real-time video editing software to overlay the current QR code on an old video feed (Attack 3).

Ok, so now we have to prevent Attacks 2 and 3. I think Attack 2 will require some kind of specially designed room. Or maybe a 360-degree camera (they do make those, they're not even too expensive). Maybe Attack 3 is too unlikely to worry about.

In real life the right way to do this would be to send the student to the nearest Prometric location, which will happily give them any test you want and supervise the student taking it...for a fee of course.
posted by miyabo at 8:18 PM on April 28, 2011 [1 favorite]

Response by poster: nice idea alms.

The qr code would be served at time A,
then the frame would be hashed by the server at time B.

This verifies that the frame was made between time A and B

If there is a contiguos sequence of frames with matching sequence of verified times, then it makes it impossible (i think) to have edited the video afterwards.

Neither the exam or the video could have been done in advance because the exam wasn't available in advance, and the video shows the work being done.
posted by compound eye at 8:19 PM on April 28, 2011

Why not just have CNN playing in the background while you're taking the exam?
An exploiter could simply record whatever CNN is playing at the appropriate test-taking time, and then play the recording back while actually taking the test.
You might argue that you could edit the CNN feed into the video after the fact,
The video wouldn't need to be edited at all.
but if so couldn't you also just edit the whole series of QR codes into place after the fact?
Presumably the point of sending the video back to the server in real time and generating a hash from a frame and the current time is so as to be able to later verify that the frame really was recorded at the time that it purports.
posted by Flunkie at 8:20 PM on April 28, 2011

This seems extremely delicate: prone to having the video be "stalled" waiting for more QR codes if the connection between the server and the video station is interrupted or lossy. What are the assumptions about the minimum amount of time necessary to alter a frame from its original composition? And why wouldn't I just modify everything EXCEPT the QR code assuming I could do the modification fast enough?

Basically here what's happening is that the QR code acts as a verification that any frame containing the code was recorded in the interval between its generation and when the frame was received. If your assumption is that it's impossible to perform frame modification fast enough (in that interval), I think that holds.
posted by axiom at 8:23 PM on April 28, 2011

You videotape the exam. Digitally sign the exam video file immediately. The verification of this digital signature depends upon the video bytes remaining untouched (as well as the integrity of your secret key). If the video data get 'shopped, when you try to verify the digital signature with the edited video and your secret key, you get the wrong result.
posted by Blazecock Pileon at 8:27 PM on April 28, 2011

Maybe I've misread the question, but simply taping and signing doesn't seem to suffice. It just proves that the video file wasn't tampered with after being signed. It doesn't prove that the video file was created at the appropriate time.
posted by Flunkie at 8:32 PM on April 28, 2011

If the video is being streamed to the server anyway, and if there's no concern of the test being taken too early (since it will be "made available at a specific time" - presumably at the valid start time), then I don't understand the need for all the complicated QR stuff and hashing and whatnot. The server can just record the video as it is streamed in, and it knows what time it is.
posted by Flunkie at 8:35 PM on April 28, 2011

It doesn't prove that the video file was created at the appropriate time.

If the video contains a timestamp that is part of the signed video data, then the timestamp cannot be forged or altered after the fact, without breaking the signature.
posted by Blazecock Pileon at 9:09 PM on April 28, 2011

Yeah all you need is live streaming of video of the exam-taking to a server, the exam to be made available at a set time, and there to be a set window for submission of the answers. That way you can't take the exam early, you can't take it late, and the exam-taking process was recorded. If you provide three feeds with slightly overlapping camera angles you can show 1) your face and the surrounding part of the room 2) your hands typing and 3) the screen. Toss in server-side editing of the exam answer file (a la Google Docs) and most cheating avenues are closed off.

The main attack vector I can think of is someone who is not in view of the cameras but can read the screen (e.g. either a carefully positioned person or someone using a remote screen viewing program) reading the questions, looking up answers on a separate computer, and telling you the answers. A high quality audio feed from the cameras would help with that, but there may be other ways to communicate (e.g. being fed Morse code via a machine that lightly taps on your foot). At that point you're dealing with a pretty sophisticated cheating setup, though.

And this assumes the entire exam can be taken without any breaks. Once the test-taker leaves the room all bets are off. Maybe equip them with a helmet cam and mic?
posted by jedicus at 9:15 PM on April 28, 2011

Response by poster: Thank you everyone for all your suggestions, even though I want to think about them for a while before I reply, I want to post this note now to convey my appreciation.
posted by compound eye at 10:35 PM on April 28, 2011

Response by poster: OK, forget my suggestion of how to do it,

How would you do it?
posted by compound eye at 12:26 AM on April 29, 2011

given the two general attack classes of taking the exam at a time other than represented, and a 3rd party passing info to the examinee,

the first can be solved by closing the time window - make questions available on-demand, and stream the video back to a trusted server in realtime.

the second can't be solved absolutely, but to even get close you're going to need either a very expensive technological solution (perhaps including a faraday room) or a much cheaper human proctor.

some problems really are better solved with meat than metal.
posted by russm at 1:38 AM on April 29, 2011

Response by poster: Hello Hades,

make attacking the system more work than just learning the material

I see the goal as to make the a system that is as robust as having a human proctor in the room, not a perfect system, but a good enough system, and to do it with the minimum of set up requirements for the student.

So the twin exploit is not a big concern, both methods are just as vulnerable. The third party assistant exploit needs to be as risky with system as it is with a human proctor in the room.

If I throw away the goal of minimal set up requirements for the student, I was thinking the student could do all the work on a sheet of glass with a white board marker, filmed from both sides, and the qr codes could be projected on the student and the glass, from projectors offset at an angle so that the qr image is distorted across their body.

I'm trying imagine the simplest solution that can replace a human proctor, but I'm not necessarily looking for a solution that is simpler than a human proctor.
posted by compound eye at 3:05 AM on April 29, 2011

Response by poster: how do you know that the person who appears on the video is the person registered for the class?

the video would stay on the server as a document testifying the person had completed the exam.
If the video isn't them, we might not know. But the video will be a public document available on the server, waiting to embarrass a cheat.
posted by compound eye at 3:17 AM on April 29, 2011

I guess I'm not 100% sure what you are trying to guard against.

If you want to make sure that the person taking the exam is doing so at the correct time then make sure that they have three hours (or whatever) from receipt of the exam to finish and return it (upload to website or whatever). The QR codes can prove that the test is done in real time, but if you set the start and end then the test pretty much has to be done in real time.

Maybe I'm dense, but it seems to me that you've come up with a solution without defining the problem. A test proctor is a solution to a problem, but unless I know what you think the problems are then I can't come up with a different solution that guards against them.

Possible problems:

1. The student has looked up the answers before getting the test
2. The student looks up the answers during the test (taking a break to do it. See 4)
2a. The student looks up the answers while taking the test (without a break)
3. The student is gven the answers by someone sitting next to them
4. The student takes longer than the specified time to take the test
5. Someone else shows up and takes the test for the student
5a. The student takes the test, but at the last minute swaps in someone else's answers
posted by It's Never Lurgi at 8:55 AM on April 29, 2011

Technological solutions are going to be easily defeated. I'd have a "virtual proctor" administer an oral exam. Have the proctor give an anti-video instruction, e.g., raise your right hand, then read off a question from a bank. I would have the tester rotate 90 degrees every question to reduce the impact of answers or notes posted out of sight. I would require hair to be pulled back so ears are exposed.

The exam would be administered via a flash-based web app that communicates back with the server to download each new question, so there'd be a flaggable time series to match against the video feed.

I think this reasonably mitigates most attack scenarios.
posted by bfranklin at 1:59 PM on April 29, 2011

« Older Is professional detailing worth the money?   |   Insight as to why this guy who is interested in me... Newer »
This thread is closed to new comments.