Me, Myself, and Eye

February 17, 2006 7:45 AM

What if there was a video game you could play only by moving your eyes across the screen? What sort of game would this be?

The reason I ask is because I have recently been placed in a position where I have the resources to carry out this project if I wish, and so I have begun thinking about the question seriously, and now I wish to discuss it on the internet. I am not concerned with developing a product to sell in a video game market, so don't come at me with logistics issues. What I am concerned with is expanding my own consciousness using video games, and I believe that this is within my means.

Here is the problem to solve: video games work by allowing the player to scan the screen, and then make a deliberate decision to act. What I am proposing is to remove the player's ability to deliberate. It will be impossible in this game to scan the screen without causing some sort of change. What kind of games would even be possible in this setup? Also, what kind of games would be fun?

posted by Laugh_track to technology (28 comments total)

I can't provide any references, but I've seen a lot of news stories over the years on the TV where such games have been produced (very basic ones) and played by disabled people, etc.
posted by wackybrit at 7:46 AM on February 17, 2006

A first person obstacle course might work and be simple to implement as far as the graphics go. You wouldn't need any graphics richer than the old vector based game Battle Zone:

• A horizon line
• 3D projections coming converging on the player as the vehicle virtually moves forward at a constant rate.
If you scan towards the left hand side of the screen you turn towards the left, the severity of the turn proportional to how far left you scan. I think a non-game could be more interesting though, like a light synthesizer. There's a quiet node someplace on the screen and around it there's swirling colours and shapes. If you scan to the quiet portion the activity is attracted to it and the quiet node changes location. Now you've got an interactive light synthesizer.

posted by substrate at 8:00 AM on February 17, 2006

*geeks out*
Ok: First off - watch the new Macross series - Macross Zero - you'll see that the new fighters come equiped with optical scanner/tracker devices that allow the pilots to glance at incoming missiles - which then activated the anti-missile explodey things to shoot out and explode the missiles that they were looking at. It is awesome.

Also see this nice piece of viral advertising: "Busted" Might provide inspiration.

Also, imagine a game wherein you had to navigate a simple cursor through a series of obstacles without looking at bright, red, flashy objects that would cause you to lose. If you looked at them.

What about a digdug type game, where you could tunnel through soil using your eyes in one mode - then in the other mode you would navigate your little digdug dude through the tunnel that you just created, while avoiding tunneling into obstacles?

Mariopaint! With. Your. Eyes.
posted by Baby_Balrog at 8:00 AM on February 17, 2006

Let me go get more coffee. I'll be right back.
posted by Baby_Balrog at 8:01 AM on February 17, 2006

Your question made me think of http://dontclick.it/ which is a web site with buttonless navigation, where all the actions that occur are the result of moving your mouse over parts of the screen. It's pretty full-featured for something that doesn't operate in ways we're used to.
posted by jessamyn at 8:11 AM on February 17, 2006

You should probably read about a few games with unconventional control schemes: Wild Divine (biofeedback), breath control, brain waves.

My initial thought is: even though our eyes are constantly in motion, it is quite tiring to actively control them to do what you want. I suspect game players would get eye fatigue rather quickly if asked to control their eyes in the same way as one would control a mouse.
posted by jellicle at 8:12 AM on February 17, 2006

Talk to some disabled people, people who can move their eyes but not their hands. They might have some good ideas about how this could work and how it could be fun.

Other ideas...

Maybe an "escape from..." game. You're trapped somewhere, bound and gagged, with a first person view through the bars of your cell. For some reason you have a monkey for a friend. (i have no idea why.. perhaps you and your monkey were driving your 18 wheeler in a blizzard in Colorado when you ran off the road...)

There are various objects outside the cell and your monkey friend, who wasn't captured, is hiding outside waiting to help you. Using your eyes, you have to signal to him to collect the objects that can help you escape.

For example, on the first (easy) level, you have to look at him, give the signal to "go get this..." (blink twice or something) and then move your eyes over to a set of keys that he can then use to unlock the door.

Accidentally looking at the wrong thing or giving the wrong signal would be bad. And of course, as the levels get harder there are more deliberate distractions. Perhaps you accidentally looked at a birdcage when the bird squawked, the monkey then let the bird out and the bird flew off with the keys. Now you have to figure out a way for the monkey to get the keys back or to open the cell without the keys.

Guards could stroll by at random times, making it harder to time things.

I have no idea if this would be fun. It may be a really stupid idea. It is an interesting thought experiment though.

Another idea might be a pick-up game where you're in a bar and you have to make eye contact with the right person. Looking at the wrong person might cause you to end up with an STD or at least a horror story. Bonus points can be awarded if you catch the eye of the bartender and, using your eyes to pick out a type of liquor, order a drink for the person you're trying to pick up. Naturally, the more drinks you buy for them the easier it will be to pick them up. As the time is running out (counting down to last call) the more attractive people start to leave until the end when all that's left is the skanky barfly and you lose.

There could even be a "Bondcliff Level" where you always end up alone and the most you can hope for is that you don't spill your drink on your crotch and get laughed at by everyone in the room.
posted by bondcliff at 8:14 AM on February 17, 2006

What I am proposing is to remove the player's ability to deliberate. It will be impossible in this game to scan the screen without causing some sort of change. What kind of games would even be possible in this setup? Also, what kind of games would be fun?

I worked with a group in college ten years ago that was able to allow computer control (patricularly the cursor) with the eyes. They asked me to study it's viability for games. At the time, the ability to deliberate (to borrow your term) was available to me, and I put together an asteroids-type game where the users' blinking constituted the desire to fire a weapon. Moderately exciting. Windows 95 wasn't particularly easy for game programming. Actually, Asteroids and Missile Command were but two of the types of games you could put together with the system. Most of the Atari 2600 library, though, fits nicely.

Removing the ability to trigger/deliberate, though, would seem to limit the whole proposition. Maybe a simple driving game or something of the Tetris or PacMan archetypes might be enjoyable.
posted by clearlynuts at 8:21 AM on February 17, 2006

I would break my own arms just to play bondcliff's monkey game.
posted by educatedslacker at 8:33 AM on February 17, 2006

Oh, and as for an idea...
What about where your eyes control an element like wind or water? For the wind level, you have to do tasks like keep a kite flying. How open your eyes are (or how fast you move them side to side, however you want to implement it) could be how fast the wind is gusting. And then you could throw in other things in the air that you have to avoid (like that old helicopter game where you constantly move to the right and the only control is the one mouse button that moves you upwards).
posted by educatedslacker at 8:40 AM on February 17, 2006

Great idea. But can you make it an input device (a mouse, a keyboard) rather than a game?

An eye-tracking MOUSE I'd love.

But, you'll have to deal with saccades and head movements.
posted by orthogonality at 8:41 AM on February 17, 2006

Pac-Man.
posted by Asparagirl at 9:04 AM on February 17, 2006

Katamari Damacy?

First person shooters use the keyword and the mouse. Could you replace the mouse with eye-tracking?
posted by Leon at 9:07 AM on February 17, 2006

Another thought - if eye-tracking doesn't have pinpoint accuracy, something with flow. Sand pictures?
posted by Leon at 9:08 AM on February 17, 2006

I'm also thinking "eye-tracking mouse." Combine an eye-tracking mouse with a foot-operated button, and your hands are completely free for keyboarding, operating a joystick, etc.

It will be impossible in this game to scan the screen without causing some sort of change.

Then prepare for non-stop change. The player can and will look at everything, and very often without any kind of conscious control. I dare you to NOT look at your right hand right now. Don't do it. See how hard that is? The most minor distractions will become major obstacles to overcome. This model also makes it impossible to predict the player's next action.

What I am proposing is to remove the player's ability to deliberate.

Are you also considering that this takes away the player's ability to mode shift? Think about the look-down/shoot-down models suggested above. They require the player to mode shift -- that is, look at the target and then press a button to shoot it down. Or in the case of a driving game, braking and accelerating are mode shifts.

I'm not sure a completely deliberation-free game is really a game. A toy maybe, but not a game.

Mode-shifting can be context-sensitive, however. I can think of a game where looking at an object allows you to pick it up. Look at it long enough and then the object "sticks" to your vision, allowing you to move it across the screen. Once at another location, the context sensitive scripting knows to "unstick" the object from your vision. So, say you're picking up balls and dropping them into a box. From there, all sorts of gameplay is possible.
posted by frogan at 9:09 AM on February 17, 2006

Pac-Man

I don't want to sound like I'm hammering this suggestion, but it's a perfect example of what I said above about minor distractions being major obstacles.

Pac-Man involves moving an avatar around the screen, but it's also about predicting your moves ahead of time, and coordinating your moves against the movement of the ghosts. This is all done with your peripheral vision while you control Pac-Man with a joystick.

So, take away the joystick and do everything with your eyes. The second you look at a ghost, you literally lose control of Pac-Man for a brief moment. Then you have to visually reacquire Pac-Man to control him again. These constant split-second losses of control will significantly hamper your gameplay experience.
posted by frogan at 9:13 AM on February 17, 2006

Tetris?
posted by iurodivii at 9:30 AM on February 17, 2006

frogan - I think that's the point. It's somewhat a test of will; can laugh-track's mind triumph over the limitations of his flesh? What can we learn from transforming automatic, unconscious eye motions into explicit onscreen movement?
posted by muddgirl at 9:31 AM on February 17, 2006

I think the challenge (if that's what you're after) of any eye-movement based interaction would come mostly from *not* looking at things, as mentioned previously, that or tracking slow-moving objects accurately. Forcing the player to try to determine things through peripheral vision may also be an interesting challenge.

Some ideas:

Put the numbers 1-16 up in random locations on a 4x4 grid, the player must focus on each number in sequence without focusing on non-sequential numbers in order to clear the grid. Repeat. The faster a grid is completed, the better.

Display 4 objects, one in each corner of the screen (e.g. a leaf, a fire hydrant, a dog and a snowflake) In the center present a command to look at one of these objects ("Look at the snowflake"). The player must use perpheral vision to determine which is the correct one before actually moving their eyes to it. Higher levels of difficulty could have more similar-appearing objects. Could also work with alphanumeric characters, colors, etc.

A simple maze. The player must guide their focal point through the maze without looking ahead to the ending.

Present the player with a buxom female who is reciting a long monologue. The player's focus must remain on the lady's face without slipping to the copious cleavage below.
posted by Durhey at 10:45 AM on February 17, 2006

Put a set of bouncing boobs somewhere on the screen. The goal is to see how long the player can go without looking at them.
posted by kindall at 10:50 AM on February 17, 2006

Start with something simple, like Qix or pong.
posted by b1tr0t at 11:06 AM on February 17, 2006

How about one where you try to fool a virtual cop during a field sobriety test.
posted by sourwookie at 11:44 AM on February 17, 2006

This isn't really a game idea, but it might be a factor to consider when you design your game's interface. How would your game account for people who have one of the varieties of strabismus, a misalignment of their eyes?
posted by Fat Guy at 12:01 PM on February 17, 2006

This is such a cool question. I really liked frogan's responses.

The only real user input is whether or not the user focuses on something. Because eye movement is saccadic, control will be discrete, not analog. A little while ago, someone on Metafilter linked to a contest in which people made games that only used a signle button as input. Maybe something like that, where any change in focus will be registered as a button push? (You could choose to redraw the exact same image at the user's new focus spot during saccades, or change it somehow.)

I was also thinking of what a game like this would look like to a spectator who wasn't playing. I've seen these eye trackers used so that most of the screen looked blank to an onlooker, but to the guy strapped in, he was convinced he was looking at an image because the tracker drew a portion of the image at just the space the subject was looking at. Something like this could be harnessed to great effect. Then it hit me: cooperative multiplayer would be amazingly interesting. Different things could happen if the two players were looking at the same spot. Maybe a game in which the two players had to find each other somehow? Or, a driving game in which at least one player had to "keep his hand on the wheel", so the speak... one player has to fixate on the road ahead while the other gets to explore the screen and grab power-ups or shot down bats or something, and they have to trade off every so often. Basically, the job is for one player to make sure the other doesn't get distracted.

You could also hide things in our blind spots, or put colored power-ups outside the area of our spectral sensitivity.
posted by painquale at 12:29 PM on February 17, 2006

I'd suggest downhill skiing, except with your idea there would probably be much crashing into the trees. I think you'd want some type of exploration theme with lots of interesting landscapes or settings, flying through clouds or underwater. Think absolute freedom.

On a personal note, I've tried out an eye tracking system with an onscreen keyboard and, for the love of God, make it something you play without having to recalibrate the tracking of your eye every time you blink. Human beings need to be able to blink, and going through a recalibration--even if it only takes a few seconds--will quickly make your game tedious. The alternative, staring at the screen unblinking for long sessions, will cause eyestrain.
posted by Soliloquy at 12:51 PM on February 17, 2006

Thanks for all your input! Here are some responses:

This isn't really a game idea, but it might be a factor to consider when you design your game's interface. How would your game account for people who have one of the varieties of strabismus, a misalignment of their eyes?

They don't get to play. What I am saying is that the logistics are worked out for me already. The interface is provided to me, and it is acceptable for gameplay. The question is, what is the game?

It's somewhat a test of will; can laugh-track's mind triumph over the limitations of his flesh?

What is this, church? I am not necessarily interested in one part of my own body triumphing over another part.

...a pick-up game where you're in a bar and you have to make eye contact with the right person. Looking at the wrong person might cause you to end up with an STD or at least a horror story.

Present the player with a buxom female who is reciting a long monologue. The player's focus must remain on the lady's face without slipping to the copious cleavage below.

How about one where you try to fool a virtual cop during a field sobriety test.

These are all very amusing suggestions -- I have not even considered the social control direction of this idea. However, that is not something I wish to pursue.

Great idea. But can you make it an input device (a mouse, a keyboard) rather than a game? An eye-tracking MOUSE I'd love.

That sounds quite useful -- I suppose I could, but it's not what I'm currently interested in. Also, I don't know if eye-tracking is really practical in the office, as it usually requires supervision. That said, I am not a business-minded person, and I am not trying to develop something to aid business. My purpose here is to create a fun video game that I would want to play.

The player can and will look at everything, and very often without any kind of conscious control. I dare you to NOT look at your right hand right now. Don't do it. See how hard that is? The most minor distractions will become major obstacles to overcome. This model also makes it impossible to predict the player's next action.

Yes, it is very hard, but it is not as hard as, say, thinking of an elephant when told not to. Training in this area can be useful. What will require training in this game, and what will be natural? The player will naturally be drawn to obvious anomalies in the display pattern on the screen. If noticing such anomalies eliminates them, then this is the beginning of a puzzle game. Now, as you said, what will be hard for the player is not noticing such anomalies. And in order to increase the difficulty of the game we can start thinking of aspects of gameplay that could teach the player to start noticing more subtle things in the pattern.

There's a quiet node someplace on the screen and around it there's swirling colours and shapes. If you scan to the quiet portion the activity is attracted to it and the quiet node changes location.

This sounds like the beginnings of the sort of game I'm looking for. There could be several nodes at one time, and it may be that new nodes are generated by leaving certain parts of the screen quiet for an amount of time.
What about where your eyes control an element like wind or water?

I think you'd want some type of exploration theme with lots of interesting landscapes or settings, flying through clouds or underwater. Think absolute freedom.

This is starting to sound less like a game and more like a medium for creating art. That idea is very fascinating but I would like to stick with the game format. It could consist in controlling the pattern of flow around the nodes towards a certain end (rescuing a princess?). Maybe you must protect the princess from the lava/wind/whatever you are controlling? Or maybe you must guide an object, using the flow, into another object, while avoiding still other objects. There could be various baddies wrestling with you for control of the fluid as well. These could take the form of specific beings or general patterns of screen activity.
posted by Laugh_track at 2:41 PM on February 17, 2006

I would like to play a first person 3D-style Robotron like this.
posted by First Post at 10:22 PM on February 17, 2006

Whack-a-mole!
posted by 999 at 7:42 AM on February 18, 2006