Of Rods, Cones, Meat and Monaco

This was orginally a guest post I wrote on the gifted game author Andy Moore‘s website. It is reprinted here without his permission, or indeed any regard for how he might feel about me duping content on his site.

I’ve spent the last few years prototyping games and generally trying to get my head around the discipline of game design.

This post is about the latter.

As you know the human eye uses Rods and Cones to detect light. Cones see colour and Rods see black and white. Since Cones see in colour and Rods don’t you’d think we could ditch the rods and just use the whole eye to see in glorious colour. Unforunatley Cones suck at a lot of things. In fact Cones are so bad at most things we don’t have very many. The human eye contains abut 7 million Cones and about 100 millions Rods.

The biggest thing Cones suck at is seeing in low-light conditions. This is why dogs see better at night but also see in black and white. They haven’t sacraficed any retinal real-estate to Cones. Cones also suck at detecting motion. Our eye harneses groups of rods together to be incredibly good at detecting motion and it can’t do that with Cones. Mabey because of this some of the pathways in our brain that process motion are also bad at dealing with colour.

This is pertinent to video games. Especially games which have a lot of motion and rely on quick decisions. Like Super Meat Boy for instance, or Monacco.

I recently started playing the PC version of Super Meat Boy. I have been eagerly eagerly eagerly awaiting the release of the PC version of Super Meat Boy and it has not disapointed. It is a wonderously tuned piece of art. The pacing and the feel are perfect and it is seriously cutting into the time I’ve spent working on my own game. But I noticed in a few levels that I was having a hard time keeping track of Meat Boy as I sent him careening around the levels. After dying a few hundred times I began to wonder if it was related to how my eye perceives motion. Here is a screenshot of a Super Meat Boy level I have trouble with:

And here is the same image completely desaturated:

This image is how the motion sensing part of your eye and brain sees Super Meat Boy. Notice how Meat Boy fades into the background? That means he’s almost invisible to the part of you that detects motion. This is bad for a couple of reasons. Mostly because you Fovea is so small.

The Fovea is the part of your eye that can see details. It’s where most of those 7 million Cones are. Inside the fovea the rods and cones are packed into a tight latice and we have very good visual acuity. Outside of the fovea we only have a vague sense of what’s around us. Outside the Fovea we have an sense of patches of bright and dark and we are extremely queued to see motion. The problem is the Fovea is tiny it’s about the size of your thumb-nail at arms length. That means we can only see a very small part of the world with any clarity. It doesn’t seem that way to us because our brain is so good at filling in the gaps. You know how you don’t notice the Blind Spots in each eye? Your brain is filling them in so you don’t notice them. It’s also doing similar tricks for pretty much 90% of your visual field.

Your brain actually builds up a scene by using tiny eye movements to flit your eyes across your field of view. These tiny movements are called Saccades and you need them to build up an image in your head of what’s out there and what’s happening. Motion is a major driver of Saccades. If something is moving your eyes will quickly move to focus your Fovea on it, take in all the details, and then flit back. Your brain will then super-imposing the data it has gained on your view of the outside world. This makes you think you have a pretty good idea of what’s going on out there even thought you exist in a fog of uncertainty

Meat Boy blending into the background causes two problems for this system. The first is, we can’t track him very easily. The part of the visual system that keeps a moving target in view is very old and very honed and it needs the motion sensitivity of your rods to do its work. If you rob it of those rods it will have a harder time doing its job. The second problem is reaquiring Meat Boy after you Saccade off to the ledge you’re trying to land on. Imagine you hurl Meat Boy off a cliff planning to land on a narrow ledge farther down. As Meat Boy travels south you can’t fit both him and the ledge in your Fovea at the same time, it’s too small. But you need acurate information on the ledge and on Meat Boy in order to land on it so your brian Saccades your eyes back and forth. It’s pretty easy to Saccade to the ledge since your brain remembers where it is. But Saccading back to Meat Boy is harder since he’s moving. In most levels your brain manages without too much trouble. It uses the movement data your rods are feeding it to keep track of where Meat Boy is moving while you Saccade off to the ledge. Then you can Saccade back because you know where his new location is. But in the levels where Meat Boy is of a simmilar brightness to the background your motion sensing system suddenly goes blind, you miss your Saccade and Meat Boy falls to his death two or three hundred times.

Note that this is only a problem in a handful of levels and it doesn’t erase your ability to play, it just makes it a little harder. It is a good example, though, of how knowledge of the eye and the brain can make you a better game author.
Now. If I haven’t totaly bored you to tears lets talk about how this relates to Andy Schatz’ upcomming game Monaco. Like Meat Boy, I am a giant fan of Monaco. I spent most of PAX this year playing Monaco. In fact it was all my Monaco playing that got me started researching all this eye-brain stuff in the first place.

In a very real way Monaco is about Saccades. You Are Not So Smart is a relatively interesting blog on how the brain operates and the author, David McRaney, managed to coin one of my favorite quotes: “Reducing chaos into a manageable mental state is a constant battle”.

A lot of our brain is dedicated to making sense of the world around us and multiplayer Monaco is a direct gloves-off challenge to this ability. It requires an extreme level of Situational Awareness. That’s a fighter-pilot term. Situational Awareness is how well you understand the state and location of all the important variables around you. Those variables might be MiG fighters or they might be french goons looking to knock the block off whover took their employer’s shinies. In Monaco you have to worry about the goons but also, more importantly, you have to worry about your team-mates.

Monaco is about working as a team and you can only work with your team-mates if you know where they are and what they are doing. The pacing of the game is such that verbal communication is often too slow to adequately coordinate everyone’s movements. You all have to be actively keeping track of what’s going on and making decisions that will further the goals of the team. In this sense Monaco is more like a basketball game than a game of Chess. In this way Monaco is about Situational Awareness. And that Situational Awareness comes from your eyes and the processing of the information your eyes provide.

Monaco sets the motion sensing part of your brain all alight. Like christmas tree lights. Your brain is trying to keep track of everything and is sending your eyes flying all around the screen trying to keep your mental map up-to-date. Andy Schatz hasn’t ham-strung our Rods by making the players blend in with the background. Monaco represents a fair challenge to the sensory system. But there is a lot of extraneous movement to draw the eye in the form of fog-of-war. As a player enters a room the scope of their vision rapidly opens up which makes objects in the room much brighter. This brightness is in turn noticed by our perephreal vision which then Saccades the Fovea off to identify it.

This is troubling because usually we don’t need that information. We don’t need to know the contents of the room someone just walked into unless it contains a guard or a shiny. We should be satisfied with the vague notion of our teammate being “off in the top left cooridor somewhere”. But every movement of our team-mates is providing many pings of movement to our brain. With so much movement the location of guards and team-mates becomes lost in the shuffle. If you played Monaco while the fog-of-war was tile based you’ll know exactly how this works. Each tile used to change brightness so each tile represented a possible source of attention for our brain. It was very difficult to seperate important input from the constant bombardment of flickering tiles.

Interestingly, I think this could be a good thing. Just as Situational Awareness seperates fighter pilots into the skilled and the dead it seperates Monaco players into the rich and the jailed. Brains are magical learning machines. Unlike the small handful of Super Meat Boy levels Monaco doesn’t blind our visual system, it just presents it with a very hard challenge. A challenge your brain can improve at subonciously. Think how much of getting better at Super Meat Boy is subconcious. It’s about training your brain with practice just like Situational Awareness in Monaco. That’s pretty much the definition of video games.

Which is a good thing since I don’t think there’s much Andy can do to change it.

In fact back when I was reading about all this I made up a simple Monaco analogue. I made a game that mimics (although mimics poorly) the Situational Awareness challenge of playing Monaco. It has a fake fog-of-war that moves around as the players move and you have to keep track of many variables while ignoring changes in the fog-of-war.

You can swap the fog-of-war from being based on changes in brightness to being based on changes in hue with the space bar. Theoretically the version where the changes are in hue and not brightness should be easier. In practice the difference is subtle.

You can play it here: Morocco

You will need instruction as it took about an hour to write:

Goal: Get a high score. Your score is the Grey Number in the top left-hand corner.

Score By: keeping your mouse hovering over the player that is the same colour as the square in the top left-hand corner

Also Score By: the other players will ocasionally turn into squares. Keep track of them and Click on them when they do for a big point bonus.

Swap Between Brightness and Hue Fog Of War: with spacebar

Restart at the end of a round: with the “R” key

I am always impressed at how broad a base of knowledge is required to author video games. I think understanding of the human brain is one of the most rewarding. Learning about the Brain is also great for shining light on every-day life. I am constantly trying better to understand my own.