User Studies

This is from a series of posts I wrote about Fantastic Contraption when I originally released it. They were originally published on our travelogue but I have back dated them and moved them over here where they fit in more.

More in the series of articles about writing the physics puzzle game Fantastic Contraption.

Today: usability. Or specifically: teaching people how to play your game.

Some of is going to make specific references to the game so you should go try out the game first. Be cool, it’s free, you can be your own little user-study.

Fantastic Contraption launched with a fairly simple tutorial system (as of the time of this writing is still has it). I basically just came up with it out of thin air. I guessed what people would need to know, what they would be confused by, and what they would figure out on their own. So I tried the make-it-up-as-you-go-along school of thought.

I took a usability class in university from one of my favorite profs Dr. Tzanetakis. And his mantra was ‘user testing’. The idea was that all the theory in the world falls down and you won’t know what is working and what isn’t without putting a user in front of your UI.

So I finally did some belated user testing (this is after the StumbleUpon traffic bomb). And I made some interesting discoveries.

For people who think like me the existing tutorial is great. Gamers tend to get it. It takes a little time to figure out the draw when stopped, hit go and you can’t touch it thing. And they try to drag and drop instead of draw. But mostly they clip through the tutorials and rock straight on through to Up The Stairs.

But I have left non-gamers in the dust. In the dust, choking on it, wondering why I hate them so much.

Just some of the things non-gamers are not taught well enough by the existing tutorial:

-draw not drag and drop (this is universal)
-have to draw inside the blue square
-you have to get the pink thing to the pink goal to win (despite pounding them over the head with it)
-you can’t draw when the simulation is running (the red reminder arrow goes completely unnoticed)
-your contraption can’t overlap terrain
-the basic principles of construction

I’m going to write a bit more on that last one: The basic principles of construction.

There is one tower of UI failure that towers mightily over all others. It may be the only thing that you need to truly understand in order to play Fantastic Contraption:

You must attach the rod to the centre of the wheel to make it spin. That little bit that is equidistant from all the other bits. The focus, the key, that of overriding importance. The Middle.

And I am a total fail on that front. I have one concession to teaching this truism in the current tutorial. There is an example contraption in the 2nd level that is a clockwise and a non clockwise wheel connected with a stick between their centers. But people don’t even get that this is an example contraption. And sometimes they do but then they start trying to replace the goal wheel with a non-spinning wheel.

Most often people put a rod between the two closest points of the wheels. Unfortunately this pretty much ends their gaming experience. They have now totally broken the contraption. They will have to find the delete tool if they are ever going to finish the level. So the thing non-gamers do most often also ensures that they will never play the game again. That is what UI experts call “bad”.

And oddly enough I never even considered it. It wasn’t before I did some good old fashioned user testing that I discovered I was turning away a huge percentage of my traffic angry and sullen, probably feeling stupid.

So now it’s time for quest redesign-the-tutorial. The tutorial is going to get very constraining and guided. It’s going to be alot of work, and it’s going to annoy the gamers (hopefully skip buttons will keep them happy). But something needs to change pretty drastically.

So there you go. More evidence that Dr. Tzanetakis’ mantra is completely correct.

Do User Testing. You don’t know what’s going on until you Do Some User Testing.

real quick: The user testing methodolgy I’m using is getting a co-worker, friend, neighbor, or friend of a friend to play the game. Then I reasure them that they aren’t an idiot and any failure is a failure on my part not theirs (this is important so they don’t get all nervous and freak out when things start going wrong). Then I just watch them, and don’t answer any questions. Let them fail and recover on their own. You won’t be there every time someone tries to play your game. I very occasionally prod them about what they are thinking. I’d do voiced-thought style studies but already have trouble getting people to participate.