• How To Mixed Reality

    How to Stream Fantastic Contraption in Mixed Reality

    We stream every Thursday at noon on Twitch and are really enjoying it. This is a great way to show people what it feels like to be in VR and is a unique way to stream games. Our techniques and equipment can definitely be improved on to acheive better quality.

    There are a number of ways to stream Fantastic Contraption, initially this guide will only cover full roomscale greenscreen streaming. You can probably devise other clever ways of streaming the game with what is covered in this guide.

    I’ve also written a guide on how to record Fantastic Contraption with a hand-held camera.

    There are a few broad sections to this guide:

    1. Studio setup
    2. Game setup
    3. Camera Sync
    4. The Director Functions

     

    1. Studio Setup

    Lets start with the hard part. Our studio has a footprint of 15×12 feet and our greenscreen is 81/2 feet high. This is basically our entire living room. This is a nice big space and gives us room to include a couch in the shot for guests. I think 8×12 feet would probably still be comfortable with a tighter shot and no couch. If you have less space you can always get creative.

    400sqft of green muslin blankets our livingroom

    The green screen setup isn’t that hard. You just need a bunch of green screen fabric and a couple big racks to hold them. You’ll also need some lights, a camera and a mic. Sarah made up a handy shopping list of all the stuff we bought.

    -> Sarah’s Shopping List <-

    All told it cost is 1000$-1200$ CAD. It will cost you less in the US.

    A quick note on webcams: a webcam that is great at a meter will be terrible at 4 meters. They are designed to take in a person sitting at a desk and are terrible at filming a whole room. We spent 300$CAD and bought the cheapest Sony handycam that does 1080p 60fps HDMI out and a 200$ capture card which much improved the quality. Of course you can always spend more on cameras. (we also tried a dslr but they are extremely finicky to get streaming well). The camera and card are listed in the shopping list.

    So hang your green screen, set up your mic, and plug in your camera.

     

    2. Game Setup

    Note that there are two ways to do game setup. If you’re using an HDMI Camera or want to do the compositing in post go here. If you’re using a webcam read on.

    • Start the game holding shift while opening the game in steam. This will open the resolution options. Set the game to open full screen at the resolution of your  monitor.
    • Open the “Settings” menu (you might also have to hit the eye button to open the left-menu if it’s closed)
    • Turn on “Enable director controls”
    • Under the “Select Webcam” dropdown select the webcam you want to use. If it doesn’t appear or doesn’t work use these instructions
    • Fiddle with the “Key Threshold” and “Key Step Range” sliders until your green screen looks good (also set the chroma key colour if your backdrop is a colour other than green)

    3. Camera Sync

    • The last thing you have to do is sync the in-game camera with the real world camera. This means moving the in-game camera to where the real-world camera is and setting the in-game camera Field of View to match the real-world camera. This is all done very manually.
    • Put on the HMD and grab the in-game camerabug – that’s flying eyeball which indicates where the virtual camera is. Peek out from the HMD so you can see the real-life camera at the same time, and drag the camerabug over so that they are in the same place.
    • Next use the keyboard wasd keys (as well as q & e for up & down) to fly the camera around and make small adjustments. Use the arrow keys to change the angle the camera looks. Shift arrow will “roll” the camera.
    • This is easier with another person to help. Have them pick up the two controllers and walk to the back of the play area. Get them to hold out their arms in a T-pose and then not move. Using the arrow keys change the angle of the camera until the controllers are as close to matching the real-world controllers as possible.  Note that the FOV may still be off so you won’t get a good match.

      Using hand controller positions to align the cameras
      Using hand controller positions to align the cameras
    • To match the FOV, open the settings menu and drag the “3rd person FOV” slider until the in-game controllers are the same distance apart as your friends (very tired) arms.
    • After setting the position and FOV set the “Camera Delay” slider. You will notice the camera lags behind the game as the player moves around. This slider will delay the game so it can match the delay of the camera. Drag this slider until the game and the player are in sync.
    • You are done setup! Because of lighthouse drift and camera jostling you will probably have to re-tune this every time you stream.

     

    4. The Director Functions

    There are a number of camera controlls that can be used by the person in-game or by a spectator using a keyboard.

    When you turn on Mixed Reality mode you will see these camera buttons somewhere in the game.

    You can select between different cameras by pointing at a button and pulling the trigger. A person outside VR with a keyboard can switch between cameras by pressing ~1, ~2… ~6

    When you’re in the game each camera is represented by a little flying insect. You can grab the insect to move it around. A person with a keyboard can fly around to get a better view of the levels and the contraptions using wasdqe and arrow keys.

    The cameras are each a little different:

    • ~1 is the POV of the player
    • ~2 is a 3rd person camera
    • ~3 is the mixed-reality camera
    • ~4 is another 3rd person camera
    • ~5 is a camera that always looks at the player
    • ~6 is a camera that always looks at the goal ball

    We have some keyboard shortcuts that are fun to use while streaming:

    ~C will hide the hand controllers. If you are affraid you’re going to start flying the mixed reality camera around by accident you can select it and then hit ~L to lock it in place.

    You can change the avatar of the player (visible in 3rd person camera) with ~A. It will cycle between all available avatars (my favorite is the pink unicorn!)

    You can give the player a hat with ~F!

     

    Happy Streaming!

    Lindsay builds while couchies Gord and Colin advise

  • Contraption coming to Oculus Touch!

    Huh? What hand controllers are these…?

    That’s right! We’re working to bring Fantastic Contraption to the Oculus with Oculus Touch controllers later this year. You may have also seen our recent teaser from Andy Moore of seated-scale Fantastic Contraption. We have plans to support seated-scale, standing-scale, room-scale and my favorite, sitting-on-the-floor scale on both the Oculus and Vive (after launch).

    Concept art of Oculus desk-scale. You are the walrus.
    Concept art of Oculus desk-scale. You are the walrus.

    We just had a blast showing Fantastic Contraption at the Oculus Game Day event at the Game Developer Conference. And for the rest of the week we’ll be showing the game at the Independent Games Festival where we’ve been nominated for the prestigious Nuovo award. They announce the winners tonight. Come visit us at booth N2618 in the North Hall expo and wish us luck!

  • Contraption launching April 5

    vancouver_twitchIt’s official: the HTC Vive will be shipping on April 5th, and it will come with Fantastic Contraption, Job Simulator, and Tilt Brush. My 3 favorite things in VR right now!

    I’m so stoked – it’s like being Super Mario Bros and/or Duck Hunt at the launch of the NES. The Vive could be the platform that changes everything… we just have to get enough people into it, because playing is believing. We showed Fantastic Contraption at a Vancouver Twitch event last week and people kept coming away stunned and stoked, trying to figure out where they’re going to fit a Vive in their apartments.

    And yes you can now preorder the Vive for $800 USD, which is the adult equivalent of the $150 my sister and I saved to buy a NES back in the 80s. You also need a gaming PC – Nvidia 970 or higher – but the prices on these and on all VR hardware will come down pretty fast in the next couple years. So get out your measuring tape and start sizing up the living room for that 1.5 x 2m (5 x 6.5ft) minimum. Do you really need that coffee table? I didn’t think so.

  • Contraption Accolades

    Fantastic Contraption has been nominated for the prestigious IGF Nuovo award, and for Best VR Experience at the Vision VR/AR Awards (read more at our Road to IGF interview). The team is thrilled and we’ll be showing the game at both the IGF expo at the Game Developers Conference in March, and at the Vision summit next month in LA. Come visit us if you’re there!

    IGF_finalist._350Vision_Summit_Awards_artwork_Finalist_150

  • Mixed-reality VR Twitch streaming

    It’s as cool as it sounds. We’ve started live streaming Fantastic Contraption on our Twitch stream every Thursday at noon PST. Here’s this past week’s stream:

    Twitch isn’t just for e-sports and speedruns anymore; it’s getting downright mainstream as a marketing tool, a way for people to check games out before buying them, participate in events, and to obsess over games while at work / any moment they can’t be playing them (guilty!). But for months we’ve been asking: how the hell do you stream virtual reality games? Especially room-scale VR using the HTC Vive?

    The standard picture-in-picture game footage + webcam technique doesn’t do VR justice. The first-person in-game feed from VR games gives at best a cropped, distorted view of what the player is actually seeing, and talking heads wearing VR headsets are even duller than regular talking heads. After an hour-long session with Youtuber Northernlion, we did some brainstorming.

    Then we geared up:

    400sqft of green muslin blankets our livingroom
    400sqft of green muslin + 2 layers of black, a 2000 watt light kit, Logitech C930e webcam and Blue Yeti mic, total: $1000 cad

    Our livingroom has huge windows on two sides, so it was a challenge to keep the green screen lighting consistent (bedsheets and cardboard were involved). But we discovered that our webcam feed has considerably less lag during the day when all that natural light lowers exposure time.

    Syncing: we move the in-game camera to line up real and virtual hand positions
    Our first trials used OBS to combine three views. We stuck a webcam on a tripod and synced it’s position with two in-game 3rd-person cameras. One only saw foreground objects, and the other only saw the sky, ground, and objects behind the headset. We first tried using a clipping pane, then tried blipping game objects between two visibility layers.

    We output the in-game cameras side-by-side then smushed the 3 feeds together in OBS:

    streaming_obs_600

    It’s not half bad without the green screens too, if you overlay the background camera at 50% transparency.

    But for our next stream we’re going to try piping the live webcam feed into Fantastic Contraption, so we can display it on a moving plane in the game. This should give us fewer blipping glitches and a higher output resolution. Thanks to Edwon for the suggestion and help!

    Lindsay builds, Colin and Gord advise
    We’ve got some in-game tools to use while streaming, like a floating Twitch comments feed that only the player can see, and director controls that let our “couchies” swap the view between various game cameras.

    We’ll keep things fresh by bringing on special guests, and will be reaching out to local Vancouver Twitch streamers to come stream from our rad green screen studio (aka our livingroom). Stay tuned, Thursdays at noon PST!