How To Mixed Reality

How to Stream Fantastic Contraption in Mixed Reality

We stream every Thursday at noon on Twitch and are really enjoying it. This is a great way to show people what it feels like to be in VR and is a unique way to stream games. Our techniques and equipment can definitely be improved on to acheive better quality.

There are a number of ways to stream Fantastic Contraption, initially this guide will only cover full roomscale greenscreen streaming. You can probably devise other clever ways of streaming the game with what is covered in this guide.

I’ve also written a guide on how to record Fantastic Contraption with a hand-held camera.

There are a few broad sections to this guide:

  1. Studio setup
  2. Game setup
  3. OBS setup
  4. Camera Sync
  5. The Director Functions

 

1. Studio Setup

Lets start with the hard part. Our studio has a footprint of 15×12 feet and our greenscreen is 81/2 feet high. This is basically our entire living room. This is a nice big space and gives us room to include a couch in the shot for guests. I think 8×12 feet would probably still be comfortable with a tighter shot and no couch. If you have less space you can always get creative.

400sqft of green muslin blankets our livingroom

The green screen setup isn’t that hard. You just need a bunch of green screen fabric and a couple big racks to hold them. You’ll also need some lights, a camera and a mic. Sarah made up a handy shopping list of all the stuff we bought.

-> Sarah’s Shopping List <-

All told it cost is 1000$-1200$ CAD. It will cost you less in the US.

A quick note on webcams: a webcam that is great at a meter will be terrible at 4 meters. They are designed to take in a person sitting at a desk and are terrible at filming a whole room. We spent 300$CAD and bought the cheapest Sony handycam that does 1080p 60fps HDMI out and a 200$ capture card which much improved the quality. Of course you can always spend more on cameras. (we also tried a dslr but they are extremely finicky to get streaming well). The camera and card are listed in the shopping list.

So hang your green screen, set up your mic, and plug in your camera.

 

2. Game Setup

Game setup is pretty easy:

  • Start the game holding shift while opening it. Make sure it’s opening full screen at the resolution of your  monitor.
  • Open the “Settings” menu (you might have to hit the eye button to open the left-menu if it’s closed
  • Turn on “Toggle Depth Camera”
  • Make sure “Foreground Colour Key” is set to: (0.0, 0.0, 0.0)gameSettings
  • Close the settings menu

3. OBS setup

OBS (Open Broadcasting Software) is popular streaming software and is pleasently flexible. Weirdly OBS makes it kind of hard to export settings files so I’m going to walk you through setting it up for Fantastic Contraption. It’s good to know how this works anyway so you can make any changes you might decide you want to make. First download OBS if you don’t have it already (download the Studio version). Then open OBS.

Note that since the game is broken into four quadrants your output resolution will be half of whatever your screen resolution is. I’m going to do this totorial with a 4k monitor which will let me stream at 1080 but you don’t need a 4k screen to stream the game. Just adjust the numbers to whatever resolution your monitor runs at.

First let’s add the game background:

  • Hit the “+” under “Sources” and select “Window Capture”. Name the layer “Background” selectWindowCapture
  • Select “Fantastic Contraption” in the “Window” dropdown
  • Flip off “Capture Cursor”
  • Hit “ok”
  • Right click on “Background” and select “Filters”
  • Hit the “+” on the Filters window and select “Crop”
  • Set the “Right” to “1920” (or half of whatever your monitor width is)
  • Set the “Bottom” to “1080” (or half of whatever your monitor height is)cropValues
  • Hit close
  • Grab the little red circle at the bottom right and resize the layer to fill the whole canvas area

 

Now let’s add the webcam

  • Hit the little “+” under “Sources” again and select “Video Capture Device”. Name the layer “RL Camera”
  • Under “Device” select your camera
  • Hit “ok”
  • Resize the camera layer to fill the whole canvas area
  • Right click on “RL Camera” and select “Filters”
  • Hit the “+” under “Effect Filters” and select “Color Key”. Name it “Green Screen”
  • Fiddle with “Similarity” and “Smoothness” until you get the desired effectcolourKey
  • Hit close

 

Lastly, let’s add the game foreground:

  • Hit the “+” under “Sources” and select “Game Capture” name the layer “Foreground”
  • Unslect “Capture Any Foreground Application”
  • Select “Fantastic Contraption” in the “Window” dropdown.
  • Check “Allow Transparency”
  • Flip off “Capture Cursor”foregroundOptions
  • Hit “ok”
  • Right click on “Foreground” and select “Filters”
  • Hit the “+” on the Filters window and select “Crop”
  • Set the “Left” to “1920” (or half of whatever your monitor width is)
  • Set the “Bottom” to “1080” (or half of whatever your monitor height is)
  • Hit close
  • Grab the little red circle at the bottom right and resize the layer to fill the whole canvas area

OBS setup Done!Done

 

4. Camera Sync

  • The last thing you have to do is sync the in-game camera with the real world camera. This means moving the in-game camera to where the real-world camera is and setting the in-game camera Field of View to match the real-world camera. This is all done very manually.
  • Make sure the game and OBS are both open and the game has the Depth Cameras turned on
  • Now open the OBS file menu and select “Always On Top”
  • Give the game focus and use the wasd keys (as well as q & e for up & down) to fly the camera around. Use the arrow keys to change the angle the camera looks.
  • This will be much easier with another person to help you. To get a rough idea of where the camera should be have the other person go into VR. They will see an insect with a big eye and wings, that’s your camera. Have them direct you, move the in-game camera to where the real-world camera is.

    Using hand controller positions to align the cameras
    Using hand controller positions to align the cameras
  • Then have them pick up the two controllers and walk to the back of the play area. Get them to hold out their arms in a T-pose and then not move. Using the arrow keys change the angle of the camera until the controllers are as close to matching the real-world controllers as possible.  Note that the FOV is still off so you won’t get a good match.
  • To match the FOV make the OBS window small enough that you can use the game’s settings menu again. Open the settings menu
  • Drag the “3rd person FOV” slider until the in-game controllers are the same distance apart as your friends (very tired) arms.
  • Make a few minor adjustments and then never touch your camera again. These settings will be saved so as long as you never touch your camera again you’ll never have to do this again. In practice we do adjust it every time we stream because things get jostled.

 

5. The Director Functions

We have some keyboard shortcuts that are fun to use while streaming. They let a person who isn’t streaming have some directorial control over the cameras.

A good hotkey to know about up-front is ~c. ~c will hide the hand controllers. There is always some lag in the system and hiding the hand controllers will make the lag less obvious.

~1, ~2, ~3 are the three different cameras.

  • ~1 is the POV of the player
  • ~2 is a 3rd person camera you can fly around to get a better view of the levels and the contraptions. Fly it around using wasdqe and arrow keys.
  • ~3 is the mixed-reality camera

If you are affraid you’re going to start flying the mixed reality camera around by accident you can select it and then hit ~L to lock it in place.

You can change the avatar of the player (visible in 3rd person camera) with ~a. It will cycle between all available avatars (my favorite is the pink unicorn!)

 

Happy Streaming!

Lindsay builds while couchies Gord and Colin advise

Share/Bookmark

7 thoughts on “How To Mixed Reality

  1. Just fair warning- I love what you guys are doing so much if I ever meet you I’m going to kiss both of you right on the lips.

  2. Thank you so much for writing up this thorough rundown on your streaming setup and method for Fantastic Contraption! Y’all are doing some amazing work, and it’s so rad that you’re willing to give so freely to the community! :-D

  3. Your VR camera compositions are amazing and really forward thinking. Do you foresee releasing a tutorial or project file on how to create a moving camera matched to a virtual camera?

  4. My vive will be here in 2 days, Im putting together my mixed reality order based on your how-to list. Are there any updates / changes that you think would work better for those of us who will be ordering the green screen cloth, a camera (will go with your suggestion of the amazon hdmi camera) and capture card?

    Thanks for the write up, it sure makes this a ton easier and much more fun to do.

  5. Thank you for this article! It was really really helpful for me. Can I translate the article in Korean for sharing with my friends?

Leave a Reply

Your email address will not be published. Required fields are marked *