Table of Contents
Virtual Reality
So, for my Build 18 project this year, my partner Alex and I wanted to do something cool and awesome. I had a few ideas floating around, and I had seen Nirav Patel's post on the subject of virtual reality gaming with the PNI SpacePoint Fusion. Alex thought it was a good idea, and so we ran with it. As it turns out, PNI has a nice student sponsorship program if you get in touch with the right people, and they were willing to send me their USB version of the SpacePoint Fusion (a discontinued product?!) as long as I did a writeup about it! So, here goes 
- YouTube Link! https://www.youtube.com/watch?v=XrXH4Cp5O3I
Tutorial Notes
Components Needed
Nirav's original design only called for one SpacePoint Fusion and a nifty portable projector that went on top of his Wii Zapper. I personally wanted to see if we could get away with both head and gun inputs and couple that with a head-mounted display.
- SpacePoint Fusion by PNI Corporation: Apparently has best 1:1 orientation information out there! They recently discontinued the USB version, but you might be able to get an old one if you ask nicely enough.
- Plug-n-Play Unity Seagull Island Demo and Windows-Only Visualization Demo (great for testing!)
- Wii Remote (Wiimote)
- Don't skimp on quality! I bought a Chinese knockoff on eBay (that included a nunchunk and sheath for $18) that seemed to work fine on a Wii, but NEVER WORKED VIA BLUETOOTH ON A COMPUTER!. When it's only a buck or two more, just go for the real one! (this applies to more than just gaming products)
- I got one “used” on Amazon (for $12 plus shipping), and it looks/feels better than the “new” one that I got from China…
- Display (HMD, or projector)
- Video quality is not great on a budget (640×480 for ~$250). But, it's a ton cooler to have your screen stay with you as you move around
- I bought a Vuzix 920 as they seemed to have the best support.
Setting up the Software
Nirav used Ubuntu, but I took the long and frustrated route of seeing if it worked in OS X as well (see the FAQ's for more info). Long story short, it didn't work, and developing in Ubuntu ended up being a much faster process!
Compile Cube and LibHID
- Install Ubuntu and apt-get the dependency packages below.
#Allow all dependencies to be installed as well. Installs SDL & dependencies, OpenGL, Zlib, libpng sudo apt-get install g++ libusb-dev libsdl1.2-dev libsdl-sound1.2-dev libsdl-mixer1.2-dev libsdl-image1.2-dev libenet-dev
- Also, you need to install libhid from source (the Ubuntu package didn't work for me). Download from here
cd ~/Downloads tar .zxvf libhid-0.2.--.tar.gz cd libhid-0.2.-- ./configure make sudo make install
- From the Cube SourceForge page, download both the unix version and the source code.
- Copy the packages and data directories from the unix folder (cube) to the source code folder (cube_source).
- Modify the g++ flags in the Makefile in cube_source/src
#Add -fpermissive to the end CXXOPTFLAGS=-Wall -O3 -fsigned-char -fomit-frame-pointer -fpermissive
- At this point, everything should compile out of the box, and you can run the cube_client program from the cube_source directory (./src/cube_client) and get mouse and keyboard input.
Modifying Cube Source Code
- There are a good deal of modifications that you need to do to the Cube source code in order for it to work. (Y/P/R = yaw/pitch/roll)
- But, feel free to skip the details and just download what I did: Zip of my src directory, diff of new/old src
- clientgame.cpp: Added SpacePoint init and reading functions. The head is controlled by player1→y/p/r directly, and gun y/p/r is saved in handRelY/P/R. BE SURE TO CHANGE THE SERIAL NUMBERS (which can be found with some platform-specific usb device probing). Also, the player is made invincible around line 479
- cube.h: Added definitions for handRelY/P/R
- main.cpp: Disable mouse functionality
- protos.h: Added libhid function prototypes
- renderextras.cpp: Mapped the relative y/p/r of the gun to a crosshair x/y coordinate on the screen. Turns out it's a simple right triangle with distance d ~ 300. Tweak as needed.
- rendergl.cpp: Adjust on-screen gun orientation to match actual gun
- weapon.cpp: Enable infinite ammo
Set udev rules
On first start, I got a bunch of errors from libhid about not seeing the SpacePoint unless I ran the process using sudo. After reading Nirav's tutorial again, I saw his note on the end about setting the udev rules correctly.
On most modern Linux distros, you can fix this by setting a udev rule for the device. In Ubuntu Karmic Koala, saving the following as /etc/udev/rules.d/45-spacepoint.rules , running sudo service udev restart , and then unplugging and replugging in the device should fix it:
# PNI SpacePoint Fusion SYSFS{idVendor}=="20ff", SYSFS{idProduct}=="0100", MODE="0664", GROUP="admin"
Set up Wii Remote
All this does is map buttons on the Wii Remote to buttons on the keyboard
-
- Create a config file located in /etc/cwiid/wminput
- To run this file below:
sudo wminput -c fps_config2
- fps_config2
# Fps config for wminput by kyrlian # needs plugin nunchuk_kb #File is located #custom buttons Wiimote.A = KEY_ENTER Wiimote.B = BTN_LEFT Wiimote.Up = KEY_UP Wiimote.Down = KEY_DOWN Wiimote.Left = KEY_LEFT Wiimote.Right = KEY_RIGHT Wiimote.Minus = KEY_PAGEDOWN Wiimote.Plus = KEY_PAGEUP Wiimote.Home = KEY_ESC Wiimote.1 = KEY_TAB Wiimote.2 = KEY_ESC Nunchuk.C = KEY_SEMICOLON Nunchuk.Z = KEY_SPACE #plugin for nunchuk stick Plugin.nunchuk_kb.Up = KEY_UP Plugin.nunchuk_kb.Down = KEY_DOWN Plugin.nunchuk_kb.Left = KEY_LEFT Plugin.nunchuk_kb.Right = KEY_RIGHT
FAQ's
- Can't you get this working with more modern games?
- Your game needs to have direct inputs for yaw, pitch, and ideally roll for both the camera and the gun in order to get this working right! Trust me, mapping YPR to relative knob/mouse/WASD inputs is not the way you want to go. I suspect that most games have this sort of functionality available, it's just not as well documented because people don't want to mod the camera view that often.
- Strafing using a knob is so 20th century! Why can't I walk around my room and have the SpacePoint figure out where I've moved?
- Oh, it'll happen someday. In the meantime, a good buzzword is “dead reckoning” Wikipedia, robotics tutorial and “Kalman filters”, both of which are very difficult mathematical problems. In this case, the SpacePoint Fusion allows you to acquire the raw accelerometer, gyroscope, and magnemometer data for doing your own tinkering, but their Kalman filters for deriving orientation data is likely very patented or kept a trade secret…and that's just for orientation! Deriving relative motion is much more complex using just motion data, which is why a lot of robotics platforms use encoders and they still aren't that accurate. :)
- Won't this work on multiple platforms?
- Yes, but library support varies. Libraries in Linux tend to be the most simplest to interface with and often free to use. I couldn't get libhid to compile on OS X in the limited time that I had, but it'd probably work if I tried hard enough. You could also make this work using libusb, but you'll have to write your own code for that. Also, Wiimote plugins for OS X were feature-limited (>.<) and never seemed to let me tweak it as I wanted to. In Ubuntu it's as simple as a text file with CWiid. If you do want to port this to other platforms, it can be done, and I wish you the best of luck! Write me a comment on the blog post and let me know how it went!
Extensions
- Finally found a tutorial for using this with Source (Valve's engine for games like TF2, Portal, etc)! Head Tracking
- Here's another guy that developed his own thing Torben Sko Paintball. Uses just a webcam…
Future Ideas
- YouTube Link! https://www.youtube.com/watch?v=eg8Bh5iI2WY
- YouTube Link! https://www.youtube.com/watch?v=wJItdmumxYY (Virtual Reality First Person Shooter)…a fake??
- PTAM Developer Page w/ source code for PTAMM!
AR Markers
- Worked on this for Build18 2012, but struggled to get all of the marker stuff compiled. Here's one that uses OpenCV 2.1+ exclusively: http://www.uco.es/investiga/grupos/ava/node/26