This is the video I transcripted, and below, the text ! Have fun !
Hi. My name is Johnny Lee, and in this video, I’m gonna show how to perform head tracking, and create desktop virtual reality displays, using the Nintendo Wii remote. Now first what do I mean by desktop VR display? Well, if you think about most computer screens, they typically use the display of flat image, a little bit like this picture in this picture frame. Even if the picture is of something in 3D like a video game, the picture is still flat so it doesn’t change, depending on what angle you view the screen at. A desktop VR display, however, is a little bit like taking the picture out of the picture frame, and then just having the frame. Now the scene actually changes depending of what angle I view the screen at. So this essentially becomes a portal or a little window into another room. Now to do this, the computer needs to know the location of your head, relative to the screen, and this is called “head tracking”.
Now to perform head tracking, we’re gonna be using the Wii remote and the sensor bar, but we’re actually going to be using them backwards. We are gonna put the Wii remote, next to the TV and actually move the sensor bar instead. The Wii remote, actually contains an infrared camera, and the sensor bar is simply two sources of infrared light.
When the camera sees the two dots of light, it’s gonna give an approximate location of my head, horizontally, vertically and in distance from the screen.
Ok, the tricky part is now we’re going to have to find some ways to mount the sensor bar onto our head. One common trick is to get a baseball cap, and then mount the hardware to the cap and this is definitely going to work but it’s a little bit goofy. So instead, some hardware stores sell these safety glasses, with LEDs built in on the side meant to be used as headlamps. Now if you replace the LEDs with infrared ones, you essentially get your headmounted sensor bar, in a nice sporty safety goggle form factor. Once we have created our head mounted sensor bar, and have connected my Wii remote to my pc, we are ready to do some head tracking Behind me, is a demo program of a 3D room, with some targets floating in it. Now because the effect only works to the person wearing the sensor bar, I’m gonna have to show you the effects, through a moving camera.
Now to do this, I’m literally just going to hold the sensor bar at the base of the camera and move it around. Just a quick note, to power the sensor bar, I simply turn on my Wii, after I’ve connected my Wii remote to my pc.
First, I’m gonna show you what it looks like without head tracking, which is what displays normally look like. You can see that although it’s a picture of 3D room. The image looks very two dimensional and bound to the surface to the TV.
Now with head tracking turned on, the TV actually looks like the entrance to a real room, just like in real life, by moving our head around we can look behind objects. If you look really closely, some targets actually appear to be floating up in front of the screen, reaching into the real world. If we get closer to the screen, we get closer to the objects, and we can even get behind the ones, floating in front of the screen.
As I pull the camera back, keep an eye on the front most target. Head tracking provides the illusion that the target is actually floating directly above the laptop screen, far in front of the TV. Now using this picture of a football stadium, if you move right, you can see more the field, if you move left, you can see more the stands, and if you get closer to the screen, you see more of everything just like a real window.
If I use my IR glasses and keep the sensor bar on the TV, I can use a second Wii remote to point and shoot like any Wii game, while also doing head tracking, so now ducking and shifting your body is actually meaningful to a game. You can also see now, how the perspective is incorrect if you are not the one wearing glasses.
So head tracking for VR displays is only gonna work for one person at the time, but for that one person, the 3D experience is gonna be far more realistic and immersive, than anything else we see in homes today. So if you are watching this and you are a Nintendo Wii game developer, I want to see some games. Anyway, as usual, you can visit my website to download this software and find out more information about my other Wii remote projects
Thanks for watching.
