The resurgence of virtual reality is still in its infancy, and while we do have very good VR headsets such as the Oculus Rift and PlayStation VR, in other areas the tech is still severely lacking. Take interaction, for example, it's very difficult to convey touching something in the virtual world with physical feedback. But what if you could interact with real world objects that appear in the virtual world?
Disney Research decided to carry out just such an experiment by asking the question: can you catch a real ball in virtual reality? The good news is, yes you can, but there's a number of prerequisites to achieving such a simple task.
For us humans, ball catching relies on good hand-eye coordination, which means being able to track the motion of the ball as it flies through the air. The main issue with allowing good hand-eye coordination in VR is the inherent latency in the system. Too much latency means the virtual representation of the ball the user sees is not actually where the ball is, which makes it much more difficult to catch.
To solve that problem, Disney Research setup a very low latency ball tracking system. It relies on an OptiTrack Flex 13 motion capture system, which tracks the ball at 120 frames per second with lag of just 8.33ms. The person catching the ball was wearing an Oculus Rift and the VR environment was created using the Unity 3D game engine running on a 64-bit Windows 10 PC powered by an Intel 2.5GHz Xeon E5-2680 processor, 32GB RAM, and a Nvidia GeForce GTX 970 graphics card. What the Oculus wearer sees is being rendered at 140 fps.
In order for the virtual ball to be represented as accurately as possible in terms of position as it traveled, Disney needed to track and predict its position and velocity. They did this by using an Unscented Kalman Filter algorithm with the data produced from the motion capture camera.
Related
- Koei Tecmo Creates 5 Sense Virtual Reality MachineKoei Tecmo Creates 5 Sense Virtual Reality Machine
As the video above shows, it was possible to predict how the ball would travel through the air once thrown and therefore allows for a very accurate representation to the catcher. So good was the system, even with the virtual ball not rendered the catcher could rely on the predicted catch point to make a successful catch.
As the Disney Research paper explains (PDF), the system was tested using 140 tosses with 132 balls being caught giving an overall accuracy of 95%. It proved to Disney that even with small latencies involved that aren't experienced in real life vision, accuracy remains high. That bodes well for a future where real and virtual worlds become more fully integrated, but there's still a lot of work to do to get there.