If you had to list the most mind-blowing tech demos in recent memory, Microsoft's Hololens AR headset would need to be included, as would its projector-enhanced Illumiroom. A company called Théoriz from Lyon, France has married both of those things to create a "mixed reality room" that uses projector tech, motion tracking and augmented reality together. Its latest technology demo video made it seem like we're closer to Star Trek's Holodeck than ever before, so we went to take a closer look.
Théoriz is located at the "Pole Pixel," a sprawling collection of studios east of Lyon used by Panavision and other cinema companies. The company's mission is as much artistic as tech-oriented, so the engineers are both bohemian and code-savvy. "We are a team mostly composed of creative engineers," says Théoriz co-founder David-Alexandre Chanel. "Engineers who have an artistic sensibility and also do good code."
To wit, the company has created some very technical and very whimsical projects, including an art installation called "Doors" featuring portals that open up to an infinite space and change perspective as the viewer moves, and "Are You my Friend," an industrial robot that communicates with the exhibit-goers via a keyboard.
Art aside, the mixed reality room tech is impressive. The team tracks the camera (typically a RED model that can record and output in real time) with an HTC Vive Tracker, and feeds the data to a computer running the Unity game engine. That generates digital environments like flying space skulls, a Minecraft-like room with holes that open up on the floor and geometric shapes that interact with actors to form stairs, wells or small hills.
The computer syncs everything together, so that when the camera operator pans or tilts, the Unity scenes tilt or pan to match. Those are then beamed into the room via six projectors -- four for the floors, and two on the walls. At the same time, three Kinect-style 3D cameras, combined with Théoriz's in-house "Augmenta" system, detect the position of the actors so they can interact with the environment. Everything must be processed and played back in real-time by the Unity based system, something that required some clever coding and computing horsepower.
In the resulting videos, live actors interact seamlessly with virtual environments, creating a hallucinogenic effect. "It's called mixed reality because we use and merge things from the virtual world with reality," says Chanel.
For instance, dancers can make the walls "move" with their movements and bat away flying asteroids. In the latest demo video (above), actors interact with bizarre geometric environments, opening up holes in the floor where they move and walking up fake stairs.
Though most of the tech is off the shelf, none of it is intended for consumers -- at least, not yet. For now, the company wants to just sell its services for things like music videos, dance performances, art installations and other live events. At the same time, they're improving the tech to make it more realistic and immersive. "We think that by changing the content creation process, we can open new creative possibilities and achieve unprecedented kind[s] of visuals," says Chanel.
The next project will test everything Théoriz has learned so far, both artistically and technically. "We're trying for the first time to show an artistic video with two dancers," Chanel says. "And they're going to dance and interact in the virtual world, moving through different kinds of totally surreal scenes."
Eventually, Théoriz might make its software available to other companies, but for now it's just trying to make its services more compelling for artists and audiences. "It's a new field," says Chanel. "We still have to evangelize it and create demand so it can eventually find its place." And the best way to do that? "Seduce the audience with something new, poetic and unexpected," he says.