According to Microsoft researchers, this is the first solution that allows you to capture anything in three dimensions and send it over the internet so anyone can see it from any point of view, just like the holograms in Star Wars. Now we only need a projection system that doesn’t require us to use the mouse—although you will be able to use it with something like the HoloLens or the Oculus Rift.
From the research abstract: “Our system records performances using a dense set of RGB and IR video cameras, generates dynamic textured surfaces, and compresses these to a streamable 3D video format. Four technical advances contribute to high fidelity and robustness: multimodal multi-view stereo fusing RGB, IR, and silhouette information; adaptive meshing guided by automatic detection of perceptually salient areas; mesh tracking to create temporally coherent subsequences; and encoding of tracked textured meshes as an MPEG video stream. Quantitative experiments demonstrate geometric accuracy, texture fidelity, and encoding efficiency. We release several datasets with calibrated inputs and processed results to foster future research.”