Move Over 360-Degree Video, Lytro's Light Field Video Is (Almost) Here

Lytro revealed the first 360-degree light field video recorded with its Lytro Immerge light field camera technology. The days of standard 360-degree video might be numbered before the technology even got a foothold in the market.

The virtual reality industry is going to go through many rapid changes in the coming years. We’re in the very early days, and innovation is still rampant. Trends that are getting a lot of buzz today are bound to be yesterday’s news quicker than you might imagine. A great example of that is 360-degree video.

Right now, 360-degree videos are all the rage. Household-name companies such as Google, Facebook, GoPro, and Kodak (and more) have embraced this immersive video technology along with several lesser known companies looking to break into the immersive media technology, such as Human Eyes and Lucid VR. There are even companies, such as IM360 and Jaunt, that are betting that 360-degree video will catch on with professional video production professionals.

There’s a problem with the approach that all of these companies are taking, though. They are all taking an existing medium (cameras) and adapting them to work with a brand new medium (VR). This is a stop-gap solution at best. For true immersion VR, you have to be able to move around within the space you're experiencing. A 360-degree image or video is novel, but it's but a mere glimpse of an experience and isn’t going to convince anyone that they are truly living that moment.

There is at least one company that is attempting to tackle that problem with a technology worthy of moving forward with the virtual reality revolution. Lytro is developing a special camera that records light field information, as well as a backend system to handle the processing of that data. We first caught wind of Lytro nearly a year ago, when the company introduced its Lytro Immerge camera technology. Lytro had said that prototypes of its cameras were to be available in Q1 of this year.

We didn’t ever hear about the prototypes going out, but they must have sent cameras to at least one company, because the first film project recorded with Lytro Immerge Light field cameras has been announced.

The project goes by the name “Moon,” and Lytro claimed that it's the first ever live-action six degrees of freedom (6DoF) VR experience ever created. Lytro said that “Moon” allows a seated viewer to lean in, lean back, or move side to side, and your view will react as if it were a real-life experience. It gives you the ability to look around objects without distorting their appearance (parallax). The light field video also gives you the ability to tilt your head. Traditional 360-degree video only lets you look around, but if you tilt your head to the side, so will the video. Light fields provide all the information needed to process the scene at all angles.

Light field technology doesn’t really record video. It records real-world light ray trajectories and then calculates what the scene would look like based on your viewpoint. Lytro then combines that data with live-action video and with 3D renderings to reproduce true to life imagery. Light field technology also has the added benefit of being able to reproduce shiny and mirror-like objects that are difficult or impossible with 3D renderings. There’s also no need to stitch video clips together when the light field is captured, which is currently a pain point with 360-degree video production.

Lytro didn’t announce when the “Moon” video experience would be available to view. Tim Trillion, the company’s VR of Engineering, said that “much more about the production of 'Moon' and Light Field technology that powers it” will be revealed in the coming weeks. In the meantime, there’s a preview clip to get an idea of what to expect.

 Kevin Carbotte is a contributing writer for Tom's Hardware who primarily covers VR and AR hardware. He has been writing for us for more than four years. 

  • SockPuppet
    Amazing. This is what VR needs to become mainstream. VR movies like this are the thing to get HMDs in the homes of "normal" people.
    Reply
  • bit_user
    This is it.

    I saw it on their website, a few months ago, and it was clear to me that it's the logical conclusion of what 360-degree cameras want to be.
    Reply
  • Geekwad
    I've been watching Lytro for some time, and only hope that this coupled with the eventual IMAX experience on a StarVR (and someday an equivalent home experience:)) is where consumer VR continues to high-step towards.
    Reply
  • Brian_R170
    I'm guessing Tim Trillion is the VP of Engineering, not the VR of engineering?
    Reply
  • WFang
    Great... Now if this doesn't get the tin-foil hat "The moon landing is not real" people going, then nothing ever will.. I'm sure they will use this as proof that it was all fabricated and this technology has existed all along in secret government labs.

    Why could they not have used a different subject matter? A mountain top or something.
    Reply
  • jaber2
    Tim Trillion is that a real name? was Billion too low?
    Reply
  • wifiburger
    just like 3d displays this will fade into obscurity and nobody will care or write about it, pass !
    Reply
  • bit_user
    18525680 said:
    Great... Now if this doesn't get the tin-foil hat "The moon landing is not real" people going, then nothing ever will.. I'm sure they will use this as proof that it was all fabricated and this technology has existed all along in secret government labs.

    Why could they not have used a different subject matter? A mountain top or something.
    I had the exact same thought. Cool tech, dumb demo.

    But, the reality is that people who want to believe the Apollo moon landings were faked will do so, regardless of any evidence to the contrary. Similarly, feeding them a little more "evidence" in support of this idea won't make much difference, especially when they were already satisfied with 1970's-era technology.
    Reply
  • DSpider
    "Light field technology doesn’t really record video. It records real-world light ray trajectories and then calculates what the scene would look like based on your viewpoint. Lytro then combines that data with live-action video and with 3D renderings to reproduce true to life imagery."
    So... it's not going to be an actual 1:1 representation of what's in front of you.

    Reminds me of an older Will Smith movie called "Enemy of the State": https://www.youtube.com/watch?v=3EwZQddc3kY That's NOT how cameras work, lol. You can't "rotate" if nothing was there (like another camera) to capture it. It even says "it can hypothesize, Chris". Yeah, right...
    Reply
  • srmojuze
    18529961 said:
    "Light field technology doesn’t really record video. It records real-world light ray trajectories and then calculates what the scene would look like based on your viewpoint. Lytro then combines that data with live-action video and with 3D renderings to reproduce true to life imagery."
    So... it's not going to be an actual 1:1 representation of what's in front of you.

    Reminds me of an older Will Smith movie called "Enemy of the State": https://www.youtube.com/watch?v=3EwZQddc3kY That's NOT how cameras work, lol. You can't "rotate" if nothing was there (like another camera) to capture it. It even says "it can hypothesize, Chris". Yeah, right...

    I wouldn't be so quick to dismiss it. From what I gather from the article this is pretty hardcore, or the start of something ~very~ hardcore... I really thought this was years away, not 2016.

    Basically what we have in the next several years will be an interplay between light-field (eg. Lytro), laser-scanning (eg. Euclideon) and image-scanning (see Blendswap for examples) to recreate an entire 3D capture of a scene.

    If it is just one camera then light-field interpolation will be difficult and you'd generally have a fixed perspective based on where the camera is. "Enemy Of The State" and "Deja Vu" will not quite be possible (unless Lytro has some big tricks up their sleeve and can actually "backtrace" light bounces to such an incredible degree).

    Regardless, imagine setting up 3 - 4 cameras and capturing a scene. That's essentially full VR cinema right there. Given the right software, that means you can have full 3D x-y-z positioning ie. viewing the video capture from any angle.

    Also, since that creates an entire 3D capture of whatever is being filmed (let's assume they start with live action movies) you could go to a cinema, strap on a StarVR or something like that, then all you'd have to do is stream the movie and the user can choose whatever angle to view it from (it might be limited to 3-5 angles at the start for practical and aesthetic purposes)... applications for home cinema would also follow accordingly.

    Over time I imagine live 3D video/light-field capture can be integrated with offline or realtime rendered 3D.

    Wow. I thought 2K each eye was impressive for VR already (with StarVR) but all this "additional" stuff happening. Just didn't expect it.
    Reply