Exclusive: Fove's VR HMD At CES 2016, First Look

By Kevin Carbotte
published

During CES 2016, Tom’s Hardware managed to get an exclusive look at Fove Inc.’s eye tracking VR headset. Fove wasn't going to be demonstrating anything new to the press, or the public, but the company CTO, Lockliam Wilson, agreed to give us a quick look.

New Hardware Revision Soon

On day one of CES, I met with Wilson, and he showed me an alpha version of the Fove dev kit so that I could get some perspective on what kind of benefits eye tracking would have in a VR HMD. The hardware he showed me is actually an old revision from six months ago, and the software is just as old. A new version of the kit and software will be ready in the coming months, but Fove isn’t ready to show that hardware just yet. Therefore, the below is a look at something a ways off from a final build. Even so, I found it compelling.

There are a number of VR headsets in development, but Fove sticks out from the crowd by including eye-tracking sensors. Fove isn’t partnering with an existing vendor for its eye-tracking components, though. The company has opted to develop its own proprietary hardware in-house, rather than use already available technology. Wilson didn’t go into detail explaining the benefits of Fove's sensors, but by creating its own, the company has full control of the design, which affords the Fove team the ability to fine tune as they see fit. 

Intuitive Extra Input Axis

Fove’s use of eye tracking is beneficial in that in addition to having head tracking, eye tracking allows you to decouple the cursor from your head movement. Imagine trying to navigate menus from a game in VR: You have to pan your head up and down and side to side to get around the menus. With the Fove HMD, navigating menus is as simple as gazing at the item you want to access. The cameras inside the headset track the movements of your pupils to gauge where you are looking. Eye tracking can also be used for aiming. Rather than panning your head around to lock your cursor on a target, you can just look at what you want to attack, and the cursor will follow.

Wilson had me put the headset on and try out a quick tech demo. Before playing anything, the eye trackers need to be calibrated. The calibration test is a simple process that takes only a few seconds. A green dot will appear in an area of the screen, and you have to gaze at it. When the sensor detects that you’re looking at it, the cursor will move to a new location. You repeat the process for each green dot that appears, and once it stops producing dots, the calibration is complete.

After calibrating the sensors, I was shown a launcher application with a couple of different tech demos. Wilson had me try a demo that had me shooting at various targets by simply using my eyes. I was surprised at how well this actually worked, and it's easy to see the benefit of such a feature.

I quickly started imagining what kinds of experiences could benefit from the use of this technology. Eve: Valkyrie came to mind as a game that could very easily make use of eye-tracking to enhance the game play. Imagine looking around through the canopy of your space ship, but when you look at an enemy, it locks the missile launcher onto the target, rather than manually assigning the target. This could be the reality of playing the game on a Fove VR HMD.

Foveated Rendering, For Real

Eye tracking has another big advantage that will affect rendering performance in a big way. With the ability to isolate where you are looking, Fove has managed to get Foveated Rendering working. Using this technology, Fove is able to perform detailed rendering for the section of the scene that is actually in your focus. The rest of the scene can be toned down in quality to free up graphics resources for the important parts.

Nvidia has been talking about getting Foveated Rendering working for VR, but at this time it is simply something the company is working on. Fove has a marked advantage with its eye-tracking technology. Wilson told me that they use a custom shader to pull this off, and he showed me the effect. It’s really quite impressive to see this effect in real-time. You can’t really tell it’s happening when the headset is on, but as a spectator watching the laptop screen, it was apparent what was happening. 

As I mentioned, the headset that Wilson showed me is an early prototype, so it is a little rough around the edges. The inside of the headset is configured with an LED below the lenses. I was told this is for debugging purposes and won’t be present in the developer kits when they ship later this year. Wilson also said that the inside of the headset will be black, unlike the current headset, which is white inside. 

Well Thought-Out Design

Although it's still a prototype, I found the headset to be comfortable. It looks rather large in the front, but it is well balanced, and the head strap is easy to secure. It's reminiscent of the strap used on the PSVR. There's a section that cups the back of your head and tightens with a dial on the back. This works well enough, but I would expect the mechanism to be significantly refined for the next iteration (not to mention the final product). I was told that the company will have a new version of the hardware that we can see in the next couple months. This upcoming revision should have significant improvements across the board.

There are a number of exciting things happening behind the scenes at Fove. When the company is ready to talk more about them, we’ll be here to let you know.

Thanks to Lochliam Wilson and Francesco Simoneschi from Fove Inc. for taking the time to meet with me, and for bringing the headset with them for this exclusive look.

Follow Kevin Carbotte @pumcypuhoy. Follow us on Facebook, Google+, RSS, Twitter and YouTube.

Kevin Carbotte

 Kevin Carbotte is a contributing writer for Tom's Hardware who primarily covers VR and AR hardware. He has been writing for us for more than four years. 

17 Comments Comment from the forums
  • sephirotic
    I was disapointed when I got to know that neither Oculus nor Vive were going to use Eye tracking on their first models. I'm much more interested in fove. After the annouced 600USD for Oculus I may be considering waiting more for the launch of Fove.
    Not to mention it has a higher resolution.

    Eye tracking is an essential part to make the immersion in VR much more efficient, specially the improved resolution only on the area where you are looking, this would allow true 4k, or even bigger resolution displays being viable even with gpu's as weak as a GTX960, although, a high VRAM for 4k may still be needed.

    My only concern is how compatible different brands of HMD VRs would work with different games.
    Reply
  • Astrophysics
    This is the headset that i will be waiting for.
    I have a 3x 25" screen eyefinity setup on a 290x, and it does all i need for the time being.
    having played with a a couple of developer HMD's, my inclination is to believe that without eye tracking and foveated rendering, VR is simultaneously missing a feeling of nuance and depth, and currently requires more horse power than is strictly necessary.
    The degree to which the area outside the center of your field of veiw can be "fuzzier" than what you are looking at is amazing. You only think the rest is a sharp image, but that is your brain creating a seamless montage, rather than the entire field of view being very sharp.
    Eye tracking will simultaneously bring better performance, and better realism. Need Fove.
    Reply
  • f-14
    there's a problem with that VR head set it's covered in plastic infront of your eyes, put some glass in there for a window to virtual reality..... some ones going to walk into a door, off a cliff, into a table, infront of a bus on a long green light.
    it's not a matter of if, it's a matter of when if it hasn't happened to one of the developers on lunch breaks already.
    Reply
  • 8R_Scotch
    The biggest obstacle for Fove is software support and compatible games and the like. The tracking technology is compelling, it both adds to immersion and reduces hardware requirements, two big necessities in VR (Palmer said we'll only get lifelike graphics with something like 16k displays the size of a smartphone, that's a tall order, but considerably less so with foveated rendering).

    My bet is that Fove won't get the needed media traction, developer support to make this an equal competitor to the likes of the Rift and Vive, but it's technology might be incorporated in future generations, Fove might be bougt or may license the technologies it's developing. Otherwise it'll be limited to more niche users, but likely won't go mainstream.

    This tech is exciting, but it's just like hand motion traking, not enough on it's own, but an awesome addition to a more robust VR kit with more consumer visibility and developer interest.
    Reply
  • Bloob
    As long as they'll embrace standards, like OpenVR or similar, while simultaneously offering their own API for the eye-tracking -stuff, they'll have a chance.

    Eye-tracking really feels like something essential for a great VR experience (so we can hit those high resolutions and FOVs).
    Reply
  • clonazepam
    The end-game is probably the digital distribution side, so I expect to see an article about them being acquired this year. Their tech in a Gen 2 or 3 unit from one of the big players.

    There's another company doing 5K screens at 60Hz with 210 degree FOV, but I was turned off almost immediately when they stated they had their own engine and planned to be the one-stop shop for all of the software and hardware. They do have the Walking Dead license, so there is that. Actually, that's a bit scary. Was that sort of a pun too? I heard those are a thing these days.
    Reply
  • kcarbotte
    17301101 said:
    The end-game is probably the digital distribution side, so I expect to see an article about them being acquired this year. Their tech in a Gen 2 or 3 unit from one of the big players.

    There's another company doing 5K screens at 60Hz with 210 degree FOV, but I was turned off almost immediately when they stated they had their own engine and planned to be the one-stop shop for all of the software and hardware. They do have the Walking Dead license, so there is that. Actually, that's a bit scary. Was that sort of a pun too? I heard those are a thing these days.

    The other company you are thinkning of is Starbreeze with the StarVR headset.
    I had a chance to try that out in September and talk to the devs.
    It's not a 5k screen. It's two 1440p screens which add up to 5k. The company is also sonly using 60Hz panels because 90Hz panels that size didn't exist yet. They have no intention of releaseing until those are avaialable.
    Reply
  • clonazepam
    17301101 said:
    The end-game is probably the digital distribution side, so I expect to see an article about them being acquired this year. Their tech in a Gen 2 or 3 unit from one of the big players.

    There's another company doing 5K screens at 60Hz with 210 degree FOV, but I was turned off almost immediately when they stated they had their own engine and planned to be the one-stop shop for all of the software and hardware. They do have the Walking Dead license, so there is that. Actually, that's a bit scary. Was that sort of a pun too? I heard those are a thing these days.

    The other company you are thinkning of is Starbreeze with the StarVR headset.
    I had a chance to try that out in September and talk to the devs.
    It's not a 5k screen. It's two 1440p screens which add up to 5k. The company is also sonly using 60Hz panels because 90Hz panels that size didn't exist yet. They have no intention of releaseing until those are avaialable.

    Nice, looking forward to hearing more about all of the players in the field.
    Reply
  • none12345
    Not going to touch these head sets until they meet at least these sets of requirements.

    1) Peripheral vision is rendered. I don't want tunnel vision. This is a complete show stopper for me, if only binocular vision is rendered, i will not buy the headset.

    2) 8k in smart phone sized screens(i don't want to be looking through a screen door, even 8k isn't enough but it wont be horrible) I may consider a 4k screen, i will NOT consider a 1080p screen. Approx 1000 ppi that close to your eyes is what i would consider minimum.

    3) Total latency from draw call to photons leaving the screen is <=11 milliseconds.(90fps). 90fps is not enough if its for instance triple buffered, that would be a 33ms latency, and that is NOT acceptable. This obviously covers more then just the headset, it covers the entire system.

    4) 2 screens, 1 per eye are used. OR they can make a single screen that renders 2 subscreens at the same time so you do not have a latency differential between right and left eyes. This is not a show stopper for me, but it is highly desired. Its pretty much required to meet the peripheral vision requirement tho; either that or a very wide curved screen.

    5) eye tracking is used. This is not a show stopper for me. However, with the other requirements above, its pretty much mandatory to reduce the processing required to a reasonable level. There is no way you could render the above at full resolution with current gpus, even with quad gpus you couldn't. Nor are you likely to be able to with the next gen cards. However with eye tracking you might be able to do it with current cards, and probably with next gen cards.
    Reply
  • kcarbotte
    17303051 said:
    Not going to touch these head sets until they meet at least these sets of requirements.

    1) Peripheral vision is rendered. I don't want tunnel vision. This is a complete show stopper for me, if only binocular vision is rendered, i will not buy the headset.

    2) 8k in smart phone sized screens(i don't want to be looking through a screen door, even 8k isn't enough but it wont be horrible) I may consider a 4k screen, i will NOT consider a 1080p screen. Approx 1000 ppi that close to your eyes is what i would consider minimum.

    3) Total latency from draw call to photons leaving the screen is <=11 milliseconds.(90fps). 90fps is not enough if its for instance triple buffered, that would be a 33ms latency, and that is NOT acceptable. This obviously covers more then just the headset, it covers the entire system.

    4) 2 screens, 1 per eye are used. OR they can make a single screen that renders 2 subscreens at the same time so you do not have a latency differential between right and left eyes. This is not a show stopper for me, but it is highly desired. Its pretty much required to meet the peripheral vision requirement tho; either that or a very wide curved screen.

    5) eye tracking is used. This is not a show stopper for me. However, with the other requirements above, its pretty much mandatory to reduce the processing required to a reasonable level. There is no way you could render the above at full resolution with current gpus, even with quad gpus you couldn't. Nor are you likely to be able to with the next gen cards. However with eye tracking you might be able to do it with current cards, and probably with next gen cards.

    Most of your complaints have already been addressed. The screen door argument holds no ground.

    1) 110-degrees of view is a lot more than you likely think it is. You really don't see the sides of the headsets at all. Tunnel vision is not something that you percieve in the current crop of VR headsets. It's there's but you really don't notice it like you would expect to.

    2) there is no descernable screen door effect in even the 1080p display used in the PSVR headset. The higher resolution in both the Rift and Vive is even better in that regard, but its not present in any of them really. It's not at all like a the DK2, for a couple reasons.
    First, they are all using low persistance OLED displays that just work better than the display in the DK2. And in the case of the PSVR, the screen has three subpixels (Red, Green, Blue) per pixel which makes a massive difference in clarity.
    It has a lot to do with the optics being used as well.

    3) Latency for VR is a problem that is being actiively worked on by everyone invloved, from the headset makers, to the game developers, to the teams at Nvidia and AMD making the graphics cards and drivers. There are already steps in motion to reduce latency to the levels you expect. Tripple buffering is being bypassed by both AMD and Nvidia so that the latest tracking data is always delivered to the system without worrying about things like tripple buffereing catching up.

    4) 2 screen per eye is already the norm. Both Oculus Rift and HTC Vive has one screen per eye. As does the StarVR headset I mentioned before. PSVR is limited to on screen because the console can only output to at 1080p. Both GPU companies are working on techniques to dedicate one GPU to each eye as well.

    5) Eye tracking will be a big boost in VR performance, but we're not going to see this happen in 2016. The first round of VR hardware will not have eye tracking. Fove seems to be on track to be one of the first to market with this ability.
    Reply