Last year at GDC, I tried Tobii’s eye-tracking device for the first time with Assassin’s Creed: Rogue. This week at CES, We checked back with the company to see its latest improvements on the technology. Once again, Tobii used an Assassin’s Creed title (Assassin’s Creed: Syndicate to be exact) to show off the new features.
We tried out the software and the game on the first laptop in the market that includes Tobii eye tracking, the MSI GT72 Dominator Pro Tobii (which was also revealed at this year's CES) which starts at $2499. Right now, it’s the only device on the market that ships with Tobii’s IS3 eye tracker already implemented in the laptop. If you want it for your home PC or laptop, you’ll have to buy the tracker separately.
What’s New And What They Do
At GDC, my eyes could control only the in-game camera in Assassin’s Creed: Rogue to turn left and right. For Assassin’s Creed: Syndicate, Tobii and Ubisoft Quebec worked together to create four new features to improve the experience.
One of the more interesting add-ons is the ability to mark enemies in Eagle Vision. In the base game, you can do the same thing, but you would have to move the camera around the area and center it on each enemy soldier to mark them on your heads-up display (HUD). With the eye tracker, you don’t have to move the camera at all. If you’re in Eagle Vision, you can mark enemies, so long as they are in the field of view.
On top of that, you can also use your eyes to target a specific enemy for assassination. Some soldiers travel in groups of two or three. You would normally have to move the camera to pick which soldier to kill in the bunch, but the eye tracker allows you to simply look at a specific soldier to assassinate.
One of the new additions to the game was the rope launcher, which makes it easy to traverse the rooftops of Victorian London. With the eye tracker, you can look at a nearby rooftop, ledge or spire in your field of view and just press a button to shoot Evie’s rope.
The last feature deals with driving one of the game’s many horse carriages. Instead of pressing a button or stick to turn the vehicle, all you have to do is look to the screen’s left or right edges to go around a corner. This also has the added bonus of the in-game camera panning in the same direction, so if you turn left, the camera will also look left so you can see what’s ahead of the curve and avoid running into another carriage or obstruction.
What Do I Do With My Hand?
At first, the experience felt strange. I’m so used to moving the camera to mark enemies, look around corners, and highlight enemies for the kill that simply looking at various soldiers to have the same effect felt strange. I actually had to fight the urge to use the Xbox controller’s right stick to move the camera to target enemies.
It took a while to get used to the idea of “looking” at parts of the screen to get the desired effect, and halfway through my session, it was somewhat easier to combine the eye tracking features with the traditional controller actions. I could enter a building from a window on its roof, enter Eagle Vision to mark the enemies inside (with the eye tracking software, of course) and then execute them one by one.
If there was one big issue, it was a combination of my own eyes and the software’s difficulties in tracking them. I could still perform all the specific actions, but it took a little more time for my gaze to register on the screen. My narrow eyes and my thick glasses meant that the software for the tracker couldn’t be as precise as it was for other users. Tobii President Oscar Werner, who walked me through the game, said that the eye tracker had a success rate in the ninety-ninth percentile for accurately detecting and tracking eye movements, but like any new technology, the software will still need even more data on the shape of various eyes and glasses to cover a majority of its users.
Almost There
At the moment, Tobii’s eye tracking works with 27 games, including Elite: Dangerous, Tom Clancy’s The Division and Arma 3, but Werner said that there will be over 100 titles that support Tobii eye tracking software by the end of the year. With each new title, Tobii works with the developers to see how eye tracking can work specifically for that game. However, there might be times when the new technology doesn’t work with a given title. Maybe the experience doesn’t feel as natural as it should for the player, or it clashes with the development team’s vision for the final product.
Still, the new features for Assassin’s Creed: Syndicate show promise. It took only a few minutes for me to get used to utilizing both the controller and my eyes to play the game. Before we left the meeting, Werner showed me a modded version of Grand Theft Auto V that used eye tracking. I could look at a car or helicopter and then shoot my weapon, and it would automatically go to where my gaze landed on the screen.
In an age where virtual reality is already in the works, eye tracking could be some sort of bridge between traditional gaming and complete immersion in VR. One scenario could be the use of eye tracking when talking to a non-playable character in VR. When you look at a certain character, they'll make eye contact and begin the conversation. In fact, it doesn’t have to be gaming. Werner also discussed the idea of using eye tracking in movies with the help of a 360-degree camera. Whatever the case may be, Tobii’s showing impressed us once again at CES, and we can only hope that more developers warm up to the idea of using eye tracking for their games and other visual media.
Update, 1/4/2016, 11:40 a.m. PST: Corrected an error that stated that only Evie used the rope launcher.
Follow Rexly Peñaflorida II @Heirdeux. Follow us @tomshardware, on Facebook and on Google+.
