'Destination: Mars' With HoloLens And Buzz Aldrin

No matter where I moved, Buzz Aldrin’s eyes followed me. Was I imagining things? No, I double checked: The famous astronaut’s hologram was looking at me, maintaining eye contact as I slowly walked back and forth in front of him. (Er, “it,” I guess, although when a Thing is looking at you, you tend to personify it, er, “him.”)

At that point, it occurred to me that I may be blocking the view of the seven or so other journalists in the room, but then I remembered that they had their own Buzz Aldrins projected into their eyeballs by their HoloLens HMDs, so it didn’t matter where I stood.

We were all experiencing the Destination: Mars HoloLens demo at Microsoft Build, on the first floor of the Moscone Center in a room that was maybe twenty feet square. The black walls and carpet were decorated with white-line drawings of the Curiosity Mars rover and a topographical map of Mars, respectively. However, what we were primarily seeing was an animated and photorealistic Buzz Aldrin standing on the surface of the red planet, acting as our tour guide.

To Space!

Let me back up. The Destination: Mars demo is best described as an elaborate (extremely elaborate) tourism video. In fact, it will be installed at the Kennedy Space Center Visitor Complex this summer for tourists to enjoy. (But there’s a breathtaking catch. More on that in a bit.)

After our group settled into the demo room and strapped on our HoloLens HMDs, we were presented with a hologram of Mars and its satellites. The whole thing appeared to be several feet in diameter. We could walk all around it, although we couldn’t interact with any of the elements.

Then, we were transported into space. Sort of. Remember that HoloLens is AR, not VR, so I wasn’t immersed in space; but within the smallish FoV of the HoloLens (which was a distraction in a demo like this one with 360-degree vistas), I could see space anywhere I turned my head, all around the room.

That was when our tour guide, Buzz, entered the picture. He was as photorealistic and three-dimensional as any avatar I’ve ever seen. He was opaque and bright and reasonably high resolution. “We look to Mars to discover if it was once a home for life, and if it could ever be a home for us” explained Virtual Buzz as the surface of Mars faded in.

It, too, was bright and photorealistic. I noticed that the resolution was mostly superb, but up close--such as on the ground beneath my feet--I could see some pixelation. However, there’s a good reason for that, as it turns out. “Every rock and ripple in the sand is exactly where it is on Mars,” continued Aldrin.

What he meant is that we were looking at images of Mars — actual photos — taken by Curiosity, beamed down to earth, and re-mapped into a hologram.

The Tour

Buzz told us to look at a mountain peak. I placed the little cursor (controlled by my eyes) on the peak, and suddenly some information about the mountain appeared. I got the game. Buzz disappeared for a bit, and we were told by a Voice of God to explore Mars on our own. I saw a small arrow indicating where I should look. When my eye-cursor landed on the right spot on the planet, an animation would appear, offering me more information about whatever I was looking at.

I saw a cracked rock, and then an animation of water flowing over it. Mount Sharp was next, and I saw an overlay of Mount Everest that gave me a sense of the scale of the 5.5 km-high Martian mountain. Another point of interest was a drilling site. I’d noticed the small bore hole earlier, but it was now magnified so I could see more details. Yet another feature was the most impressive: It was a rock, no more than six inches off the ground, with a slight overhang that according to the animation, had been carved by millions of years of wind erosion. (How did they get a photo of the underside of a few-inches-high rock overhang on Mars?)   

The scene shifted, and we were taken to a different place on Mars. I could see the Curiosity rover rumbling slowly into view. (Although it was as photorealistic as possible, it was clearly an animation.) Aldrin introduced Erisa Hines, a person whose job it is to drive Curiosity. Just like Buzz, Hines was rendered photorealistically and in 3D, and both of their facsimiles were remarkably persistent. I had to vigorously shake my head to get any smearing in the images at all.

She thanked Aldrin and proceeded to explain more about the Rover itself. (For example, did you know that it’s nuclear powered? And seven feet tall? And that it uses a laser spectrometer to figure out what Martian rocks are made of?)

Hines and the Rover faded out, and then it was night. (It was at this point that I noticed a “rainbow” effect on my HUD. A couple of minutes later, it looked to me as though it had gotten worse.) A new scene unfolded in front of me: humans on a colony on Mars. As Aldrin waxed wistfully about humans someday populating two planets, a rocket ship off in the distance blasted off of the surface of Mars.

I looked into Buzz Aldrin’s eyes, and he looked into mine, and the demo ended.

The Twist

Back to that twist I mentioned earlier in this article. Although Destination: Mars is indeed designed for tourists and outreach, it’s also an actual tool used by NASA. “This experience, at its core, is a real mission operations tool,” said Dr. Jeff Norris, Mission Operations Innovation Lead for the NASA Jet Propulsion Laboratory (JPL). It’s an adaptation of a Curiosity mission tool called OnSight. JPL uses it every day, as Curiosity sends back images of Mars.

“The terrain you’re going to see in this [demo] is absolutely real,” he continued. “The way you’re looking at it [in the demo] is the way many of our scientists are looking at Mars every day.” They use the images not only to gather data about what’s out there, but also to decide where to send Curiosity next.

And by wearing HoloLens, we can see what NASA scientists see — the actual surface of Mars.

Seth Colaner is the News Director for Tom's Hardware. Follow him on Twitter @SethColaner. Follow us on Facebook, Google+, RSS, Twitter and YouTube.