Augmented Reality, Art, And Mastering The Medium

I just wanted to get the damn shot. It was proving difficult because I had to shoot over the tour guide’s shoulder to capture what was on the screen of the Lenovo Project Tango tablet she was holding, and the camera crew along on my tour kept edging into my frame. To make matters worse, because Lenovo was sending dozens of us through an exhibit in the Museu Nacional D’Art De Catalunya in small groups (I was Group 16), we didn’t have time to tarry. “We need to keep going,” whispered the guide as I tried to grab some b-roll of her holding the device.

It was a scramble, and it seemed like our run through the gallery was over in seconds. And the moment I exited and found myself back in the cocktail area with the rest of the journalists and Lenovo staffers present at the event, it hit me that I’d just blown a special opportunity.

I was given a chance to see works of art that I’ll probably never see again, and in a museum—achingly beautiful, like everything in Barcelona—that I may never have the chance to visit again, and I couldn’t tell you a single piece I saw.

According to the video, I shot a Rodin. I never actually looked at the painting in person, though. I was glued to my little camera screen, trying to get a good look at a tablet’s screen, and recursively failing.

Considering the device Lenovo was showing off was an augmented reality device, the bitter irony that I had “augmented” nothing stung; instead, I had reduced my experience in the art museum.

I stood there blinking for a moment. I couldn’t get back into the gallery, because other groups were getting their turn, and at that point in the evening, all the other exhibits were closed. I decided to head outside, where the museum steps—up on a veritable mountainside—afforded a panoramic view of the city.

This is not a diatribe against screens, nor against technology. Show me a grumpy old man carping about the good old days when people actually talked to each other at breakfast, and I’ll show you a guy who had his nose buried in a newspaper, absently muttering Mm hm, that’s nice to his family through the morning meal every day.

No, people have always found ways to disengage and distract. It’s just that the distraction du jour is often a mobile device.

To be clear, though, smartphones and tablets are not inherently “distractions;” they’re tools. Exceedingly powerful tools, at that. Think about what a smartphone is: a computing device that fits in your pocket and is equipped with an always-on portal to the collected knowledge of all of humanity, and with which you can instantly video chat with your family even when they’re an ocean away.

Of course, you can also use it to sit with your back to the breathtaking Barcelona skyline, as dusk descends upon the city and paints the Sagrada Familia in oranges and purples, and play Angry Birds for an hour.

It’s an easy (and already tired) trope to point to the growing tidal wave of AR and VR and pontificate on the dire state of disengagement. Images of zombified kids wearing VR HMDs are no doubt keeping Very Concerned Parents up at night. But that’s completely missing the point. Augmented reality, and even virtual reality, are actually about deeper engagement.

Imagine that you were able to navigate a city with a HUD (what Google Glass tried and ultimately failed to be) that fed you information about everything you looked at, with the names of businesses, restaurant ratings, and more all superimposed on the real world. How about the ability to look at a job site and have blueprints painted on the wall that you need to see inside of, and then collaborate with colleagues remotely on a solution, as Microsoft demonstrated with HoloLens.

Even in VR, wherein you immerse yourself in an entirely different world, there’s room for engagement. Companies like AltSpaceVR and Campfire Union are creating opportunities for social interaction in fun and exciting ways. Soon enough, you’ll likely be able to enter a game in a virtual world and play with your friends, just as so many of us do now in front of 2D screens. (In VR, too, immersion is a deeply engaging experience; yes, you remove yourself from the real world, but you dive into another.)

And on and on. We’re just now getting a glimpse of what these new technologies can do. 

AR and VR are about to deliver us a whole new set of tools for working, communicating, designing, dreaming, and yes, playing. The challenge is to learn what to expect from, and how best to use, those tools – to master the medium.

This is nothing new. We’ve all been faced with this need to master new media in our lifetimes. The Internet, PCs, smartphones—there are better ways and worse ways to use them all.

The Project Tango implementation that Lenovo showed us is actually rather simplistic, but it’s a marvelous way to navigate any museum. If you want to learn more about a given artwork, you can point the tablet at it and gain access to all kinds of information. What is this piece? Who was this artist? When did he live? Where? What were his inspirations? Who were his contemporaries? And then you can look at the tablet’s display to see a blue dotted line superimposed on the floor, escorting you along through the galleries.

Or…you can stare at famous paintings through the tablet’s display only, and miss the art entirely. The former is what augmented reality can give you, even in a simple application like this one; the latter is a waste, and it’s not the technology’s fault.

I’m glad Lenovo chose the art museum for this event. Had we been, I don’t know, in some mall using the Project Tango tablet to find sales or something equally banal, I wouldn’t have learned anything other than the fact that augmented reality sure is neat.

It was getting dark. I popped back inside the Museu Nacional D’Art De Catalunya and asked one of the Lenovo handlers at the front desk what the arrangements were to get back to the convention center. There would be a shuttle bus, or I could get a ride to the nearest metro station half a mile or so away.

I thanked her, slipped my phone in my pocket, and walked the distance to the metro, down to Plaça d’Espanya. Barcelona is gorgeous at night, and I didn’t want to miss anything else that evening.

Seth Colaner is the News Director for Tom's Hardware. Follow him on Twitter @SethColaner. Follow us on Facebook, Google+, RSS, Twitter and YouTube.

  • falchard
    The guy in the first photo looks like an 18th century painting of a woman.
  • scolaner
    The guy in the first photo looks like an 18th century painting of a woman.

    That is a woman. But I agree that it looks like a painting, and I kind of love the shot. Wish I could say I did that on purpose, but it was dumb luck. ;)
  • d_kuhn
    Probably not the forum for an art discussion, but I manage a group of Imaging Scientists (spend a LOT of time around cameras) at a tech. company and my wife is an artist and works at a world class art museum... so we've had this discussion many times before.
    There are many ways to experience art, I’m not one to tell anyone their way is ‘wrong’ but the technique of visitors only viewing an artwork through a camera lens is very common and there’s a cost attached. It seems most prevalent among visitors who are on prepackaged tours (here in the US that most often means folks from Asia). These individuals have limited time to view the museum so they speed through the works, stopping long enough at each to snap a picture or three before moving on (similar to the authors experience above). I’m not sure if the time constraint drives the behavior or if it’s cultural, but the result is that the experience of the art pieces is consumed once the individual has returned home not while actually in the museum.
    My wife and I agree that while this technique does give the viewer a permanent record of the artwork… it is a second hand account, the individual MISSED the opportunity of a firsthand experience due to their choice to spend their limited time using a camera instead of interacting with the art. I’ve done this myself on a trip to Shanghai (Shanghai Museum – HIGHLY recommended) until a guard warned me that cameras weren’t allowed… at which point I switched to experiencing the art instead of just capturing it. The result is that I have lots of pictures of one wing of the museum, but few memories of that wing. The pictures are nice, and give me something to show and talk about with folks… but what we talk about is actually the parts of the museum where I wasn’t using a camera!
    Putting down the camera and experiencing the art first hand allows a much deeper engagement with the various pieces than a static image can provide. Art doesn’t need to be moving to be dynamic, sculpture needs to be viewed from many angles, even paintings on a wall only really come alive when you can view from multiple distances and see how your frame of reference changes what you get out of it. I’m not talking about the (silly to a scientist) art snob discussion of subjective impressions like they are objective truth, but rather the emotional response to a work (that you don’t need an art degree to have) and how that response shifts as you move around it.
    Now we have the advent of VR and AR… two very different topics. VR at some point in the future could provide you with the same opportunity to interact with a piece that you have in person. It will need more resolution than current systems and also huge data requirements. Even a ‘2d’ painting would need to be captured in 3d and at very high resolutions to allow viewers to experience things like the artist’s stroke technique… something that adds a lot of ‘depth’ to paintings, resulting in data sets in the tens to hundreds of gigabytes of data per piece. Some will still argue that the ‘museum experience’ is missing, but IMO that’s getting a bit hair splitting… and the value of this to people with limited mobility could be huge.
    AR seems like a ‘low hanging fruit’ to me, rent a hololens and get access to interactive content for each piece as you travel through the museum. Maybe it’s the artist pointing out different features of the work or historians talking about the impact of the last restoration (and overlaying ‘before’ imagery!). For someone like me who gets into the history of a piece as much as ‘the moment’, I view this sort of capability as a fantastic enriching of the museum experience. My wife on the other hand thinks it’s just another distraction… she’s the person you see sitting on the bench for an hour looking at the painting with frustrated glances at the tourists standing in front of her snapping pictures (or God forbid… SELFIES). Her ideal museum experience would be sneaking in after closing, with no noise, technology, or other distractions. Which one of these experiences are ‘right’… I’d say both and neither, in an ideal world we’d all get enough time for both approaches, but if you have to choose one then choose the one that works for you. AR is already available at many museums… though it tends to be in the form of audio or internet ‘self guided tours’, simple but still capable of delivering more context than you’d otherwise get. If distractions aren’t for you… plan your trip off-season and go on a weekday – then shut off your phone and get as luddite as needed to meet your requirements for disconnection. There’s no right answer, but as someone in the field I look forward to technology offering more OPTIONS when it comes to consuming Art onsite as well as opening it up to people who would never otherwise have the opportunity to view it.
  • d_kuhn
    I agree that the photo in the article is fantastic.
  • bit_user
    This is the main reason I don't take many pics when I'm out with people. I don't want to be distracted by always looking for good photo opportunities. On the other hand, when I'm bored, I sometimes occupy myself by snapping pics.

    Anyway, just think how much more you'd appreciate the art if it had taken you 3 months to get there. That's another downside of technology - it's hard to have the same level of appreciation for something just handed to you. But I'll take that tradeoff, any day.

    17657914 said:
    The guy in the first photo looks like an 18th century painting of a woman.
    That is a woman. But I agree that it looks like a painting, and I kind of love the shot. Wish I could say I did that on purpose, but it was dumb luck. ;)
    I briefly wondered if it was even a shop. Like maybe some kind of inside joke to other art aficionados.
  • scolaner
    That's another downside of technology - it's hard to have the same level of appreciation for something just handed to you. But I'll take that tradeoff, any day.

    That's a really great point. Many of us remember how music, for example, was a scarce and expensive resource. Purchasing a new album was an event that including standing in a record store at those demo stations with the big headphones. Now? Spotify. ALL THE MUSIC. Instantly. Which I love, does reduce the value of it, in a way.