Jon Favreau's Lucid Dream: A Night With Wevr And 'Gnomes & Goblins'

Next week, Wevr, in conjunction with Reality One, will introduce a preview experience of Gnomes & Goblins, a delightful VR creation from Jon Favreau. Yes, that Jon Favreau.

At an evening gala featuring what felt like green-screened-perfect weather, a food truck straight out of the movie Chef serving Korean Kim-chi tacos and goblin cake pops, with a swank that seems only possible in Hollywood, I got to try out this magical adventure and meet with the production's animators, Wevr's co-founders, and Favreau.

It’s difficult, after all of that, not to feel good about what’s to come, and yet like everything else in VR in its early life, I’m left wanting more. And I mean that in the best, if not most frustrating way.

Fortuitous timing places us in the position of documenting the early history of virtual reality, chronicling its modern-day inventors, sorting out hype and myth, predicting winners and losers, wearing our hats of skepticism, dampening our enthusiasm in the name of objectivity, and replaying the eureka moments. It is all still, quite honestly, more promise than real, with VR’s modest footprint, performance challenges, and an omnipresent lineup of opportunists who are here today but most assuredly gone tomorrow.

In other words, it is a lucky time for all of us, but it feels especially lucky to be a technology journalist, sampling at a cake pop buffet.

I keep seeing these glimpses of what could be; no, what’s going to be. And it thrills me. It thrills me for me, and for you. And then sometimes I get home and let what I just witnessed begin to sink in, and it all feels so incomplete.

In the case of Gnomes & Goblins, that is by design. It's a preview experience, a trial balloon. For now, it doesn't go very far. Oh trust me, Jon Favreau has it all mapped out, plans in that curly head of his that are oh so money, he doesn’t even know how money they are. But first he wants to see what we think and if we like it.

Lucid Dreaming With Gnomes & Goblins

We do like it. This is not really a movie, nor a game. Maybe it’s something in between, but in Favreau’s head, VR is a blank canvas. At one point during our conversation, he said that he didn’t set out to make a movie with Gnomes & Goblins, but that it was more of a simulation experience.

“Where games tend to be puzzles, and movies tend to be voyeuristic experiences,” Favreau said, "we wanted something that feels immersive, more like lucid dreaming, and you’re flowing through the world, and there’s integrity to the logic of it . . . and you have agency within it. You let people go wherever they want but make sure wherever they go, something interesting happens, like Disneyland."

But here’s what it is so far: Little story triggers move you into and through the actual experience. Grab a candle, and use it to light one of the “preview” signs, and away you go. You enter the preview in a lush forest world, surrounded by trees, lit from within, and you are surrounded in a colorful dream-like scene. You can see doors into the dwellings built into the trees, and you can peer in, studying your surroundings, guided by the HTC Vive Chaperone system. Trees contain acorns and glowing orange fruit. Acorns litter the ground. You can pick them up and toss them. 

But those acorns and that fruit are desired by a tiny little goblin, which you first hear, and then see, running across a bridge and out of sight.

At first, the goblin is shy. It makes direct eye contact and then slowly backs away when you approach it with an acorn, so you have to set the acorn on the ground and let the goblin come get it. It hesitates as it picks up the acorn--eyeing you, grabbing it--and then it runs away with its prize. But what it really wants is what looks like a giant peach on the tree, and you can use an acorn to knock the fruit off the tree. The goblin will come a little closer for that.

Image 1 of 9

[Image credits: Wevr]

Eventually the goblin emerges ringing a bell, offering it to you. It hands it shyly to you, and when you ring it, the world around you dances with fireflies. Ring it more, and suddenly you’re transported, and you become smaller so that you’re in the trees. Ring it even more, and you’re with the goblin, who is now your size, and it's drinking out of one of the many cups strewn about the tables inside of the house within the tree. Ring the bell yet again, and you’re up on the ledge of the treehouse, feeling as if you’ll fall to your death if you step over the side. One more ding-a-ling, and you’re on the ground.

It’s hard to know where the story goes from here, though, and for the preview, that’s really the end.

How Gnomes & Goblins Began: Thank theBlu

This is theBlu (also from Wevr) and Oculus Studios' Henry, taken to new levels. In fact, Gnomes & Goblins is a direct descendant of theBlu. Andy Jones, who is the animation director for Gnomes & Goblins, and was the animation director for Favraeu’s recent version of The Jungle Book (and also, the academy award winning animation director for Avatar) tells the story of leaving The Jungle Book studio at Disney one day to go to Wevr, and Favreau decided to tag along, never having experienced HTC’s Vive. Favreau was mesmerized by theBlu, and he called Jones into his office the following day to say he’d been up at 4am and had already plotted out an idea to pitch to Wevr. That was a year ago.

While you can spend a fairly unlimited amount of time exploring Gnomes & Goblins, it is clearly just a short, trial experience for now. Jake Rowell, who directed theBlu at Wevr and is the creative director for Gnomes & Goblins, said that although it's tempting to try to build out everything, the team’s focus was to start small, and on the most important element, which was building a sense of presence and character connection. Rowell said that early discussions focused on early Disney animated characters like Mickey Mouse and Steamboat Willie, which represented early attempts to create charm and connectivity with an animated character.

MORE: Wevr and its history

MORE: Wevr's content transport platform

AI Systems And Open Worlds

The preview, then, is a foundation. The team created an AI system specifically for Gnomes & Goblins (there is an entirely separate one specifically for theBlu), and each person will experience the environment differently, based primarily on what you, as the viewer, do. In the preview, programmatically, the bell is the final part of the story, but the way you get there, Rowell and Jones explained, is different.

Here is more from Wevr co-founder Neville Spiteri on AI: "Wevr has developed a VR character AI framework which is being applied in Gnomes & Goblins to enable digital characters with interaction capability. The AI framework is designed to allow "digital acting" - character performances that are directable by a director. This can be seen in the facial and body behaviors of the goblin and the responsiveness to real time actions by the player in the VR experience."

In the bigger version of Gnomes & Goblins, those story arcs will be practically infinite, much like an open world game. This is the part that makes this somewhat of an animated piece of content and VR game simultaneously, and also what makes this kind of content so compelling and fresh. There’s no script, Rowell said; it’s just your own personal agency, and the goblin changes what it does each time, for each person, for every viewer action.

I asked Favreau what he learned from this experience, and how creating Gnomes & Goblins was different than directing regular films. “You’re not thinking about where the camera is,” he said, “or what the cut points are, you’re purely thinking about character, connection, emotion and story flow.”

One aspect that’s important to these interactive VR experiences is the AI systems. Favreau referred to the Turing test aspect of the creation process. "How convincing that other character is when it’s looking you in the eye, and you feel like it’s alive,” he said. Although the stylized nature of the world in Gnomes & Goblins is important, Favreau said the team spent the most amount of effort on the AI, because the success of the experience hinges on how convinced you are of the connection you’re making to the character, and in the world you’re in.

Favreau was careful to point out that Gnomes & Goblins is being built with an Indie game-scale group, in a VR lab, without the machinery of a big film like so many he’s worked on to this point. It’s purely about deciding what experience the creators want to serve, then listening to the audience, and building.

Favreau said that the team has a well thought-through game plan for what the entire experience will be, and that he hopes creating it will help build a bridge for VR as a more mainstream medium. “If you can make good content . . . it moves the ball forward, like theBlue and Tilt Brush.” He added, “I was compelled by work others had done; everyone contributes a little bit.”

We discussed AR, and he quickly grew excited, talking about the role Pokemon Go has already played, introducing a mass audience to the concept of AR. “Those steps get us to the next thing,” he said, “and everybody benefits." 

"It’s like you’re hosting a party, you’re building an attraction at an amusement park, or your a dungeon master. You’re curating an experience."

Hosting, building, curating. The "ing" part has me impatient. I got a taste, and now I want more. Such is our place in the history of VR. Wevr is creating an experience with a team of people who have either film or game creation backgrounds, and they are bound by both, and yet also by neither; in VR there are endless and undiscovered possibilities.

The creators are animating hand-crafted worlds and characters, but with new-school tools such as AI states.

They've created a stylized look, but also one that allows them to use tricks to enable 90fps performance. They are setting the stage for a story that you the viewer tell, through your own personal interaction. Thanks to modern-day technology like the Vive, the creators know where your eyes are, where your hands are, how fast you're moving, your height, and so on, and therefore the goblin knows it, too.

That's the magic they've seemed to capture here, and which they will expand upon. Will it work? I can't wait to find out.

You can download Gnomes & Goblins for free on September 8, for the HTC Vive only, on the Wevr Transport platform, Steam and HTC's Viveport.