Thursday, May 10th, 2012 Posted by Jim Thacker

FMX 2012: In search of virtual production

The FMX 2012 festival takes place in Stuttgart from 8-11 May. In the first of our reports from Europe’s premier CG industry event, CG Channel goes in search of a definition for this year’s buzzword: virtual production.

It isn’t really possible to say that FMX has a single theme. Europe’s premier CG show, held this week in Stuttgart, spans hundreds of presentations, panels, screenings and workshops, from the cinematics of Mortal Kombat to GPU-based sand simulation. But if FMX 2012 does have a theme, that theme is virtual production.

The show’s Virtual Production strand is scheduled to run for 16 sessions over the course of three days, taking in everything from Real Steel to Bollywood along the way. But on the first day, a rather more fundamental question was in the air: just what is virtual production, exactly?

Is virtual production Avatar?
The answer seems obvious at first: virtual production is Avatar – or at least, the style of film-making James Cameron pioneered on it. John Kilkenny, Executive Vice President at 20th Century Fox, even namechecked the movie in his keynote address, calling Avatar “the birthplace of pure virtual production”, and describing his “lightbulb moment” of seeing the director pacing around a seemingly empty warehouse space, only to discover he was scouting locations on the planet Pandora.

James Cameron on the set of Avatar: the “birthplace of pure virtual production”, according to FMX keynote speaker John Kilkenny. But virtual production is much bigger than the technology Cameron developed.

Virtual production, then, is about virtual set discovery. It’s about virtual cameras, on-set previz, real-time rendering, live playback and motion capture. In short, it’s all about technology, right?

Wrong, says Weta Digital CTO Sebastian Sylwan.

“While virtual production is enabled by technologies – previz has played a great part exposing the production world to it, and mocap has enabled interactive visualisations to move to an environment directors are familiar with – in my view, none of these definite it. There’s no checklist of things you can go through and say: ‘If you have all of these, this is virtual production’.”

But if it isn’t Avatar…
But if virtual production isn’t simply the technology used on Avatar, what is it? Over to Sylwan’s colleague at Weta, The Adventures of Tintin VFX supervisor Wayne Stables:

“If you go to that great source of knowledge that is Google and type in ‘virtual production’, you get back very little information. That might lead you to think that virtual production is a new thing, and I want to challenge that.”

“It’s not just about technology … the technology has helped but [that way of thinking] is limiting us. It’s also bigger than giving back the director creative control. It’s about taking all the lessons we’ve learned about film production and applying them to your virtual world.”

Virtual production is just film production
As evidence, Stables pointed to the fact that even in such a high-tech production as Tintin, Weta relied heavily on traditional skills. The clothes you see on screen may all be digital, but to act as reference material for the artists, the studio’s costume department still produced over 300 real items of clothing.

The hats weren’t just for the film-makers: for Tintin, Weta produced over 300 real costumes as reference, filming an unlucky member of its IT staff wearing them in front of wind machines, in water tanks, and even half-buried in sand. “John had the misfortune of looking vaguely like Tintin, and we abused him throughout,” said VFX supervisor Wayne Stables. “He’s no longer at the company, and I hope that had nothing to do with it.”

Stables also discussed the process of designing the movie’s environments, both interiors such as Tintin’s apartment, and exteriors such as the Moroccan town where much of the later action takes place. “We really don’t see a difference whether you’re going to build a set physically inside a sound stage or build it virtually,” he said.

Weta’s approach brings into question what many would assume to be the defining characteristics of ‘traditional’ and ‘virtual’ production.

“We try to take everything we know about making films and apply them to our world,” Stables said. “At the end of the day, you’re making a movie – and if you’re doing it on a computer, you call it virtual production.

That’s great – but what’s actually new?
But clearly, there is something new about virtual production. We wouldn’t all be here in Stuttgart if there wasn’t. Sebastian Sylwan, who led the Previs 2020 group organised by the American Society of Cinematographers, the Visual Effects Society and the Art Directors’ Guild to investigate the matter, proposed three key goals.

Firstly, virtual production should offer artists the ability to iterate more quickly: a process he likened to rapid prototyping. Many of the key sequences on Tintin – for example, the early scene in which Snowy chases a car through a herd of cows, leading to what one reviewer described as “udder chaos” – were developed by previs artists quickly prototyping visual gags.

Weta animation supervisor Jamie Beard, who headed up the previz team, used a similar process with live actors, heading to the studio’s mocap stage to ‘workshop’ story ideas.

Cast of thousands: key sequences in The Adventures of Tintin were developed by workshopping ideas in previz. Animators riffed on visual gags, while previz supervisor Jamie Beard used a similar process with live actors.

Secondly, said Sylwan, virtual production should be about avoiding “creative islands”. Rather than dividing film-making into discrete stages such as previz and visual effects, each isolated from the other, virtual production should “melt the linearity of the process”.

At its simplest, this means using the same assets for previsualization as for final-quality work. Wayne Stables commented that because he knew that the digital sets used during previz were going to make it through to the finished shots, he could start lighting them much earlier, smoothing the decision-making process.

But as Sylwan pointed out, virtual production is also about not getting stuck with other people’s workarounds. It’s about knowing whether a shot looks the way it does because that’s what the director wants – or simply because it was the only way to previz it quickly enough.

“When you consider the visual complexity we have to put into our work, there are shortcuts we have to take to generate a render at an interactive frame rate. We have to make sure those shortcuts are understood and taken into consideration, and that the virtual artefacts we are introducing don’t get confused with creative intent.”

Finally, said Sylwan – and this is where technology does come into it – virtual production should enable artists to focus on images, not the tools they use to create them.

“We want to have very sophisticated tools that hide their complexity away,” he said. “They should serve the artist’s purpose of experimenting rather than being the front [of the experience].”

Virtual production is end-to-end production
In other words, virtual production is about getting more done earlier, working in a more joined-up way, and ultimately, more creativity. Or perhaps that should be rephrased as: if you can get more done earlier and work in a more joined-up way, it gives you the freedom to be more creative.

Chris Edwards, CEO of previz house The Third Floor, later formulated this as “excitement, efficiently paced”. If artists don’t feel like they’re under the gun the whole time, it enables them to contribute creatively to other aspects of the film-making process – and not just the visual ones.

“The most satisfying shows we’ve worked on are the ones where the filmmakers have used pre-viz for storytelling, not just for visual effects shot development,” he said.

Virtual production is collaborative production
It was this aspect of the new production process – that art can shape the story, not just vice versa – that Sylwan felt went to the heart of virtual production.

“It’s much more about the evolution of film-making process: it’s about introducing a broader collaboration, about introducing additional contributions to the creative decision making.”

Tintin’s iconic motorcycle chase sequence arose from the creative collaboration of director and previz artists. Collaboration is key to understanding virtual production, says Weta’s Sebastian Sylwan.

Ultimately, this collaboration between everyone involved in the film-making process – director, artists, writers and technologists – will be what shapes virtual production. We may not yet have a good definition of what the term actually means, but a consensus will arise out of what we all do in practice. As Sylwan summarised:

“If it’s hard to come up with a definition that serves all the possible aspects of virtual production, that’s mainly because this is just a process of learning and exploration: a process we are all involved with.”

Visit the FMX 2012 website
(Includes live streams of today’s sessions)