7 things we learned from creating Flight in the cloud
Its budget may have been a fraction of that of Life of Pi, but Flight still set a milestone in visual effects. Paramount’s 2012 plane-crash drama was the first movie from a major studio – and, in the shape of Robert Zemeckis, a major Hollywood director – to be rendered entirely in the cloud.
The movie’s 400 VFX shots were created by a single facility: Emeryville’s Atomic Fiction. Although founded only two years earlier and lacking the infrastructure of its established rivals, Atomic Fiction was able to take on the job thanks to its partnership with ZYNC: a cloud rendering platform that promises ‘Goliath power for the Davids’. Its on-demand render service, which is based on Amazon’s S3 cloud, scales up to “an ILM-sized farm in minutes, then back down to the iMac on your desk”.
In his presentation from the Cloud Computing track at this year’s FMX conference, Atomic Fiction co-founder Ryan Tudhope ran through the lessons the studio learned from working on Flight, and provided his tips for other studios considering working in the cloud.
1. Choose a local provider
VFX may be a global business, but think locally when choosing a cloud services provider. Minimising the distance of the artist from the data on which they are working is critical if a pipeline is to remain responsive.
“The data center location is particularly important,” said Tudhope. “For us in the States, Amazon has a west coast and an east coast data center, and there are different configurations of machines in each one. [Being based in the Bay Area] we obviously use the west coast one because it’s a lot faster to get stuff in and out of.”
For larger companies, the problem becomes one of finding a provider that can provide data centers local to all of their individual studios. “Being able to spin up instances in any part of the world is critical,” said Todd Prives, VP of business development at ZYNC. “We have customers in Asia, in Singapore, in Western Europe, in Australia [but since ZYNC uses the Amazon cloud] we have the ability to build duplicates of our original US farms anywhere in the world. That’s critical as we see a more globally distributed workforce.”
2. Check your connection speed
“Obviously, connection speed is extremely important to get all of that data back and forth,” added Tudhope. While ‘private cloud’ systems like those of the Vancouver Studio Group – set up to pool resources between facilities including Rainmaker Entertainment, Image Engine and Digital Domain – use dark fibre connections, a 1-10GB connection should suffice for studios working at a greater distance from their data center.
3. Preparation is crucial
Bringing plates and other assets online is a time-consuming task – and therefore one that becomes all the more significant when working remotely. Atomic Fiction ‘pre-heats’ static data at the start of each job. “When all the plates come in at the beginning of a show, we immediately convert them to EXRs and upload them to the cloud,” said Tudhope. “When [artists] come to render or do comps, those frames are waiting for them.”
4. Minimise ‘stale’ data…
While the cost of renting server space may be less than that of powering, cooling and maintaining local servers, that’s no reason to incur unnecessary charges. “As you’re paying for S3 storage, having 15TB of data up there you don’t need is obviously a problem,” said Tudhope.
In order to minimise this ‘stale data’, it helps to adopt a more games-like mindset. Be rigorous in eliminating unnecessary geometry from your scenes, and be wary of uploading textures at a higher resolution than they will be seen, ‘just in case’ you need the extra detail later. “In the new world order that is the cloud, there are all these [new] things you have to learn and start doing,” said Tudhope.
Tudhope noted that the fact that Atomic Fiction has used the cloud from day one helped its staff adopt this new mindset: “The artists realise we’re paying for rendering … so they work hard to optimise their scenes,” he said.
5. …but don’t take things down too quickly
However, don’t be tempted to delete files too quickly. “When you create large data sets in the cloud, keep them there,” said Tudhope. “[When you generate] a really expensive big render [you can] literally leave it on S3 so when your comp goes to pull that element, it’s already up there: you don’t need to upload it and download it.”
6. Don’t put all your eggs in one basket
While cloud rendering may have been critical to its work on Flight, Atomic Fiction hasn’t entirely abandoned the idea of building its own infrastructure: something Tudhope described as providing a ‘waterline’ for future work.
“Over the course of a show there are peaks and valleys [in processor usage] – there are test screenings, deadlines here and there, and the final push at the end of the project,” he said. “What we want to do is create a local infrastructure that fills in those [valleys] and makes [the peaks] look more like islands. That way you’re not completely reliant on the cloud if it should go down: you have a small local farm that can handle things.”
7. Maintain consistency of file paths
Finally, consistency is critical. “We have to maintain parity between our cloud location and our local location,” said Tudhope. “The paths to our Amazon storage and our local storage are identical.”