Post Syndicated from Molly Clancy original https://www.backblaze.com/blog/how-to-run-vfx-workflows-in-the-cloud/
An hour from Queens. An hour from Jersey. Two hours from Staten Island. That’s how long it would take Molecule VFX IT staff to travel from their homes to the closet in Manhattan that housed the team’s LTO device. All those hours, just to spend five minutes switching out one tape.
It was a huge waste of time, not to mention subway fares. The hassle of tape wasn’t the only reason Molecule decided to make their production workflows fully cloud-based, but the IT team certainly doesn’t mind skipping that trip these days.
Moving production entirely to the cloud allowed Molecule to unlock the value of their artists’ time as well as the IT staff to support them, and save money in the process. If your media team has been contemplating a fully cloud-based workflow, read on to learn how Molecule did it—including how they managed to maintain the ability to move data from the cloud back to tape on demand without maintaining on-premises tape infrastructure.
Molecule VFX is a visual effects studio based in New York and Los Angeles that provides the elemental building blocks to tell a customer’s story. They have been servicing episodic television and feature films, like the Apple TV series, “Dickinson,” and the Hulu series, “Only Murders in the Building,” since 2005.
Molecule’s Case for the Cloud
Visual effects artists want to be able to hop into a new script, work on it, render it, review it, QC it, and call it done. Their work is the most valuable element of the business. Anything that gets in the way of that or slows down the workflow directly impacts the company’s success, and an on-premises system was doing exactly that.
- With IT staff working from home, LTO maintenance tied them up for hours—time that could have been spent helping Molecule’s visual effects artists create.
- Beyond tape, the team managed a whole system of machines, networks, and switches. Day-to-day issues could knock out the company’s ability to get work done for entire days.
They knew moving to the cloud would optimize staff time and mitigate those outages, but it didn’t happen overnight. Because much of their business already happens in the digital workspace, Molecule had been slowly moving to the cloud over the past few years. The shift to remote work due to the COVID-19 pandemic accelerated their transition.
Strategies for Moving VFX Workflows to the Cloud
Molecule’s Full Stack Software Architect, Ben Zenker, explained their approach. Through the process, he identified a few key strategies that made the transition a success, including:
- Taking a phased approach while deciding between hybrid and fully cloud-based workflows.
- Reading the fine print when comparing providers.
- Rolling their own solutions where possible.
- Thoroughly testing workflows.
- Repurposing on-premises infrastructure.
1. Take a Phased Approach
Early in the transition, the Molecule team was still using the tape system and an on-premises Isilon server for some workloads. Because they were still deciding if they were going to have a hybrid system or go fully cloud, they took an ad hoc approach to identifying what data was going to be in Backblaze B2 Cloud Storage and what production infrastructure was going to be in CoreWeave, a cloud compute partner that specializes in VFX workloads. Ben explained, “Once we decided definitively we wanted to be fully in the cloud, connecting CoreWeave and Backblaze was simple—if it was on CoreWeave, it was getting backed up in Backblaze B2 nightly.”
2. Read the Fine Print
The team planned to sync incremental backups to the cloud every night. That meant their data would change every day as staff deleted or updated files. They figured out early on that retention minimums were a non-starter. Some cloud providers charge for deleted data for 30, 60, or even 90 days, meaning Molecule would be forced to pay for storage on data they had deleted months ago. But not all cloud providers are transparent about their retention policies. Molecule took the time to track down these policies and compare costs.
“Backblaze was the only service that met our business requirements without a retention minimum.”
—Ben Zenker, Full Stack Software Architect, Molecule VFX
3. Roll Your Own Solutions Where Possible
The team creates a lot of their own web tools to interact with other technology, so it was a relatively easy lift to set up rclone commands to run syncs of their production data nightly to Backblaze B2. Using rclone, they also built a variable price reporting tool so that higher ups could easily price out different projects and catch potential problems like a runaway render.
“There are hundreds of options that you can pass into rclone, so configuring it involved some trial and error. Thankfully it’s open-source, and Backblaze has documentation. I made some small tweaks and additions to the tool myself to make it work better for us.”
—Ben Zenker, Full Stack Software Architect, Molecule VFX
4. Test and Test Again
In reflecting on the testing phase they went through, Ben acknowledges he could have been more liberal. He noted, “I went into it a little cautious because I didn’t want to end up incurring big charges for a test, but Backblaze has all sorts of safeguards in place. You can set price limits and caps, which was great for the testing period.”
5. Repurpose On-premises Infrastructure
The on-premises Isilon server and the physical tape system are no longer part of the active project workflow. They still utilized those devices to host some core services for a time—a firewall, authentication, and a VPN that some members used. In the end, they decided to fully retire all on-premises infrastructure, but repurposing the on-premises infrastructure allowed them to maximize its useful life.
But What If Clients Demand Tape?
While Molecule is more than happy to have modernized their workflows in the cloud, there are still some clients—and major clients at that—who require that contractors save final projects on tape for long-term storage. It no longer made sense to have staff trained on how to use the LTO system, so when a customer asked for a tape copy, they reached out to Backblaze for advice.
They needed a turnkey solution that they didn’t have to manage, and they definitely didn’t want to have to resort to reinvesting and managing tape hardware. Backblaze partner, TapeArk, fit the bill. TapeArk typically helps clients get data off of tape and into the cloud, but in this case they reversed the process. Molecule sent them a secure token to the exact piece of data they needed. TapeArk managed the download, put it on tape, and shipped it to the client.
If Molecule needs to send tape copies to clients in the future, they have an easy, hands-off solution and they don’t have to maintain an LTO system for infrequent use. Ben was grateful for the partnership and easy solution.
Cloud Workflows Free Up a Month of Time
Now that the staff no longer has to manage an LTO tape system, the team has recouped at least 30 payroll days a year that can be dedicated to supporting artists. Ben noted that with the workflows in the cloud, the nature of the IT workload has changed, and the team definitely appreciates having that time back to respond to changing demands.
Ready to move your VFX workflows to the cloud? Start testing today with 10GB of data storage free from Backblaze B2.
The post How to Run VFX Workflows in the Cloud appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.