Moving Day: Connecting Media To and From the Cloud

READ MORE: 10 key learnings from moving petabytes to AWS, Azure and Google (Signiant)

Cloud object storage can be a powerful ally to media companies. Whether it’s a simple backup or disaster recovery operation, overflow when there’s an unexpected surge in storage needs or a more complex workflow where assets are required to be in the cloud, every media company should be equipped to leverage the cloud, if and when a need arises.

While moving assets to the cloud may seem simple on the surface, as with many things in M&E, it’s not as trivial as it sounds.

In an industry that deals with huge files, complex supply chains and growing security challenges, there are several factors to consider. Fortunately, there are a wide variety of tools available to move content into and out of cloud object storage, including many offered by the cloud providers themselves.

A new guide, courtesy of file transfer specialist Signiant, outlines the main criteria for moving assets into and out of the cloud. The company acknowledges that major public cloud providers — Amazon, Microsoft and Google — are actively rolling out innovative new services in this area while angling towards its own SaaS solution.

Speed

In M&E, where deadlines are tight and datasets are measured in gigabytes or even petabytes, standard tools just don’t do the trick. That’s especially the case when moving large files over long distances or congested networks.

“The bigger the files, the longer the distance or the more congested the network, the more performance will be impacted,” says Signiant.

It claims to have observed huge variances in network congestion at certain times of day with the public cloud providers, which “without the right software” can lead to unpredictable performance.

For example, Amazon’s speedy transfer service, S3 Acceleration, “can eliminate the distance of the initial transfer into AWS [but] congestion may still be an issue.”

S3 Acceleration also only works with Amazon storage so, while according to Signiant it can be an easy way for developers to speed up workflows they have built using Amazon tools, this still leaves you building your own custom solution.

“While that might be necessary in situations with extremely unique requirements, there are out-of-the-box, full-stack products that offer equal or better performance at a much lower total cost of ownership. For most situations, building your own solution simply doesn’t make sense.”

Reliability

When dealing with large files, network instability and file corruption can cause disruptions and delays so transfer tools must be able to navigate interruptions and deliver files unaltered, with byte-for-byte accuracy.

Security

Security a constant concern in media workflows. For companies that may not have the in-house security expertise and appropriate physical security protocols, cloud storage can be more secure than housing files on-premises, assures the vendor. “But, that is only true if configured properly and only if the transfer tools secure assets in flight.”

If you do choose to store assets in the cloud, make sure your AS3 buckets, Azure blobs and Google buckets are configured properly, Signiant advises. Each cloud provider offers guidelines to help ensure your assets are secured properly once they land in their storage.

Monitoring, Reporting and Alerts

Hand-in-hand with reliability and security is visibility — the ability to view transfers in progress, report on transfer history, and receive alerts when transfers are complete or when things go awry. “Asking IT resources to dig through log files when there is an exception or to verify a valid transfer is not a scalable solution,” says Signiant before warning, “The tools provided by major cloud vendors offer robust logging of events but will leave your teams processing large log files, building your own dashboards and reports, or deploying third-party tools to view the required information.”

Hybrid Cloud/Multi-Cloud

While Amazon remains the dominant player in cloud, Microsoft and Google continue to grow their footprints and each are innovating nonstop.

“The cloud wars are on and so it makes sense to maintain the flexibility to work with multiple cloud providers for specific workflows and to be able to easily switch providers as capabilities evolve,” Signiant notes.

Hybrid cloud seems to be the direction of most media companies just now. Rather than moving all their assets to the cloud, they are maintaining a strategy with some assets in on-prem storage and others in the cloud, sometimes across multiple providers and regions. This is what Signiant refers to as a hybrid cloud/multi-cloud strategy and it’s a trend that doesn’t appear to be going away.

“In a world defined by constant change and innovation, it’s important to consider tools that act as an abstraction layer, making it seamless to work with on-prem or cloud storage and across multiple cloud providers.”

Working with Multiple Locations

In cases where content is aggregated from multiple producers in different locations, where creative teams are collaborating from locations around the globe, or where content is distributed to multiple platforms, the cloud can be a powerful resource.

“While dedicated connections have a role in some workflows, consider whether content will flow to and from a single location, what your utilization will be and if you’re 100% committed to a single cloud provider. In almost all cases, you’ll find there is a more flexible and more cost-effective approach.”

Frequency & Utilization

The next question to ask is how frequently and consistently you will be moving data to the cloud. Is this a one-time move, is it an ongoing backup operation with a steady flow of data, or will the flow of data have unpredictable peaks and valleys?

Estimating utilization is a challenge. “More often than not, companies overestimate their intended utilization leading to unfavorable economics,” says Signiant. It claims that services like ExpressRoute from Microsoft and DirectConnect from Amazon are costly when not fully utilized, “and it’s rare that operations keep these pipes full 100% of the time.”

Unsurprisingly the solution is a software that offers consumption-based pay-as-go style pricing.

User Access & Administration

It goes without saying that the big three public cloud providers offer incredibly powerful tools to work with their storage, but they are targeted at developers building solutions, not end users, according to the vendor.

“APIs and command line tools are great for developers, but how do your creative end users easily and securely access the media assets they need? These users require a simple user interface to access media assets that is consistent regardless of whether assets are stored on-prem or in the cloud.”

Signiant advises buyers to look for tools that offer “delegated administration” so that operations can easily manage teams, partners and projects without IT intervention and do so regardless of which storage type IT chooses to deploy for a given project.

Automated Transfers

Simple, automated backup and disaster recovery operations are a great way to leverage cloud storage. More distribution workflows are now including cloud, including fan-out distribution where a large asset, such as a DCP, is delivered to multiple places at once. In these cases, the asset is moved first into cloud storage and then is simultaneously delivered to several locations, leveraging the essentially “unlimited” bandwidth of the public cloud providers.

Automated workflows should be easy to deploy and manage and should abstract storage so that unattended content exchange can happen between on-prem storage locations and the cloud without having to retool.

While automation tools have historically been “complex and prohibitively expensive,” now modern SaaS solutions make it easy and accessible for any business to automate content exchange between all types of storage.

Inter-company Content Exchange

The final consideration in the list, inter-company, is one that is tied closely to automation and is also a growing challenge across the industry. With increasingly complex media supply chains driven by multiple production partners at the front end and multiplatform delivery on the back end, there are simply more companies, big and small, interacting to create and distribute content.

With inter-company transfers, each side may not know, need to know, or even care, what type of storage their partner is using. They just need to get the content to the right place in a timely and secure manner. While the basic principles of inter-company content exchange are similar regardless of storage type, cloud storage can introduce some complexity with regard to costs and cost sharing. Each side needs visibility and control of their own systems while keeping their partners out of their networks and storage.

Are you interested in contributing ideas, suggestions or opinions? We’d love to hear from you. Email us here.