Categories
moviola.com Post Production

Removing glare from glasses

There are all kinds of finishing effects that are within the grasp of the average editor. You just need to know the recipe. Removing reflections and glare from glasses is one of those tricks that seems impossible, but is perfectly achievable if you understand the steps involved.

In the video below I’ll walk your through the entire process. In fact, this process is a pretty standard approach for any situation where you need to remove artifacts from a shot. The steps break down like this:

Step 1. Create a cleanplate

A cleanplate is a version of the shot with the artifact removed. This requires the most creativity. You need to find different frames where the artifact has moved and copy those pixels into your cleanplate, or recreate the area underneath using a clone (‘rubber stamp’ in old-school Photoshop parlance) brush. However, it doesn’t usually need to be perfect. You’ll see in the video that I was intentionally lazy and left the cleanplate quite splotchy. However, once it’s applied to the shot you don’t notice the little inconsistencies.

Step 2. Track the surface to replace

When I say track, I mean Mocha track. Boris FX’s Mocha Pro really is the only game in town here. Lots of other applications claim to have a good planar tracker, but I’m yet to find one that rivals Mocha Pro. Mocha is especially good at handling skin deformation–essential when you’re doing facial surgery on a shot with a lot of dialog.

Step 3. Matchmove the cleanplate using the track

Now we take the cleanplate and hook it up to the track we just created. The result is that the cleanplate moves with the source footage so that our cleaned up pixels always line up with the surface they’re intended to replace.

Step 4. Composite the cleanplate

With all the movement aligned it’s a fairly simple matter to mask off the area to be replaced and composite the cleanplate over that area. A little judicious blurring to the matte and the cleanplate should blend in nicely. Often you’ll also need to animate some color correction to the cleanplate to compensate for lighting changes.

Step 5. Fudge the rest with a warp

This is the crucial step that most of those tutorials out there miss. Warpers are magical tools. They allow you to take images that don’t quite fit together and force them to line up–in a very believable way. The cleanplate you created in step one will often start to misalign due to changes in perspective of the tracked surface, change in distance from camera, or even deformation in the surface being tracked. By throwing a warper into the mix you can nudge the shape of the cleanplate back into place.

This is all part of our Impossible Shots series. If there’s an impossible shot you’d like to see us tackle, email help@moviola.com.

Categories
Production

Virtual Production: It’s the future you need to know about

In an age of dizzying technological churn, it’s hard to know which emerging fringe tech is worth paying attention to. Not so with virtual production: Anyone who’s experienced it can instantly see where things are headed. So what exactly is it, and when will you—the Indie filmmaker—be able to get your hands on it?

 

Bringing the virtual into the physical

Most people—if they have an awareness of virtual production—think of it as some kind of previsualization system, or picture Peter Jackson on a greenscreen set looking at CG trolls through a VR headset. Now while all this is true, and part of the history of virtual production techniques that has brought us to today, the virtual production of tomorrow will be far more intrusive into the physical realm.

Lux Machina, Profile Studios, Magnopus, Quixel, and ARRI in collaboration with Epic Games, have been showing a prototype of a virtual production set based on real-time projection of background environments via massive LED panels. At its basest level, this is an updated version of the old rear-projection systems used for shooting in-car scenes, where moving footage of the background was placed behind the actors. This is, however, a gross oversimplification. The system is superior to its historical predecessor in every way.

The benefit of using the LED light panels is that they actually provide a physical lighting source for the live action actors and props.

An example of a game engine skybox unwrapped.

Essentially, we’re dealing with the physical equivalent of what game developers call a “skybox,” a cube containing the HDR lighting environment intended to illuminate the game set. Full-scale LED panels project the background image from stage left, stage right, the main rear projection, and the ceiling. Presumably foreground lighting could be provided by either more traditional stage lighting sources, or perhaps a matrix of LED PAR cans that can simulate the portion of the HDR coming from the front of the scene. (And alternative option is a curved background screen configuration.)

What’s even more significant is the ability for the background to update its perspective relative to the camera position. The camera rig is tracked and its ground-truth position is fed into Unreal Engine, which updates the resulting projections to the LED panels. The specific region of the background directly within the camera’s frustum is rendered at a higher resolution, which looks a little strange on set, but obviously makes sense from a processing efficiency standpoint (the entire setup requires three separate workstations running Unreal Engine: one for the two side stage panels, one for the background panel, and the third for the ceiling).

The result of the perspective shift means that backdrops feel much more natural with sweeping jib and crane moves. The real-time nature of the projection also means there’s the potential to have secondary movement like a breeze blowing through trees, or last-minute directorial changes, like literally moving a mountain to improve the framing of the shot.

Lighting can also be changed in an instant. Switching from sunset to sunrise to midday is as simple as dialing up a different sky. Changing the direction the sun is coming from is as simple as rotating the sky image. Try doing that in the real world.

The beginning of the end for exotic travel?

One of the most exciting aspects of this virtual production stage is the potential to take a script with globetrotting locations and bring the budget from the nine-figure realm down to an indie level. Imagine being able to shoot in Paris, New York, the Everglades, and the deserts of the Middle East from the same soundstage, all in the course of a three-week production schedule? We’re not there yet, but it seems likely that over the next three to seven years the GPU compute power, the sophistication of the software, and the display tech will get us there. Crowd simulation software could even provide digital extras for the mid and far-ground.

And then there are the exotic sci-fi and fantasy scenes. Such sets are extremely complex to build, and greenscreen composites are often unsatisfying. The CG often feels a little too ethereal. With the CG set illuminating actors and foreground props on the virtual production stage, there’s an immediate improvement in the way the real and computer-generated scene elements mesh.

No doubt Tom Cruise will still be filmed hanging from real skyscrapers in Dubai, but for indie features and episodics hungry for a “bigger” look, the virtual production soundstage will almost certainly kill the exotic production budget. Film crew members will need to find themselves a cosy travel magazine show to work on if they want to any jet-setting perks.

Limitations

I’ve been careful to talk about this as the future; there are a couple of limitations that hold it back from being 2019’s solution to location shooting. Part of this is simply the nascent state of the tech; everything is still very much in the prototype stage. This is not to say that it can’t be used for production today, just that there are certain compromises that would need to be accepted.

I was extremely impressed with Epic’s David Morin, who is spearheading the virtual production initiatives at Epic, and his cautious approach to promoting the venture. He’s clearly mindful of the way overhype has caused the bubble to burst too early on many VR and AR technologies (along with the entire stereoscopic industry) and is thus being careful not to reach for hyperbole when talking about the current state of the art.

To me, the biggest present limitation seems to be the display technology. Don’t get me wrong: they look pretty. The LED panels simply lack the dynamic range to replace the 14-16 stops a modern cinema camera can capture in the real world. The dynamic range is probably adequate for most television and OTT content, but with the industry pushing hard into HDR standards, a reduced contrast range will have limited appeal.

A possible compromise is the system’s ability to instantly convert the background to a greenscreen. This allows directors to visualize the scenery on-set, while still performing the final composite as a post process with floating point precision. Of course, at that point you lose some of the benefit of the interactive lighting provided by the panels. The scene can still be lit from above and the sides, but much of the background illumination will obviously be replaced with greenscreen illumination. Still, a significant improvement on a traditional greenscreen.

(I can’t help feeling that by pulsing the refresh rate of the LCD panels the engineers could extract some kind of segmentation map, which would allow a higher dynamic range background to be composited in post, while still using the LCD panels to illuminate on set…)

The solution to this problem is mainly dependent on future advances in the display hardware. Another limitation that’s more intrinsic to the system is the directionality of the light sources. This is an issue that also affects HDR lighting in CG scenes. The panels can simulate the general direction from which a light source is coming, but it can’t angle that light to only illuminate a specific object or set of objects in the scene. You couldn’t, for example, simulate a light raking across one actor’s face without it also affecting another actor at the same time.

This is the kind of granular control DP’s and gaffers expect on a shoot. That’s not to say traditional lighting hardware can’t augment the virtual production lighting of course. Additionally, the virtual production stage lends itself to outdoor scenes. Exteriors favor the sort of ambient lighting the virtual production system excels at, so directional lighting control may not be as important for these shots as it would be for interiors. This isn’t so different from the way many outdoor scenes are shot now, with bounce cards and reflector being used to control lighting rather than high-powered practical lighting.

Baking for speed

Obviously, an essential component in the adoption of virtual production is the realism of the computer-generated scenery. There are three main ways to produce realistic background scenes in CG: baking, Unreal Engine’s dynamic lighting system, and real-time raytracing.

In baking, lighting and shadow information is rendered at offline speeds, then applied as a texture map to the scene geometry. Think of it like shrink-wrapping artwork onto a car, only in this case the artwork you’re shrink-wrapping includes shadows and shading detail. And instead of wrapping a car, you’re potentially shrink-wrapping an entire landscape.

The benefit of this method is that you can take as long as you like to render the imagery, because the render doesn’t need to happen in real-time. The real-time renderer then has very little heavy lifting to do, since most of the lighting is already calculated and it’s simply wallpapering the scene geometry with the pre-rendered images.

The downside of this method is that it locks down your set. As soon as you move a background set piece, the lighting and shadows need to be recalculated. It’s also unable to account for specular highlights and reflections. These change with the camera’s POV at any given time, and so need to be calculated in real-time rather than baked into the scene geometry’s textures.

Baking, then, can work for situations where the background set design is locked down and you’re not dealing with highly reflective objects, like metallics.

The next alternativeUnreal Engine’s Dynamic Lighting systemuses the traditional “cheats” animators have been using for years to simulate the results of bounced light without actually tracing individual rays. Using shadow maps and techniques like Screen Space Ambient Occlusion (SSAO), real-time changes to lighting can look quite realistic, again depending on the subject matter. The more subtle the shading and scattering properties of background scene surfaces, the harder the resulting real-time render is to sell to an audience.

What we really need is real-time raytracing.

RTX in its infancy

Raytracing is—as the name suggests—the process of tracing virtual rays of light as they bounce around the scene. Due to the way microfaceted surfaces scatter light rays, it requires a crazy amount of calculation work by the computer. Nvidia announced last year a new generation of graphics cards with silicon dedicated to real-time raytracing, dubbed RTX.

Right now, artists and developers are still coming to terms with how to best use the new hardware. I have no doubt that we’ll see some very clever ways of squeezing more realism out of the existing cards, even as we anticipate enhancements in future generations of the hardware.

All that to say, with today’s tech there are certain virtual environments that lend themselves to in-camera real-time production, while others would not pass the same realism test. As RTX evolves, we’ll see just about any environment viable as a virtual production set.

Too expensive for the independent filmmaker? Perhaps not.

Before you tune out and decide that this is all out of your price range, consider that a day rental for a virtual production studio may not end up being significantly more than a traditional soundstage. Obviously the market will set the price, but there’s no reason to assume that the cost of renting a virtual production stage in the near future will be stratospheric.

Even the expense of creating the environment to project can be quite reasonable. Companies like Quixel allow access to a massive library of scanned objects and backgrounds for just a few hundred dollars a year. Putting together a set can be as simple as designing a custom video game level.

And if you don’t want to create your own set? Stick on a VR headset and do some virtual scouting of predesigned environments, then simply pick the one you want to license for your shoot.

Even more affordable “in-viewfinder” solutions will soon be available for everyday use. These systems are more akin to the early virtual production systems used by Peter Jackson and James Cameron, but they will allow filmmakers to see a representation of the final composited shot through a monitor or viewfinder as they shoot.

Virtual production in your hands today, thanks to Epic’s free Virtual Camera Plugin

Is there any way to get your hands on virtual camera technology today? For free? Well, if you’re interested in virtual production for previsualization, the answer is yes. Last year Epic released a Virtual Camera Plugin for their Unreal Engine editor. You can download both the editor and plugin for free from their website.

With the plugin and an iPad (and a computer running Unreal Engine) you can physically dolly and rotate a virtual camera to experiment with framings. This is a great way to previs upcoming shoots.

Take moviola.com’s course on Previs using Unreal EngineCheck out moviola.com’s free series on Previs using Unreal to see how you could quickly build an entire replica of your shooting location, and then use the Virtual Camera Plugin to block out your shoot.

For more information on the Virtual Camera Plugin, check out this guide by Epic.

Virtual production is the future

Virtual sets have already become a major part of filmmaking. And not just for your Game of Thrones-style fantasies; plenty of primetime dramas rely on CG backdrops to enlarge environments far beyond what their budgets would permit to shoot in camera.

Bringing the virtual elements onto the live action set is the next logical step. Ultimately, it’s poised to democratize the use of computer-generated environments the way digital cameras and affordable NLE’s have democratized the rest of the filmmaking process. If nothing else, this is definitely a space to watch.

Categories
moviola.com Post Production

Blender 2.8: A Maya killer for free?

Last week we took a brief look at the news that Autodesk had released an affordable Indie version of Maya, and that the Blender Foundation had simultaneously released the long-awaited version 2.8. Regarding that last one: What exactly is Blender, do you care, and is it set to take over from Autodesk Maya as the industry reigning champion of 3D?

There’s been a lot of smack talk on the forums of late about how Blender 2.8 is about to drive Autodesk Maya out of the DCC market. Much of it is fueled by anger at the Autodesk move to subscription-only licensing, less of it by a careful comparison of the merits of each application.

Blender is to Maya as Linux is to Windows

Let’s get this out of the way up front: I don’t think Maya is remotely in danger of being supplanted by the new, improved, “free’er-than-ever” Blender. It’s not even a contest. Maya is a tool that is deeply integrated into the pipelines of major studios across the planet. Blender 3D is a very cool tool championed by indie animation studios and passionate hobbyists.Studios pay Autodesk support contracts to solve issues that arise during production. In fact, large studios like Blue Sky actually serve as in-production beta testers for Maya. Both parties have a lot riding on such collaborations. In contrast, the Blender Foundation does what it can with the funds provided. In theory they could provide higher tier premium support contracts for big studios, but that would violate the egalitarian spirit of the entire enterprise.

A passionate, altruistic computer scientist may well produce a Blender feature far superior to something commissioned internally by the product development team at Autodesk. But when it fails to behave in production, who at the Blender Foundation can guarantee that the problem will be solved in a timely manner? Autodesk, on the other hand, is contractually obligated to either fix a problem with the software, or help the user to bend their workflow to the idiosyncrasies of the software. Now you can argue about the speed of response, but with Blender there’s no guarantee that your issues will be addressed, no matter how well-meaning the team working on the technology.

I remember in the nineties Linux fanatics being, well, fanatical about Linux, and how it would soon take over the desktop market from Windows. It was free after all. History has proven that while Linux has come a long way, it hasn’t replaced Windows or OS X on the average computer desktop. (Mobile and embedded devices are, of course, an entirely different matter.)

The same is true of Blender. It’s an amazing application, but the very open-source nature of the project also counts against it for corporate adoption. The real question should be: Does it matter to you?

Blender 2.8: It kinda’ has everything

What is quite remarkable about this latest incarnation of Blender is that it really does check most of the boxes for a professional 3D tool, and then some. There’s a physically-based, GPU-accelerated renderer (Cycles), a high-quality look dev real-time renderer (Eevee), character animation tools, 2D animation tools, dynamics, cloth simulation, fluids, pyro effects, camera tracking, node-based compositing, paint tools, texturing, scripting, and even a non-linear editor. And then there are the additional plug-ins created by non-canonical developers: tree generators, human character generators, custom physics tools, even crowd simulation systems. And most of the features go far beyond what should reasonably be considered a free tool. In fact, there are many artists working with commercial software that are taking a hard look at abandoning their native simulation tools for the ones in Blender.

So getting back to the question: does it matter to you that Blender is unlikely to be the backbone of Pixar’s next mega-release? Are you personally going to be creating a major animated feature in the next two to three years? If not, chances are that the wealth of features shipping in Blender 2.8 will be more than enough to cover even your most ambitious projects.

Now if you’re dedicated to pursuing a career in animation, you may want to look a little closer at professional tools like Maya, Cinema 4D, or 3D Studio Max. But if it’s just you, or you and a small team, why not save money on the software licenses and spend it on GPUs to get your masterpieces to render faster?

It’s all the same Siggraph paper

There’s an old joke in the industry when several companies add the same new filter or effect to their software: “It’s all the same Siggraph paper.” Every year computer graphics researchers at universities and corporate R&D groups release papers outlining new techniques for doing everything from optical flow video tricks to a better raytracing recipe. And very often software companies will take the algorithms in those papers and implement them in their software.

So it’s very possible that the high-end features in the Blender toolset are just as sophisticated as the comparable offerings from commercial animation packages. After all, they’re probably implementing the same (or similar) algorithms. Now how they expose the tool to the end user in terms of UI and UX is another matter.

If you’re not already invested, it’s time to blend

If you’re already paying for 3D animation software, you most likely have more than money invested in it. You have muscle memory and a working knowledge of the interface. So ‘free’ is relative if it means investing work hours in learning a new application. For everyone else, now is a great time to learn Blender.

These days creating pretty 3D animation is much easier than it used to be. After watching the free, twenty minute tutorial from moviola.com (below) you should have more than enough knowledge to put together an infographic or simple event recreation, perfect for generating B-roll to fill a slot where you lack live action footage.

To do or not to undo

I can’t in good conscience conclude this article without mentioning what I see as the Achille’s heel of Blender: Undo. For most tasks undo works just great. But if you start to build bigger scenes with lots of objects, it starts to become cripplingly slow. My understanding is that Blender actually caches the entire scene file into memory as an undo state, so undoing is effectively reloading a previous version of the scene from RAM. (Feel free to correct me in the comments if I’m wrong about this.)

Now if you’re doing fairly simple animations, you’ll most likely never notice this. If you’re trying to create an animated short with backdrops and the like, you almost certainly will. And we’re not talking 8-10 seconds delay, we’re talking potentially a minute or more.

“I’m sure they’ll fix that soon, now they have all that extra funding,” I had someone comment to me on the Siggraph show floor this year. Really? As someone who has personally made the mistake of trying to implement an undo system after the fact, let me assure you that the undertaking is not trivial. It’s painful and mind-bending. And given the sheer size of the Blender codebase and feature set, well…I don’t envy the poor developers assigned to that task. I won’t place bets on it, but I can’t see the undo system being magically fixed any time soon.

Learn Blender in 30 minutes or it’s free

OK, so it’s all free anyway; both the software and the training in this case. If you want to ramp up as quickly as possible on Blender, check out the moviola.com Survival Guide below. Once you’ve watched the guide, you should have the fundamentals down and you can then mine YouTube for the techniques that are needed for your specific project.

Categories
Post Production

2019: The year of affordable 3D

With the announcement of Maya Indie and the release of user-friendly Blender 2.8, the democratization of high-end 3D animation has finally arrived.

Back in the nineties, an aspiring animator could expect to pay upwards of $80,000 USD for a software license of their favorite 3D animation package. Over the last two decades the prices have dropped radically, but paying over $2,000 for an annual Autodesk suite rental, or even a little over $700 for an annual rental of Cinema4D, has kept quality 3D animation out of the grasp of most casual users. With the announcements at this week’s Siggraph, that all changed.

Maya Indie goes for $199 USD per annum, and Blender 2.8 is, well, completely free.

What use does an editor have for 3D animation? Beyond the obvious titles, logo animations, motion graphics, and visual effects, there are simpler B-roll applications like animated graphs, infographics, client product mock-ups, simulations, incident recreation animatics…the list goes on. Even rudimentary 3D skills can translate into life-saving “screen filler” moments when there’s no footage to communicate what a voiceover is articulating. With that said, just how affordable and easy is it to get into 3D in 2019?

Maya Indie: An affordable onramp to the high end

Like it or not, Maya is the undisputed industry-standard in the film animation world. Its customizable user interface and SDK have meant that studios the world over have integrated it deeply into their pipelines. And this week Autodesk just announce indie pricing for Maya of $199 USD per year, an affordable price point by any standards. What are the eligibility requirements?

At first look they seem extremely generous: the site states an annual gross company revenue cap of $100,000 “from design work.” We’re seeking clarity about that statement from Autodesk and will update the article with confirmation shortly. It sounds though, like a freelancer who primarily edits for a living would still be eligible for the license if their gross revenue exceeds $100K, if “design work” only accounts for a small fraction of their gross revenue. Again, we’ll confirm shortly. We receive clarification from Autodesk: the $100,000 cap is for the entire company’s revenue, not just the portion attributable to design work. Still a good deal for freelancers and smaller studios starting out.

This isn’t the only big news on the Maya front. In fact, it isn’t even the headline as far as Autodesk is concerned. They’ve also announced a major new simulation systemBifrostthat allows artists to create convincing fire, explosions, snow, and other dynamic effects from professionally-designed presets. The system is a node-based procedural graph, similar to the architecture of SideFX’s Houdini, the current reigning champ for dynamics sims in the motion picture industry.

Maya has actually become easier to learn in recent years as Autodesk has invested effort in redesigning workflows and UI. There has also been a significant overhaul of its efficiency, with recent versions dramatically improving caching and memory handling and loading of large scenes. The Autodesk team has worked closely with heavy adopters like Blue Sky Studios to ensure the entire native toolset scales well to feature film level projects.

If you’re skeptical about your chances of learning enough of Maya to be useful, check out moviola.com’s 30 minute survival guide. You’ll be surprised just how quickly you can get your head around the basics.

Blender 2.8: World class animation for free

Blender is the Linux of the animation world: open source (free), highly capable, and passionately championed by fanatical fanboys (and fangirls). What makes the announcement of Blender 2.8 so significant is that there is a real sense that Blender has come of age as a genuine, production-worthy tool. A great deal of effort has been put into making Blender a more intuitive user experience, and tools like the Cycles progressive-refinement renderer and the Eevee look-dev preview renderer, make Blender a tool capable of producing animation on par with what you’d see coming out of Maya.

The other big news for Blender is industry backing: Epic Games recently announced a $1.2 million USD grant to the Blender foundation, and Ubisoft has also made an annual, albeit smaller, funding commitment. It’s likely that we’ll see other industry bodies follow suit. This means real development funds going into maintaining and advancing the features of Blender, ensuring a solid longevity.

We’ll take a deep dive into Blender next week, including a quick start survival guide from moviola.com. For now, though, know that Blender removes any argument that price is a barrier to getting into 3D animation. Free is free.

Choosing a package

Learning two different 3D animation packages is a little like learning to fly planes and helicopters: they both move through the sky, but they do it in such profoundly different ways the knowledge of one doesn’t directly translate to the other. For most people then, you’ll want to choose a software platform and stick to it.

If you’re happy to work in a silo and produce animation as a completed product for others to consume, Blender is a great choice. If you plan to collaborate with the broader industry you might want to choose Maya. Ultimately, Maya has the pressure of being used by major studios with expensive deadlines, so of the two it’s more likely to have the production roadworthiness for mission-critical tasks.

The real message here: you really can’t go wrong choosing either package. At $199, Maya Indie is an incredible deal. At $0.00 Blender is just a ridiculous amount of high tech power.

Update 8-1-19

Amended the article based on feedback from Autodesk clarifying that the $100,000 cap for Maya Indie includes all revenue a company receives, not just revenue from design work.

Categories
Post Production

Unboxing the tech of the new Mac Pro

If you’re secretly confused by the techno jargon being used to market the new Mac Pro, you’re not alone. To many, the real question is: Which users will benefit from the kind of futuristic hardware this new Mac will offer? Read on as we set out to explain and demystify FPGAs, graphics APIs, RAM bandwidth, and more.

The new Mac Pro carries a base price of around $6,000 USD, and by one estimate the top-of-the-line model might set you back $45,000 USD (without the monitor). For any size of studio that represents a significant investment. So, will it be worth it for you? Will the improved speed/efficiency/workflow improvements/”desktop sex appeal” create an ROI that justifies the expense? And are there more affordable alternatives that hit the same goals hardware-wise (like the iMac Pro, or—dare I say it—a Windows workstation)?

I’ll tackle the issue by breaking down the various components of the new Mac Pro, explaining the potential benefits of each, and what kind of performance gains you can expect from them. While there are no new Mac Pros out in the wild to benchmark, with the exception of the Afterburner FPGA, the tech is all a variation on existing hardware. In other words, we can easily extrapolate the performance and benefits of each component from current market offerings.

Starting with the CPU: there can be only one

The new Mac Pro boasts as many as 28 cores in the form of an Intel Xeon processor. Of course, that’s the top model config; purchasing that CPU on its own (without a computer) would set you back over $10,000 USD. I’m estimating it’ll add another $8,500 or so on to the $6,000 base price.

One surprising design choice here: a single CPU socket. Most high-end PC workstations accommodate dual CPUs, allowing consumers to get core counts up without paying the premium associated with a single chip design. (This may not be economical at initial release, but a couple of years into ownership it can make for a very affordable and powerful upgrade to a system.)

There are three important metrics in a modern CPU: base clock, turbo boost clock, and core count. The base clock is the normal speed in GHz the CPU will run. The turbo speed is the highest speed the processer can operate at when intensive calculations are required.

For digital content creators, not only are we constantly doing tasks that require turbo speed, but those tasks tend to run for a long time (decoding multiple streams of video, rendering 3D animation , etc.) The problem with this is that the CPU quickly heats up with the higher clock speed.
Without adequate cooling, the CPU is forced to throttle the speed back down to prevent overheating. Apple boasts significant cooling in the design of the new Mac Pro, but it’s not clear how long the Mac will be able to sustain turbo clock speeds before reducing to its nominal rate. Further, we don’t know how many cores will accelerate to this higher clock speed. The highest turbo clock quoted (4.4GHz on the 28 core top-of-the-line CPU) is for acceleration of a single core, not all 28.

Then comes core count. Each core is essentially a separate CPU, capable of executing a bunch of commands in a software program. By dividing tasks between cores, software developers can execute processes in parallel.

So 28 cores means 28x the processing power of a single core, right? Well, not really. Multithreaded programming (programming to use multiple “threads”, or processes) is really quite tricky. It only works for certain tasks; many processes are by nature serial, with each step requiring data from the previous step before being executed. And even when tasks lend themselves to parallel processing (like crunching all the pixels in a big image) writing bug-free multithreaded code is much more difficult to develop and troubleshoot than single thread code. Processes accessing and altering the same memory space cause all kinds of problems.

The most recent updates to C++ (the primary programming language used by high-end software developers) automate a lot of the tricky stuff with multithreaded programming, but it’ll be several years before common development platforms and libraries are updated to support it. In summary: for many software tasks your extra cores may be spending most of their time sitting around just warming your office.

“The sweet spot for general DCC work is probably the 12 core configuration”

What it means to you

The first take away: don’t assume more money spent means better performance. It depends not only on the software you’re running, but also the tasks you’re doing. A perfect example is a 3D animation package like Maya. If you spend most of your time rendering using a CPU-based renderer like Arnold, you’ll want all the cores you can get. But if the majority of your time is spent modeling or animating, then you’ll probably find that a lot of the action is happening on a single core.

Looking at the specs for the Mac Pro CPUs, the sweet spot for general DCC work is probably the 12 core configuration. It offers the second-highest base clock at 3.3 GHz and the maximum Turbo boost. So for the many interactive processes that only utilize a single core it will give you maximum punch, but you still get 12 cores for multithreading.

In contrast, the 28 core CPU will drop you down to 2.5GHz base clock in sustained, high-heat conditions—significantly lower than the 3.3GHz of the 12 core. So if you’re doing a lot of general interactive tasks like dragging layers around in After Effects or Photoshop, or previewing and trimming clips in an NLE, you may find that the flagship 28 core is significantly slower than the 12 core (which is much, much cheaper).

Now if you’re crunching an algorithm through MATLAB to solve world hunger, you’ll be squeezing every last ounce of performance out of your 28 cores. The moral to the story? Do a little research online with the apps you most commonly use and find out whether they favor single core operations or multithreaded operations. I’ll wager that you’ll be surprised by how many prefer a faster single-core clock speed over parallel processing.

Moving on to RAM

The new Mac Pros boast a staggering maximum of 1.5TB of RAM. For that you’ll need either the 24 or 28 core model; it seems the other configs max out at 768GB—still not an inconsiderable number. This is expensive ECC (Error-correcting code) RAM typically found in servers. While the price of ECC memory has come down in recent years, that 1.5TB of RAM will still set you back over $16,000 USD at today’s prices.

There was one trick we all used to count on when purchasing a new Mac: buy the minimum RAM from Apple, the replace it with third party RAM purchased at a fraction of the price. This is probably possible with the new Mac Pros, but it remains to be seen. It’s also uncertain whether Apple will allow the Mac Pro to run with cheaper non-ECC RAM. There have been many studies that indicate that ECC is pretty much unnecessary in a workstation, so purchasing its cheaper sibling would be a helpful saving.

Also of note: the 8 core base model uses RAM with a speed of 2666MHz, while the higher core models clock their RAM at 2933MHz. (Read on to see if that matters to you.)

What it means to you

Given that the computer you’re using today may max out at 32GB, you may be wondering what you could possibly use 1.5 Terabytes of RAM for. In fact, for digital video, more RAM is a pretty big deal. Take for example a frame of 8K video with alpha channel (e.g. ProRes 4444). To process the data in 32 bit, floating point precision requires 7680 x 4320 x 4 x 32 bits of data. That works out to 531 MB. That’s just a single frame. On a 32 GB system, with headroom for the OS and your editing/compositing software you could maybe fit 40 frames in memory. Start blending layers of video together and doing motion effects and your system quickly comes to a spluttering, choppy stop.

“You’ll almost certainly feel the difference between say, 32GB and 128GB, even with a modest HD project.”

It’s not just about real-time performance either. Most applications cache different image states into RAM, pushing them to the hard drive when RAM fills up. The more RAM you have, the more responsive your NLE or node-based compositor will be when you tweak a filter setting or perform color corrections. 1.5 TB might be overkill, but you’ll almost certainly feel the difference between say, 32GB and 128GB, even with a modest HD project.

Oh, and before you object that you’re working with compressed media, know that in most cases the software decompresses your media and stores it in RAM as an uncompressed frame for image processing operations. So while your hard drive storage needs may be smaller with an efficient codec, your RAM needs don’t necessarily follow.

One question often comes up: on a budget, should you buy more RAM of a slower speed that’s more affordable, or less of the faster, expensive variety? The answer is almost always, “Buy more RAM”. The performance differential between the 2666MHz of the base Mac Pro model and the faster 2933MHz of the higher core models will probably amount to 1-2%. In comparison, buying a larger amount of RAM may see significant boost in overall performance (assuming the higher end models will take 2666MHz RAM as well, which they should).

Onto the GPUs: Where the action’s at

Where the Xeon CPUs tap out at twenty-eight cores, the cores on a modern graphics card (GPU) number in the thousands. Rather than use them purely to make pretty graphics, modern developers use them to accelerate massively parallel tasks—everything from deep learning processes to real-time ray tracing. They’re especially well-suited to crunching lots of pixels all at once.

Up until around 2009, advances in general computing were all about the CPU. True to Moore’s law, processor speeds doubled every 18 months or so. Around 2009-2010, a lot of the action shifted to the GPU. It’s the reason you can buy an old workstation from 2010 (Mac Pro tower or Windows Z800 for example), throw a modern graphics card into it, and end up with a usable system.

The new Mac Pro can be maxed out with the equivalent of four AMD graphics cards (two dual card boards, taking up a total of two double-wide PCI Express slots). They also feature the fantastical-sounding “Infinity Fabric Link connection.” It’s a way for the graphics cards to share resources, allowing them to behave more like a single unit with shared storage. Nvidia has a competing technology called NVLink, so nothing particularly startling here; nonetheless it’s an important inclusion.

Apple boasts 28.4 teraflops of floating point processing power in the top model graphics config (at a potential add-on price point of over $12,000). What’s a teraflop? It’s a trillion floating point calculations per second. Before you get too carried away comparing teraflops, there are many factors that go into final performance. Just as having 28 cores on your CPU doesn’t mean your computer will be constantly using all 28 cores, the final efficiency of a graphics card depends on how the software uses it. It’s unlikely that all your graphics cores will be “teraflopping” all the time.

Wait, no Nvidia?

I’m not entirely sure what Nvidia did to make Apple dislike them so much. I’ve heard rumors, but none that I feel confident enough to perpetuate. Apple engineers have confirmed that there’s nothing stopping Nvidia cards from running on the new machines. Nothing that is, except the small matter of Nvidia writing drivers for Apple’s Metal (an API technically in competition with their CUDA library), and Apple approving said drivers. Why is this a big deal? Read on.

Metal, CUDA, and OpenCL

You may have heard of OpenGL, but possibly not OpenCL. OpenCL is an API (Application Program Interface—basically a software library for developers) championed by AMD that allows everyday developers to tap into the power of graphics cards for non-graphics related work. The API is a “black box” that hands off massive parallel processing to the graphics card so that the developer doesn’t need to know the tricky technical details of how it’s achieved.

CUDA is Nvidia’s proprietary graphics API. It only works with Nvidia cards, unlike OpenCL which can technically work across any platform (it’ll even fall back to run on your CPU, albeit at a much slower pace).

Then there’s Metal. True to Apple’s history, their development team went back to the drawing board and came up with their own graphics API to work with Apple products (both OS X and iOS hardware platforms). Metal replaces both OpenCL and OpenGL. i.e. it handles both computational and graphics acceleration. The argument goes (and this has been repeated many times in Apple’s design philosophy), “We design the hardware, so we know the best way to design software to interface with it.” It’s a reasonable argument, and I’m not aware of anyone who actually thinks Metal is a bad implementation.

OK, so first off: OpenCL is open-source, runs anywhere, and is backed by a major hardware developer. What’s not to love? Well, it turns out most developers prefer CUDA, partly due to the way the API is written, and partly due to the fact that historically developers have been able to squeeze better performance out of Nvidia cards running CUDA than competing offerings.

Turning our attention back to Metal: Apple have already deprecated OpenCL and OpenGL on the Mac, and the latest OS—Mojave—doesn’t support CUDA. In other words, if you want graphics acceleration on a Mac your only option is Metal.

What it means to you

If you’re an editor you’re probably already OK on this front. Companies like Blackmagic Design and Adobe have been optimizing their code base for Metal acceleration for some time (although last time I checked there were a few pieces lagging on the Adobe side; perhaps that’s changed with Mojave).

If you do more exotic things like GPU rendering or machine learning/deep learning, the path forward is not so clear. Regarding GPU accelerated rendering: It was interesting that Apple’s press release paraded both Redshift and OTOY as soon-adopters of Metal. Both 3D renderers have been exclusively Nvidia and CUDA since their inception. Redshift (now part of Cinema4D’s parent company Maxon) claim to be shipping a Metal solution by end of year. OTOY are equally enthusiastic, although they’re known for overambitious release goals. It remains to be seen if the price/performance ratio works out with Metal versions of these renderers ($12,000+ is a lot to spend on graphics processing power).

Unless you’ve been living under a rock, you know that Machine Learning and its sub-discipline of Deep Learning are the hottest areas in computer science right now. They are already radically changing the landscape of just about every market on the planet. We’re only just beginning to see the transformation in the media and entertainment sector. Well, if you’re a machine learning computer scientist, you probably won’t be buying a new Mac Pro.

Why? Because the vast majority of software machine learning practitioners depend on—at least for prototyping—are CUDA (and thus Nvidia) dependent. TensorFlow, Keras, Caffe, PyTorch, and MXNet all lean heavily on CUDA to crunch the math. While Apple has developed its own library of Metal-accelerated machine learning functions, don’t expect the heavily academic community of machine learning experts to suddenly migrate to a new set of tools based solely around the Mac.

Beyond these considerations, be sure to review benchmarks for your favorite applications to see how GPU-intensive they are, then budget the graphics config on your new Mac accordingly.

Finally, the dark horse in the race: Afterburner

For video professionals, Afterburner is where the Mac Pro gets really interesting. It’s a board dedicated to accelerating the processing of video. Confused as to what that is? Evidently so is Apple’s marketing department.

The product page on Apple’s site calls it “an FPGA, or programmable ASIC.” Well, to get a little pedantic here, ASICs and FPGAs are two different things. An Application-Specific Integrated Circuit (ASIC) is a computer chip that has been architected and “cooked to silicone” for a specific processing task. It’s when very smart engineers design their own custom chip to be etched onto a silicone wafer. Unlike a CPU, which is designed to handle whatever processing job is thrown at it, an ASIC is designed to perform very specific operations like, say, crunching video files.

In contrast, a Field-Programmable Gate Array (FPGA) is a collection of circuits that can be effectively “rewired” after manufacture (in the “field”) to operate in a specific way. This is more than just burning software code into a chip’s permanent memory (like an EPROM). FPGAs are electronically different once they’ve been programmed.

So an ASIC is designed and cooked during manufacture; an FPGA is a more generic chip that is “rewired” to more efficiently handle a specific software task. FPGAs are often used to prototype ASIC’s (the latter are far more expensive to manufacture), but they are almost certainly not the same thing. For the relatively small number of people purchasing an Afterburner card for their Mac Pro, the expense of designing an ASIC couldn’t be justified, especially when advances in GPUs and CPUs will probably make it obsolete and needing a redesign in a couple of years. (Just ask the poor guys at Media 100, who bet against Moore’s Law with their Pegasus architecture.)

If anything, the fact that the Afterburner is an FPGA is somewhat more interesting, since it opens the possibility of reprogramming it for other tasks. It’s doubtful that Apple would ever provide that function however, nor is it known whether the interface on the board allows for reprogramming once it leaves the factory.

Marketing semantics aside, the Afterburner may actually be kind of awesome. It claims “up to 3 streams of 8K ProRes RAW at 30fps”, or “up to 12 streams of 4K ProRes RAW at 30fps.” Of course, that assumes you have RAID storage that can serve the data at those speeds (that’s another massive expense to consider), but the processing power is impressive. How many software manufacturers come on board to support it remains to be seen, but you can expect the major players to be ready (or close to) by the time it ships.

(As a total aside, does anyone else think “Afterburner” an odd choice of name for a real-time device? Isn’t it more of a Duringburner?)

What it means to you

This is one of those, “If you don’t know you need it, then you probably don’t” kind of things. The price of this card is a complete unknown at this point, but I’ll wager that for studios doing 8K online, price will not be a significant part of the purchasing decision.

If you’re more on the indie side of things (offline editing, or editing with compressed formats rather than RAW) it’s probably a luxury you can forego for now. That said, I wouldn’t be surprised if companies like Blackmagic Design and AJA follow suit with their own third-party offerings; FPGA’s are definitely in their wheelhouse. We can only hope Apple’s Afterburner stimulates more activity in the custom hardware arena for video work.

The big picture: pass or play?

OK then, is the new Mac Pro worth your coin? The entry level system sits at $6,000 without a monitor—that’s already a major chunk out of most boutiques’ annual hardware budget. But then, that base system will possibly only prove useful to someone using it for moderate graphic design, post audio, or music creation work. For video editing, mograph, VFX, and 3D work, start piling on the expensive options.

If you’re doing client-driven sessions like color grading or other finishing tasks, this is probably a reasonable investment, both in terms of the billing rate for that kind of work, and “sex appeal” to the client. Those are environments where real-time responsiveness is crucial, and latency legitimately translates to frustration from clients who pay by the hour.

For the more “offline” crowd…let’s weigh the alternative Apple options first. I must stress that I’m moving very much into editorial writing mode here, so take the comments for what they are: my opinionated musings.

New Mac Pro vs iMac Pro

For my money, the Mac Pro wins here. Until the iMac Pro sees a refresh (which may well be imminent) you’re paying for 2017’s hardware if you buy an iMac Pro today. The base model is $1000 less than the Mac Pro base model and includes a display, but you’re then stuck with a non-upgradeable system that already has two years on the clock in terms of its tech.

The Mac Pro is slightly faster, but more importantly, upgradeable. You can add RAM later and—perhaps most significantly—replace an aging graphics card for a new lease on life a couple of years down the road.

And you may of course already have a monitor. Regardless for a lot of editing, mograph, VFX, and 3D tasks you can buy an adequate 4K monitor for under $400. No, you won’t be grading Scorsese’s next film on that monitor, but it’ll serve. It certainly won’t be as pretty as the iMac Pro’s display, but again: it’s replaceable later.

Now if Apple comes out with an iMac Pro refresh before you go to purchase a system, that may be a whole different story.

New Mac Pro vs Trashcan Mac Pro

Get the new one. No contest. Unless someone is selling one for a few hundred bucks…The ancient CPU and graphics cards, limited expandability, and overall “blahness” of the Trashcan Mac Pros makes them hard to recommend going forward.

New Mac Pro vs Westmere Mac Pro tower

Apple-loving post professionals have been clinging onto their Westmere Mac Pros for dear life. It may be finally time to pry their fingers from the skin-lacerating cases.
It’s often been said that these systems still hold their own, especially if you up the RAM and throw in a new GPU. Alas, it’s hard to still make that claim here in mid-2019. The latest flavor of OS X—Mojave—no longer supports the older Mac Pros (even though some users are determined to work around the limitations). Hey, we’re talking about technology from 2010-2012. You had a good run: Time to say your goodbyes and turn it into a Minecraft machine for the kids.

Heresy time: New Mac Pro vs a Windows workstation

Before you hate, let me just say that I’m a big fan of Apple, right down to my iPhone, my Apple Watch, and my $25 Chinese AirPod knock-offs. (Yes, they sound as bad in real-life as in print. No, I’ll never buy them again.)

While I still use Macs in my studio (to be honest, mainly now for Logic Pro) my workhorse machines these days are all PCs. I’ve learned to live with Windows’ quirks, and Windows 10 has copied enough of OS X’s user experience over the years that it actually isn’t a bad place to do business. I still run into the occasional QuickTime-related frustration, but those are becoming fewer and fewer.

SO…if someone handed me a new, $30,000 Mac Pro would I take it? Heck yeah. (If you’re offering, please contact Pro Video Coalition’s editorial staff immediately). And I would thoroughly enjoy using it. But if I had to suggest an alternative?

I see two routes on the Windows side:

1. The expensive: if you want to spend $10-15,000 on a solid workstation, look at a system like the Lenovo P-series, or an HP Z-series. You should be able to deck out those systems with a ton of RAM and a couple of powerful RTX Nvidia cards before grazing $15K.

2. The surprisingly affordable: look at picking up a used workstation (like the HP Z840) with two Xeon E5-2687W v3 CPUs installed. The ‘v3’ is an essential point; previous generations with the same moniker were not nearly as powerful. You should be able to haul one in for under $2,000 ($1,500 if you shop well).

This will give you a 2015-technology machine with 20 cores running at a base clock of 3.1GHz and a max turbo of 3.5GHz. I’ve found this to be a sweet spot between single processor speed (a healthy 3.1-3.5 GHz) and multithreading (20 cores is a lot of cores).

Plan on spending at least another $1000 on a modern RTX2080 graphics card and extra RAM. But for under $3,000 you’ll have an extremely fast machine capable of outperforming the new $6,000 Mac Pro in any multithreaded situations (at least by my estimates—I have absolutely no empirical evidence to back that up. Would kind of need a new Mac Pro to compare to…)

You always run a risk when you buy used, but my personal, anecdotal experience has been that if hardware has lasted three or four years, it’ll probably sing along happily for another five or more. (Of course when your experience differs, it’s your checkbook you’re taking the gamble with, not mine). My advice: see if you can find a local refurbisher and build a walk-in relationship with them. They’re more likely to accommodate an exchange if things go wrong.

You could try to turn it into a Hackintosh, but that would be naughty and painful (just talk to anyone who’s tried to run one for more than a few months).

But look, I get it: it’s hard to put a price on the Mac desktop experience. Wait, no it’s not: it’s anywhere from $6,000 to $45,000 (plus monitor, plus $1000 monitor stand).

One final thought: dimming the lights of Metropolis


Here’s my one more thing: power. The new Mac Pro requires 1.4 kilowatts of power to run effectively. For those of you in Europe and Australia with your fancy 240 volt outlets, no biggie. For those of us living in the Americas, that means over 12 amps of power draw. With a standard outlet tripping at 20 amps, it means that if you try to power your new Mac Pro, a Pro Display (Apple is yet to publish its power requirements), and a couple of hard drives while brewing a cup in your Keurig, well…lights out baby. Might as well put the Keurig in the closet. (Seriously, get yourself a decent espresso machine and you’ll double your client base anyway.)

What it means to you

Check your available circuits before buying. You may just need to purchase a new studio to go with your new Mac…

 

Updated 6-19-19: A side note I feel compelled to add: I mention Maya as an example of an application whose CPU demands vary depending on the kinds of tasks you’re doing in it. While the principle is true, upon reflection Maya is probably a poor example these days: Since Autodesk overhauled Maya’s code base with the release of Maya 2018, overall the app is highly optimized for multithreading. There will still be plenty of single-threaded tasks (especially user-generated Python scripts), but a lot of the everyday operations like object manipulation are highly multithreaded.
So I’m heading off here any objections that might deservedly arise from the Maya dev team. Props where props are due…

Categories
Post Production

Switching to Windows: the hidden cost

As with many folk in the film industry, the absence of Nvidia options from Apple’s line of laptops forced me to look at Windows offerings. A not inconsiderable amount of research led me to the conclusion that Alienware would be the best match in terms of power (quad-core top-of-the-line Intel CPU, and full GTX 1080 graphics card). And I have to say I loved it…when it worked. And that, dear friends, is the beginning of our cautionary tale.

The only real downside I had initially with the Alienware 17 R4 was the size and weight: it’s portable in the way that a dorm room refrigerator is portable. Once I accepted that part of my daily workout regimen would now include carting the laptop to and from the car I got over the heft. The laptop is fast, includes all the modern amenities (Thunderbolt 3, and USB 3) and you can thankfully turn off the multicolored gamer light show.

As far as the operating system goes, I didn’t mind the transition at all. There are a few OS X niceties I miss in Windows, but there are a lot of Windows conveniences I now miss when I’m back on a Mac.

However, there was one crucial part of the Apple experience I had taken completely for granted: 21st century support.

When a Mac goes bad under warranty you walk it into the local Apple store and if the problem warrants repair they’ll typically hand you a replacement unit, or arrange for one to be picked up within a few days. Not so with Dell, Alienware’s parent company.

The first visit with Dell support

The great misadventure began with delivery. Upon boot up I discovered that a minion at Dell’s technical support center in Texas had managed to install the OS onto the spinning disk instead of the SSD. After alerting Dell to the problem they promptly scheduled two separate remote sessions to clone the drive to the SSD. I suggested that they could just nuke everything and start over since I hadn’t yet installed anything of value to me. They insisted that cloning was the way to go. This worked for about two weeks, then the boot manager finally gave up (actually there were somehow two boot managers installed) and refused to boot in. I had the option to void the warranty to try to recover my data or just wipe the drive and do a clean install of Windows myself. I chose the latter.

In the meantime the sound card died. Now if I were to give you advice at this point it would be: live without sound, it’s not that important. You see, that was the beginning of the end.

The technician who came to the house to service the laptop brought an entirely new mainboard. After several hours of wrestling with the disassembly and assembly and cursing as he lost part of the wifi antenna, he handed me back the laptop. After I was unable to boot it due to the aforementioned boot manager issue, he rebooted a dozen or so times and finally managed to trick the boot manager into booting. With that he declared his work there done and disappeared. (The next morning I was back to performing a dozen or so reboots to achieve the same trick.)

It wasn’t until the next day that I discovered the annoying gamer LEDs were now permanently stuck on. Oh, and there was also the fact that one of the chassis’ screws was sticking out a few millimeters. Obviously the technician had found it too hard to screw back in and finally gave up. In addition, the CPU and GPU fans seemed to kick up a lot more often and more quickly than the previous system, but that was the least of my worries.

After Dell put me through a 5 stage software diagnosis test of the lighting systems taking several days of back and forth emails another technician was sent out. After he wrestled with the system for a few hours he completed the repair. The stuck LED was still on. He left explaining that he’d completed his part of the repair. Fortunately after he left I performed the complete reinstall of Windows and the LED turned off. Finally, I had a working system. Or did I?

Parts come loose

A month or two went by with the system purring away when all of a sudden my space bar and enter keys decided to periodically stop working. To the point that in one of my support emails to Dell I had_to_use_underscores_to_actually_type_out_the_response. Time for—you guessed it—another technician visit.

Somewhere in here there’s a missing SMD resistor…

The third technician showed up, spent two hours replacing the keyboard then told me to test it. I noted that the keyboard was raised on the left and that the caps lock and num lock lights failed to illuminate. He confirmed that was indeed the case and proceeded to spend another two and a half hours re-replacing the keyboard. As he did so he discovered a surface mount resistor (presumably from the mainboard) floating around in one of the cooling fans. He took a photo of it (he called it a “fuse”—evidently he wasn’t all too familiar with the parts inside a laptop) and made note of it. The technician finished the reassembly and confirmed that the laptop keyboard was now working.

Wait, what? Yes, you heard me right: components from the motherboard were evidently floating around inside the chassis of my laptop (well, at least one).

At this point a 21st century support service would be mortified by the customer experience: three technician calls within 3 months, with the first technician causing more damage than he fixed. And now a technician is documenting components detached and floating around inside the laptop case. Time for a humble apology and a replacement laptop, right? Alas no, we’re in the land of Dell.

The adamant position of Dell’s support team was that software diagnosis needed to be performed to determine where the component may have come from, then yet another technician would be sent to replace the offending item. Now if you know anything about electronics you know that the absence of a resistor may never show up on a test, since it’s just as likely to mess with the tolerances of the circuit to which it belongs than to stop things working all together. If it’s a pull-up or pull-down resistor it might cause intermittent instability like, say, causing certain keys on the keyboard not to work…And as we all know based on historical evidence, the problem is unlikely to reveal itself until exactly one day after the warranty expires.

Descent into Dell…

Ah, but there is another option: I can send the laptop into the Texas service center (the same capable group who initially installed the OS on the wrong drive) and wait an indeterminate amount of time for it to be fixed. No loaner unit while I wait. The support manager suggested the wait times are typically 5-7 days (not counting shipping times), but again based on past experience this sounds like it could be two weeks between the laptop leaving my hands and returning to them. That’s a long time to be without a workstation laptop.

Now I get it: Dell is kind enough to provide on-site support and that’s admirable. In fact, the response time once I’d jumped through all the software test hoops was never more than a day. However, at some point when a customer has experienced the embarrassment of errors that I did, surely there’s room to make an exception and just ship a replacement unit. Evidently not: in fact the manager assured me that was never the case. (Strangely the last tech informed me that another customer had been successful in getting a replacement; I guess I didn’t get irate enough with the phone staff.)

Presumably I could have another dozen of these on-site visits until there are no chassis screws left unburred, so many SMC’s floating around in the chassis that you can hear them hitting the fan, or the warranty period runs out.

What’s the solution at this point? It looks like my machine will spend two weeks in Dell’s Texan purgatory while I pray that it returns with less problems than those with which it departed. How will I get any work done in the intervening period? Time to dust off that old Macbook Pro…

Categories
Post Production

Resolve 15: Blackmagic integrates Fusion into Resolve

Blackmagic’s grading tool turned NLE is on track to become a complete finishing platform–rivaling Autodesk’s Smoke and Flame platforms–thanks to the incorporation of the Fusion compositor directly into the application. In fact, with the enhancements to its Fairlight audio engine and collaboration tools, Resolve 15 offers a complete, closed ecosystem for the entire post workflow.

Just announced at NAB, Blackmagic’s Resolve 15 is a major feature release across the board. Most significant though is the bold move to integrate Fusion directly into the application. Up until now Fusion has been Blackmagic’s dedicated node-based visual effects and motion graphics platform used on many major feature films, high-end commercials, and motion graphics projects. Fusion still lives on as a separate application but now editors, colorists, and effects artists can access it right from the Resolve timeline.

Before we dive into the details, a quick note about the project status: Resolve 15 is most definitely a public beta version of the software. Don’t by any means think that the build you can download today is ready for heavy production work. But if you want to start getting a feel for the application in anticipation of the full release version, the download is ready and waiting on Blackmagic’s support page. Also of note: those of you who already own a dongle for Fusion Studio will be able to access the new Resolve Studio builds free of charge.

OK, so how does it all work? It’s actually one of the more elegant integrations I’ve seen. Simply park your playhead over a clip in the Resolve Edit page timeline and switch to the Fusion page. The clip or stacked clips will appear as source nodes connected to a single output node. Footage comes in sized according to the timeline resolution and already set in linear space according to your global Resolve color management settings. No need to mess with LUTs and codecs–that’s all handled by Resolve. Drag additional media into the node view via the standard Resolve Media Pool pane.

Then choose from the hundreds of Fusion tools–including sophisticated elements like the planar tracker–to build your motion graphics, visual effects and style filters. When you’re done, just make sure the last node connects to the provided output node. That’s it–no need to render. You can switch immediately to the color page and start performing additional grades or return to the edit page to resume editing. A simple timeline representation of all shots (like the one in the Color page) allows you to quickly jump between Fusion comps and create and switch between multiple versions of the same composition.

There are definitely beta software “quirks” to be found in the current build, but the biggest word of caution is hardware related. At least in the current state, the Fusion page is most stable on modern machines with lots of RAM. If you have an aging laptop or one with 8GB or less of system RAM don’t expect the Fusion page to show you too much love. No doubt as the beta process goes on and more users with modest systems try their hand we’ll see optimizations for these less endowed machines. For now though, look on the bright side: You finally have an excuse to go out and buy that sleek gray 128GB iMac Pro you’ve been dreaming of. You deserve it, and it’s for work after all.

New features across the board

While the integration of Fusion into Resolve obviously the biggest news, Blackmagic have added a solid set of new features and enhancements across the board. Let’s start with the the Fairlight audio page.

Fairlight in Resolve is now a real thing

If you weren’t around in the 70’s and 80’s you may not realize that the Fairlight company was one of the great pioneers of modern electronic music. And in more recent years their technology has been highly regarded in the post world, if not as well-known as Avid’s ProTools line.

With last year’s beta release of Resolve 14, Fairlight felt very much “bolted on” to Resolve. The Fairlight page of Resolve 15 has matured considerably and is now a place where both video and audio editors should be able to get real work done.

In addition to stability and performance improvements to the Fairlight page, Blackmagic has added a comprehensive new ADR (Automated Dialogue Replaceme`nt) tool. This is one of the better implementations of ADR I’ve seen in a broad market NLE or DAW. It includes exceptional audio and visual prompting and count-in features. The system is based on software the Fairlight company had previously released as a commercial ADR product.

ADR takes are stacked in convenient vertical layers for review. The review process is a little tedious with clips needing to be shuffled to the top for preview instead of soloed from their place in the stack. Not a deal-breaker, but somewhere to look for improvement in a future release.

Over a dozen new Fairlight plugins have been added–including post-audio essentials like ground hum killer, a de-esser, and noise reduction–in addition to the support for third party VST and audio unit plugins.

One nice new feature I wish every NLE and DAW had as an option: fixed playhead playback. In this mode, the playhead stays still and the entire set of tracks scrolls like a seismograph read-out. So much easier to read waveforms than trying to figure out what’s happening when a playhead wraps around in a conventional moving playhead system. (This is a selectable option; you can work with a moving playhead as well based on preference.)

Color improvements

Colorists still make up the majority of Resolve’s professional user base. (I’m still a little baffled by how often I hear, “What, Resolve edits too?” The free version of Resolve is now an extremely capable editor and an easy transition for Final Cut Pro 7 editors. I guess we just all love paying our Adobe Creative Cloud subscriptions…) So Colorists have been known to get up in arms over feature updates to Resolve that they’ll never use. Never fear: Resolve 15 offers plenty of improvements for the color correction crowd.

The most immediately noticeable improvement to the color tools is the new LUT browser. Like gallery presets, every LUT installed on the system is available for preview as a thumbnail.

But what makes this really powerful is the ability to hover and scrub over a given LUT and see a live preview of that LUT in the viewer. This live preview scrubbing also works for the existing Gallery system.

Speaking of the gallery, at the bottom of the gallery there’s a now a “Timelines” section that allows you to browser all the grades in the current timeline and quickly live preview and apply them to other clips. This is a great new workflow tool and could replace the need for constantly creating stills for many colorists.

A solid addition to the node view is shared nodes (aka cloned nodes). You can apply the same shared nodes to multiple grades. Tweak one instance and all others are automatically changed. This opens up a world of possibilities for controlling and adjusting primary corrections across multiple shots while allowing those shots to maintain unique secondaries.

Another nice touch is the live preview of composite modes when merging in the node graph. Scroll through the list of modes (eg. screen, soft light) and instantly see the effect in the view before committing to one.

There are many other requested features that have made their way into Resolve 15: multiple timeline playheads, marker overlays, new split screen functionality, draggable qualifier controls (finally), new matte finessing controls, node-specific color and gamma settings (allowing you to choose the color and gamma space you want to correct in per node), support for Dolby Vision and HDR10+ to name a few.

Here’s a final one that in past years could have saved me spending hundreds of dollars with Deluxe and Technicolor’s lab services: Resolve 15 includes native support for (unencrypted) DCPs and native IMF encoding and decoding. No EasyDCP license required. For some that’s worth the price of the upgrade. Wait, that’s right: Blackmagic doesn’t charge for upgrades. Will someone remind me how they make money? (Oh, the cameras and boring little converter boxes.)

Editing improvements

The editing improvements are substantial, if a little less visible than the color and audio additions. Of these, the most noticeable is the tabbed and stacked timeline system. Now you can have multiple timelines added in tabs, or stack timelines above each other in the UI to make it easy to drag media between timelines.

If your clients like to doodle, they can now draw all kinds of obnoxious notes on your clips for later review. These appear as markers you can quickly navigate to after a review session. In Resolve’s collaboration system they could also be left my remote team members to communicate desired changes etc.

Something to make users transitioning from Media Composer happy: dynamic trim has been improved to be a little bit more automatic. (Then again, there’s probably nothing that’ll make a Media Composer editor happy outside of Media Composer. You can’t win every war…)

A new text tool leveraging Fusion’s powerful albeit often cryptically esoteric Text+ tool has been added to Resolve’s editing features, along with dozens of pleasantly un-ugly preset text effects.

For those who love structure as long as it’s their own, Resolve 15 allows you to drag a series of nested folders into the Media Pool to create a project structure template.

A nice adaptive upsampling algorithm has been added to source footage properties, allowing you to scale media 2X, 3X, or 4X, great for mixing 2K media into 4K projects. Optical flow algorithms for retiming and noise reduction have also been significantly improved.

In addition to these there are a slew of other enhancements, some of which fall into the “What took you so long” category: sub-clip drag-and-drop creation, keyframeable Resolve effects, improvements to clip naming, improved timecode entry. There’s also a nice implementation of “Delete all gaps” which protects sync as it deletes.

ResolveFX Revival

For those needing to repair archival footage, Resolve 15 includes some very cool-looking tools for dust-busting, localized flicker removal, and patch replacement (i.e. a video version of Photoshop’s heal brush). I’m cautiously optimistic; these are the sorts of amazing tools that seem to work on everything except the shot that’s in front of you. Nonetheless, on the demo footage, the effect was extremely impressive.

Subtitles and Captions

A major new feature of Resolve 15 is the ability to add subtitles and captions. Titles can be imported in .SRT format and placed in a dedicated timeline track, respecting the spacing in the source file. A nice touch is that the created caption clips can be trimmed and edited using the same edit tools used for video tracks.

Subtitles can be created directly within Resolve too. When finished, they can be exported to file as SRT or WebVTT, burned into exported video, or included as embedded captions in MXF OP1A and Quicktime movies.

Performance optimizations

In addition to the previously mentioned optical flow and upsampling improvements, the Resolve engineering team have clearly put a great deal of effort into GPU acceleration. They’ve incorporated support for all three of the leading APIs: CUDA, OpenCL, and Apple’s Metal. Of these Metal is the most significant to Mac users who have suffered from Apple’s insistence on not using Nvidia, making it extremely hard for CUDA-loving artists everywhere to get an Nvidia card working on their favorite Mac. With native Metal support, Mac users should see some solid performance gains from their Radeon powered hardware.

For the punters on Linux (we know you’re out there somewhere reading this on your open-source browser) there’s now native audio support without the need to send your sound out via a Decklink card. And for those of you working with multiple monitor speakers you can now leverage the system-native audio engine and thus send audio out via different audio interfaces.

In the realm of codecs and the like, Resolve 15 adds the ability to work efficiently with temporally compressed codecs (eg. H264) without messy transcoding or the like, and support for Kakadu DCI–in addition to the previously mentioned native support for DCP and IMF.

One app to rule them all?

What makes Resolve 15 truly interesting at a philosophical level is the idea of a single app that contains all aspects of post production: Editorial (Edit page), sound (Fairlight page), motion graphics, 3D animation and visual effects (Fusion Page), color correction (Color page) and delivery (Deliver page and the new captioning toolset). Whether one operator chooses to wear all hats or a team of Resolve artists collaborate on a unified project, such an arrangement effectively eliminates what is arguably the most painful aspect of post-production–conform.

Will it work? Maybe not today–certainly not at a large-scale commercial level–but it’s a fascinating direction. Blackmagic is stressing that the implementation of Fusion here at NAB 2018 is the beginning of an 18 month(-ish) journey, and that’s probably not too unrealistic a timeframe.

There will always be limits to what can be done in one app. Case in point: The Fairlight tool has a long way to go to rival the comprehensive editorial toolset of ProTools. As Fusion evolves inside of Resolve however, it’s hard to see it not becoming a significant replacement for Smoke, if not Flame. And with a more than capable free version and the studio version maxing out at the paltry sum of $299 (there are plugin suites that cost 10 times that number) it’s a hard value proposition to pass up.

Want to know more? Try it for yourself by downloading a copy from Blackmagic’s support page. (In the “Latest Downloads” section of the page.)

Damian Allen is a VFX supervisor and pipeline consultant based in LA. He specializes in picture-lock emergency effects work and AR and VR tool development through his company Pixerati LLC. In addition to his hands-on production work, Damian lectures and trains artists around the world in compositing theory and technique.

Categories
DAM Post Production

Easy, fast and secure media file transfer–the Bigfoot of post

I have a long list of “It’s 2017, how come we haven’t figured out how to…” questions. Topping that list (alongside why Siri still can’t look up a phone number for me without mistakenly calling someone to whom I owe money instead) is the issue of transferring video files securely across the web. There typically seems to be a choice: send files easily or send them securely. The easy way is an email attachment if it’s small enough, wetransfer.com if it’s a little bigger, or a thunderbolt drive couriered across town if it’s colossal. The secure way is an expensive, industrial service that typically requires the installation of software on your client’s machine whether they like it or not. So taking all that into account I was curious to try out Signiant’s offering, Media Shuttle. This is a service specifically geared toward ease-of-use and is perfect for small to mid-sized companies (i.e. boutique post houses). It promises the simplicity that busy people demand without compromising security. Let’s take a look.

You can hold my hand, but don’t waste my time

I’m not a big fan of user guides or guided tours. I’ll waste a couple of hours trying to figure things out by myself before asking for help. (In other words, I’m a typical male.)

Having said that, I was pleasantly surprised by the screenshare introduction to Media Shuttle. Maybe I got lucky with the session being hosted by Signiant’s Tyler Cummings, but within about 20 minutes of my time (thirty counting my obnoxious questions) Tyler had pretty much shown me the entire system from start to finish.

So what exactly is Media Shuttle and how does it work? There are three main ways to move files across the internet using Media Shuttle: “Send”, “Share”, and “Submit” (the first two are for sending, the latter is for receiving to your server).

The Send Portal

The Send Portal option is a very simple email web page–think wetransfer.com. Type in a recipient’s email address, drag the files you want to send to the browser, and click send.

Now when I make the comparison to wetransfer.com it’s a pretty shallow one; there’s a whole lot more going on here than your standard WeTransfer session.

First of all, there’s no file size limit. That’s a bigger deal than it seems at first. Even with open source, self-hosted systems like ownCloud large transfers frequently fail to deliver successfully.

Secondly, you can send entire folders and subfolders without zipping them first. This is huge if–like me–you’re constantly dealing with massive EXR and DPX image sequences. A multi-layered EXR can easily weigh in at over 100MB per frame; trying to zip those suckers up first (if you actually succeed) can take longer than sending the actual files.

Media Shuttle actually uses its own proprietary transfer protocol optimized for large file sizes rather than the office documents that generic transfer services are designed for. We’ll talk a little more on that later, but in essence it means that your transfer speeds won’t suffer from being uncompressed. (In fact, media files rarely benefit from being zipped–they’re already compressed at the file level.)

Next, you can send with whatever level of security required. Is the content extremely sensitive? You can require that the recipient logs in securely to the Media Shuttle portal before downloading the files. For additional protection, you can require a user-generated password for download. Or do you have a technology-challenged client that wants a straight download without the hassle? Assuming the content isn’t too sensitive, you can make set up for the download to be a single click from the email.

Another nice touch is the ability to customize the look of each portal page. This can make your one or two man shop feel a lot bigger and slicker than it really is. (After all, the client never needs to know that you finished the conceptual art for their campaign in your bathroom on a laptop.) Unique pages can be created for different clients or projects, at no additional cost.

The Share Portal

Now I’ve worked on small campaigns and big Hollywood blockbusters, and I’m amazed at how often proprietary information just gets thrown on an unsecured FTP server for me to download. This is especially true in the last two weeks before picture lock on a movie; studio execs with little tech savvy will find the path of least resistance when they need to get files to a post house during crisis hour. Security goes out the window and production staff find FTP servers an easy way to give vendors instant access to a set of files.

Media Shuttle offers a no-nonsense “FTP-like” interface to entire folder structures, so a client can browse and choose the files and folders they want to download. But it’s not FTP–it uses 256-bit AES encryption system, the security scheme recommended by the MPAA.

Security is, of course, paramount, and FTP is an inherently insecure way to share files. SFTP is better, but typically frustrates non-savvy users who have to configure an FTP client to the correct protocol.

As the host of the files, you’re completely in control of which folders a given client can have access to. Permissions are easily configured from the portal management page. On numerous occasions I’ve been frustrated trying to set permissions on SFTP servers only to end up either locking out the client or showing them too much. The Media Shuttle configuration is blissfully simple in comparison.

The Submit Portal

A third transfer option pushes files to your storage. This is basically a drop box that allows a client to send you a file (and add an optional message). Again the emphasis here is on simplicity: drag files or folders to the browser and you’re done. To benefit from Signiant’s advanced transfer over UDP, the Signiant app needs to be installed (it’ll be automatically triggered by the browser page).

The only thing I would have liked here is an option to automatically generate a timestamped folder name for incoming files (e.g. incoming_05_31_17_1500) to help keep versioning straight under deadlines, but again this starts to add custom complexity. That kind of thing can always be handled via a local watch script sitting on the server anyway.

Simplicity beats complex efficiency

I feel the need to digress for a moment on this point. I can’t emphasize enough how simplicity trumps level of customization. The obvious example in the world is Mac vs. PC. But I’ve seen it time and again in the post-production world. Complex systems break down in crisis. And production staff are great organizers but not always the best technologists. They’re also way too busy to spend a day in a seminar learning how to use a proprietary digital asset management system.

So even though Media Shuttle emphasizes simplicity, it no doubt fits well with the larger studios. I remember when we first implemented Shotgun at my studio: for the first three months I had a PA translate the reports and enter data for me because I simply didn’t have the downtime to learn how to use the interface. Media Shuttle does its job and hides the complexity. No need to write a thousand lines of python scripting just to get files moving.

As a quick note: APIs do exist for larger studios looking to implement Media Shuttle with their DAMs and MAMs; they’re just not necessary for taking advantage of Media Shuttle’s file transfer services.

Managing the system

Configuring a file transfer service is typically a nightmare for those of us who don’t spend our lives grepping, piping, and rebooting Apache services on Linux servers. What surprised me as I went to work on Media Shuttle is the fact that I didn’t need to pull up a single help doc or call for tech support. Let me give you an example workflow:

To add a client to the Share Portal (the “FTP-like” web page) I choose “Person to/from File System” from the choice of portals at the top of the management page.

Then I select the members tab and click Add. I type in their name, email address and the expiration date–the date I want their access to the site to cease. By default they have access to the entire server. But if I simply click the Change button, I can choose a specific directory level of limited access. Finally I check the boxes for upload access, download access, and edit access and I’m done. The client is now able to access the relevant files and only the relevant files.

There is a help section at top right of the page that explains options in plain English. At no point did I feel like I needed to be a Microsoft Certified Systems Engineer to manage the system. I’ve had a harder time configuring my home wifi router.

You can create as many of these portals as you like to help partition work between clients, or tasks (e.g. VFX vs editorial). So the system can be extremely flexible, even while keeping the complexity to a minimum.

Hosting

Just to be clear about one point: Media Shuttle is a transfer service, not a hosting service. They don’t own the storage your stuff sits on. Who does? Well, you do probably. Media Shuttle works either as an application sitting on top of your existing Linux or Windows file server, or in partnership with a cloud-based hosting service.

The only bummer here is lack of OS X server support. While many studios have switched to some kind of Linux file server, there are plenty of small creative boutiques still serving their files from a Mac Mini, so that’s something to consider when switching to Media Shuttle. When I mentioned this to Signiant they recommended using a virtual machine like Parallels on the host server. Seems like that would work just fine; data transfer to/from the outside world would be a bottleneck long before the overhead of the VM became an issue.

As far as configuration goes, Tyler from Signiant assured me that the install was painless and that if any complications arise Signiant’s tech support can walk you through configuration in a matter of minutes.

For my evaluation I took advantage of the cloud-based hosting option. Media Shuttle is ready to work out of the box with either Amazon’s S3 offering or Microsoft Azure. In the case of cloud hosting, Signiant attaches to either of the services. The process and offering is the same, albeit with the addition of a monthly payload allotment to account for cloud service transfer fees. I can’t speak to the locally hosted experience (although I have a feeling I’ll be implementing it sometime over the coming year), but the cloud-based service behaved flawlessly.

Security and transfer rates

I am not an internet security expert, nor a dark web hacker, so I can only take Signiant’s word as to the security of their system. However, having previously endured the process of MPAA certification at my own studio and having been involved in security audits while working on the lot at one of the majors, I can attest that Signiant’s transfer process being in line with MPAA recommendations is no small thing.

As far as transfer rates, in my very informal testing I found them to be “alarmingly fast.” How fast is that? Well, 2.1 GB of EXR files took 52 seconds to download from the Amazon host server. 20 seconds of that time was file prep and handshake (which I imagine would be pretty consistent even on larger pulls), so that puts the actual transfer speed at around 525 Mbps. Now my connection on speedtest.net measures at around 370 Mbps. How can Media Shuttle be that much faster? I have to assume it’s because speedtest.net is measuring based on highly-redundant TCP packets, while Media Shuttle is leveraging its proprietary protocol sitting on top of UDP. Whatever the case, it’s fast. (In a very unscientific comparison test, 150MB of EXRs took less than 10 seconds to upload–including the handshake period at the front end, while the same files took over 3 minutes to send via the free wetransfer.com service.)

What’ll it cost me?

When enterprise-style solutions are discussed in the post-production world, most people assume that “if you have to ask the price, you can’t afford it.” As we’ve mentioned though, Media Shuttle is perfect for boutique studios, so the price obviously needs to work for that market segment. The “professional” packages begin at $7,500 a year but there’s special pricing for companies with fewer than 20 employees. Everything is based on the number of active users per month; there’s no limit to how much data you’re permitted to transfer. The Signiant definition of an “active user” is someone who sends a file or receives three or more files in a given month. These are floating user licenses, so you can have as many users as you like in the system (e.g. multiple clients) and in a given month only those users that meet this definition of “active” will be counted against your quota.

If you’re just getting started and running a lean ship, that might seem a little steep. But one thing that was clear to me from my conversations with Signiant: their pricing structure isn’t set in stone. Signiant is eager to make the service work for smaller companies (under 20 employees) and will endeavor to design a package around the needs of the client.

Overall, I’ve been impressed by the flexibility and transparency of the pricing structure. Want to create separate portals to the physical storage at your studio and a different setup at your house? No extra charge. Have media storage pools distributed between offices in New York, Los Angeles, and Sydney? Create a unique portal for each. What about local storage for team collaboration and an Amazon S3-based portal for clients? Again, no additional hidden fees.

Of course if you’re trying to court bigger clients, even the $7,500 annual base could end up being a trivial cost in comparison to the payoff. Larger client businesses will instantly balk at unprofessional work practices and media transfer is one of the first encounters a new client will have with you in this connected world.

Conclusion

As you’ve probably already guessed, I was impressed with the Media Shuttle ecosystem. It’s rare that I come across enterprise level software that has as much intuitive usability. With the caveat that I’m yet to test the system on my locally-hosted file server, I’ve found little not to like with the service.

Boutiques already well established will instantly see the value proposition here. The alternative is to run your own file services. This might seem like a good (cheaper) approach out of the gate, but once you add configuration, IT costs, lack of security, and the sheer amount of time spent trying to manage permissions and maintain everything, $7,500 a year can be a downright bargain.

Visit Signiant’s Media Shuttle site for more information.

Categories
Post Production

Fusion 9 in depth review

There was a lot of chatter around Fusion a couple of years back when Blackmagic acquired the company. But in the ensuing time spent porting to Mac and Linux, the application fell far behind industry standard Nuke in terms of features and capabilities. That’s all set to change with the new Fusion 9.

On paper, Fusion 9 checks all the boxes. Let’s take a deep dive into the new version and see if it can deliver.

The 3D Camera Tracker

To me, this is a must have for a modern compositing system. Yes there are standalone applications like Syntheyes and PFTrack dedicated to the task and able to solve extreme cases, but 3D tracking is so fundamental to the task of compositing in 2017 that it needs to be inside the compositor. (It doesn’t hurt that you’re not paying for an additional app).

The Fusion camera tracker is very much like Nuke X’s tracker and Nuke artists should be able to adapt very quickly. Similar workflow, similar controls, and (as you’ll see) similar results.

My first experience with the new Fusion 9 tracker was…a little frustrating. That is until I talked with the product manager, Steve Roberts, who explained that some of the defaults needed tweaking. The main one is the “Auto select seed frames” option, enabled by default. This causes Fusion to appear to hang. It turns out it’s not hanging, it’s just taking a really, really long time to choose the seed frames. Disable it and choose two of your own and things are good.

In general, the defaults are less than ideal–the Detection Threshold and Feature Separation are far too high. It’s hard to take off marks for this though. It’s common for first version defaults based on internal test footage to need adjustment once they hit the real world.

I ran the same undistorted footage through Nuke X’s current version tracker and Fusion’s camera tracker. Both took similar times to track and solve (Fusion’s solve was longer). I received a reprojection error of 1.03 in Nuke and 0.88 in Fusion. While that technically places Fusion as the winner, for a first pass without having culled any outliers that puts them both pretty much in the same ballpark.

I asked both apps to refine the lens value. I started with a lens focal length of 18mm. Fusion returned a 17.75mm lens, while Nuke proposed a 17.35mm lens. Again both were close.

After my first culling of points I had a reprojection error in Fusion of 0.32 (good enough for me). A similar culling in Nuke produced a 0.43 reprojection error. To be clear: This was not a scientific test and I’m not proposing Fusion beats Nuke based on one test (I’m not even sure Nuke and Fusion use the same RMSE test for their reprojection). What it does seem to show though, is that Fusion’s tracker is of comparable standard to that found in the $USD8,000+ NukeX. A test export of the tracker results showed a rock solid solve.

One caveat: Fusion seems to have a hard time refining a lens value that’s way off the mark. When I set the initial lens value to 50mm Nuke managed to refine it to 18mm without trouble. Fusion “spat the dummy” (if you’re not Australian, look it up) and returned the original 50mm lens unchanged with a large reprojection error of 17. In other words, you currently need to get your initial focal length close to the real world value for good  results in Fusion.

Overall this in an amazing first version implementation. I find myself missing a few features from Nuke, like the ability to graph number of valid tracks per frame, or some of the more flexible alignment tools available in the Nuke viewer. And even though the documentation talks about aligning the ground plane from a point selection, I was unable to find the option in the release version (this may well have been user error on my part).

Planar tracker

I’m a little biased when it comes to planar trackers. Mocha Pro is king of the hill in this realm and every time I see a host application attempt a native planar tracker, it doesn’t come close. Nonetheless, having a planar tracker built into the app is handy.

In my informal testing I’d say that Fusion’s planar tracker wins some and loses some over Nuke’s native planar tracker. It seems to lock on very well for the most part. Nuke seems to handle deformation better (such as facial skin during dialog), though still not as well as Mocha Pro. I wouldn’t use Fusion’s planar tracker for 4K dialog close-up skin grafting for example.

In summary, the new, native Fusion planar tracker is a great tool to have available for general tracking purposes like roto assist (via the handy Planar Transform utility node). For heavy duty applications like cosmetic fixes you’ll still want to shell out the extra few hundred bucks for the Mocha Pro OFX plugin for Fusion. (I’d say the same thing of Nuke though.)

Delta Keyer

Single click key in Fusion

Fusion’s previous default keyer, the Ultra Keyer, was competent but by no means world class. “Ultra” was perhaps a bit of a stretch. And while the Ultra Keyer has actually been improved in version 9, the real news is the brand new Delta Keyer.

Delta Keyer is a color difference keyer which seems to combine the best of Nuke’s Keylight keying system with Fusion’s matte finessing tools. In fact to a Nuke artist the Delta Keyer will feel very much like Keylight (my personal preferred keyer), complete with a shadow, midtown and highlight tuning system for selective matte hardening.

In my initial tests Delta Keyer performed famously. In fact, I found Delta Keyer’s single click initial results to be much more pleasing than Keylight. Delta Keyer also includes a nice clean plate generator (think Nuke’s IBK, but without the 1990’s performance) that can feed into Delta Keyer to help with uneven green screens.

Single click key in Nuke

Of course I haven’t put any truly nasty production green screens through just yet, but I have a suspicion Delta Keyer may be my new, favorite keyer. Interestingly enough, Blackmagic acquired Ultimatte last year but there’s no sign of their keyer added to Fusion (even though it’s currently licensed for Nuke). Perhaps it’ll make it’s way in at a future release, but if Delta Keyer proves itself in production Ultimatte may just be a redundant offering.

 

Studio Player for shot review and versioning

Complete confession time here: I haven’t even cracked the lid on this yet. But I will say that the feature set described in the release notes perfectly matches what’s required of such a tool. I’ve often scratched my head at The Foundry’s Nuke Studio and Hiero offerings. I’m not sure if it’s a Smoke system that’s fallen short of the mark, or a rudimentary review tool with too many bells and whistles.

Fusion’s Studio Player–at least as I read it on paper–has just the right feature set for a basic review tool: A simple timeline with versioning, annotation and basic color correction. Add to that playback through Blackmagic hardware and you should have a solid review system for a VFX house.

Interestingly this is the first logical connection between Fusion and Blackmagic’s hardware (where they make their real money). It will be interesting to see just how solid the play out is via the Blackmagic hardware.

Studio Player also includes timeline synchronization between workstations. Not sure how useful this would be; it’s also not clear from the release notes as to whether this would work for multisite over a VPN.

VR Toolset

OK, I guess I need to talk about VR… While the marketing for Fusion 9 has “VR” splashed against the title, I personally don’t think it’s what will move the adoption of Fusion. After all, the noise is already subsiding over VR for live action.

That’s not to say that Fusion’s offering is weak. In fact it offers a competent feature set of VR tools and 3D 360° rendering while sensibly avoiding the ugliness of the stitching process. Think of it as Cara VR without the stitcher. There are plenty of third party stitchers out there so using Fusion to clean, patch, and composite is a logical choice.

Apple ProRes Encoding!

One quiet little showstopper tucked into the Fusion 9.0 release notes: Apple-licensed encoding to ProRes on both Windows and Linux. I’m not sure any other app on the market has this. Yes, I know, there are several third party developers offering it, but without exception to my knowledge they are unlicensed offerings that could be shut down at any moment by Apple’s legal team. In the case of Fusion, ProRes encoding is legally licensed for the software.

Alas, while the free version can read ProRes on all platforms, only Fusion Studio can encode to ProRes on Windows and Linux (presumably because Blackmagic has to pay a licensing fee to Apple. They can’t be giving away for free what actually costs them money in licensing).

Is Fusion now poised to take Nuke’s market share?

Probably not; well certainly not any time soon. Studios invest a lot of money in the pipeline glue code that sits between Nuke and their other apps, not to mention their employees and the freelance talent pool. As cheap and competent as Fusion is, the cost of the software is just a fractional cost in the big studio equation.

However, to me Fusion is the no-brainer choice for the individual and boutique. How can you justify spending $USD8,000+ for a license of NukeX, then another $USD1,500 a year to keep your software current when Fusion Studio will set you back just $299 with (so far at least) free updates? I would argue that your average creative cannot. And those After Effects houses who have been eyeing node-based compositing as the alien but technologically advanced neighboring planet now have a justification to colonize it.

It’s Fusion, not Nuke

While Fusion is very similar to Nuke, these apps are still different. There are some things I miss from Nuke and other things that simply require adapting to a different workflow. I find the roto and paint tools functional but a touch clunky when compared to the refined Rotopaint node in Nuke. Lack of cornerpin concatenation is a big issue for my typical workflows (there is a solution: download the third party, free plugin suite Krokodove for an acceptable if limited workaround). I’ve also experienced problems with certain codecs and image sequences simply failing to load (I think this is probably just a new release bug that will work its way out in the coming weeks), and color management is more manual and tedious than it needs to be.

One thing that Fusion has that Nuke does not: an interactive connection with the DaVinci Resolve timeline. That effectively turns Fusion into a Smoke, and that with some of the best color grading tools in the industry.

Is it worth it?

The $299 price tag on Fusion 9 Studio is almost insulting to the developers. The fact that a large portion of the features are available in the free version is baffling as a marketing strategy (but we thank you anyway Mr Petty, whatever the thought process). Don’t forget that these new features are adding to an already competent core (with powerful stereoscopic and optical flow tools in the studio version).

As for me, I’ve run out of reasons to work exclusively in Nuke. Sure, when a large studio is footing the bill for the license I’m happy to comp shots in NukeX. But when I’m spinning up and down work at my own studio, Fusion is a whole lot better of a financial model.

Damian Allen is a VFX supervisor and pipeline consultant based in LA. He specializes in stereoscopic and picture-lock emergency effects work through his company Pixerati LLC. In addition to his hands-on production work, Damian lectures and trains artists around the world in compositing theory and technique.

Categories
Post Production

Siggraph 2017: the beginning of the end for assistant editors

Siggraph 2017 is essentially an NAB show for Pixar computer science geeks and VFX artists. But it also reveals the cutting edge of technology poised to make its way into the general post-production world. And for those paying attention, this year’s conference gave prescient insight as to how the job market may be eroded in the coming years.

The show floor itself was filled with the usual collection of VR hardware, motion capture companies, and VFX and 3D software (including the excellent new release of Fusion 9–stay tuned for an in-depth review shortly). But the real story was found within the dozens of technical papers presented during the conference sessions. Many of these have bizarre academic titles like “Variance-Minimizing Transport Plans for Inter-Surface Mappings”, “Harmonic Global Parametrization With Rational Holonomy” and “Spherical Orbifold Tutte Embeddings.” (I’m not making these up–they’re straight out of the Siggraph program.)

However there are several sessions that are much more direct in application. For example three of the lectures I attended involved researchers recreating facial animation from a single audio source with no video. Some of the results were remarkably organic.

To me one of the most interesting sessions was titled “Computational Video Editing for Dialogue-Driven Scenes.” In other words, how a computer can edit your scene for you.

Now the good news is that Michael Kahn and Paul Hirsch aren’t out of a job just yet. Their assistant editors, however, may well be. The most surprising thing about the paper is that the edit that the computer came up with wasn’t horrible. I’d encourage you to check out the short overview video below. Now it’s not going to win an academy award, but it was actually better than a lot of the mediocre editing that constantly makes it out into the real world.

The Stanford paper in a nutshell

Here’s how the system works: audio analysis aligns various takes with different portions of the script and different speakers. Facial recognition both identifies the actors in a given shot and the kind of shot based on the size of the face relative to the frame (e.g. a face that is close to the full height of the frame would be considered a close-up or extreme close-up, while a small face in frame would be considered a wide.) The system was actually pretty simple in its current implementation; given modern machine learning algorithms it would be relatively trivial to train such a system to identify and tag shots as OTS, cutaways, shots with camera bumps to be avoided etc.

Then idioms–the papers’ authors’ chosen term for cinematic language–are applied to get a desired edit. These idioms include: no jump cuts, start wide, and focus on a specific actor for emotional emphasis. Obviously all kinds of other idioms could be implemented.

The AI assistant editor

So again, to reiterate: I’m not suggesting that a computer will replace the important nuancing of emotion that makes great editors great. But think for a moment about the job of an assistant editor. Sort through footage, break out and arrange shots based on shot type, and sometimes create baseline edits from which another editor can begin. All these tasks are performed extremely well by this prototype project from Stanford.

Ultimately where this and similar technologies are disruptive is in eroding the traditional entry level platform into the industry. With less need for assistant editors, how does any editor get in the door to start working on high level content in the first place?

Like all disruptions in life, you can see this evolution of technology as threat or opportunity. It threatens to take away a slew of traditional jobs from the industry, but it also opens the door to individual creativity by removing more of the tedium that takes away from the creative.

Other takeaways from Siggraph

A few other observations from Siggraph: the film & video VR bubble, as predicted by many writers here, has burst. I found even the most avid evangelists from last year’s show were willing to concede to me that narrative storytelling in VR at a Hollywood level lacks a financial model. Most of the VR technology vendors are focused on VR and AR as a gaming platform or for industrial applications (arch viz, mechanical repairs etc.)

In the drone world, an interesting paper on multi-view drone cinematography showed promise for using drones to replace traditional steadicam and dolly shots, both indoors and outdoors. This includes the automatic avoidance of actors (i.e. not slicing your talent up with copter blades) and coordinating between drones so they don’t show up in each other’s shots. The indoor flight paths are still quite noisy due to air perturbations, but with cameras getting smaller and lighter I could see this being a viable alternative to dolly and jib work within the next couple of years.

Light field capture and display technology were also big at Siggraph. One particularly interesting light field capture method was a paper titled “Light Field Video Capture Using a Learning-Based Hybrid Imaging System .” What it essentially showed was the ability to combine a cheap Lytro camera with a standard DSLR to create effective light field video at a fraction of the price and bulk of existing methods. The paper addressed mainly using the technique for defocusing and rack focus in post, but it seems like it could be used for pulling quality keys without the need for a green screen. And while they combined the Lytro with a DSLR, there’s no reason that the same technique couldn’t be used with a higher-end camera.

The age of AI

One thing that ran throughout the technical papers: almost every paper used machine learning or deep learning (artificial intelligence algorithms) somewhere in the process pipeline. We don’t have to worry about Terminators or being plugged into a Matrix just yet, but there is a very real threat to jobs in highly technical areas previously considered immune.

I always feel like technological advancement is a mixed bag. I love having a smartphone, but I also remember a day when people left you to get a solid few hours of work done without having to constantly reply to emails, text messages and voicemails. At the end of the day, while I can be nostalgic for those less connected times, I’m not going to be able to stop the artificial intelligence revolution. The clear message: artificial intelligence is coming to our workplace and we need to start adapting.