By Jennifer Walden
Picture-lock, what’s that?
Filmmaking is evolving into a parallel editing process. When talking about their recent sound work on “The Batman,” supervising sound editors Doug Murray and Will Files discussed the creative benefits of having the sound and picture departments working in tandem. Also, it’s not uncommon to have updated VFX or new picture cuts making their way onto the dub stage during the final mix. (There are even rare instances — such as on “Pokémon Detective Pikachu” — when VFX changes came in after the printmaster was complete, according to sound designer John Marquis).
Each new picture cut requires a reconform of the sound and/or VFX sessions — to take what was already created and realign it to match the new picture cut before beginning work on that new version. A reconform also happens when changes in the picture (such as updated VFX) require additional sound work.
With the picture, sound, and VFX teams working concurrently, the need for robust reconform tools is on the rise. Here we check out three options — The Cargo Cult’s Matchbox, Nuendo 11’s built-in ReConform tool, and mfChangeNote .
MATCHBOX FROM THE CARGO CULT
In August 2020, The Cargo Cult released Matchbox, a reconform tool that has become a favorite in the post-production industry. It was nominated for a 2021 AMPS “Excellence in a Post-Production Audio Product” Award, and two CAS Awards for “Outstanding Product – Post-Production” in 2020 and 2021.
Instead of simply updating their Emmy-winning Conformalizer software, The Cargo Cult’s Founder & CEO Justin Webster decided on a fresh start. “Conformalizer was a really old codebase, was Mac-only, and used old 32-bit tech that Apple was killing off (in particular, QuickTime). So it really couldn’t be upgraded without massive amounts of work; it made more sense to start from scratch using modern, cross-platform tools. I also had a long wish-list of innovations and new approaches to solving the change management problem,” he said.
Matchbox is a change management solution that tracks changes in sound, picture, VFX, and even closed captioning. For instance, it can intelligently match VFX shots as they evolve. Matchbox can identify which shots have been updated, and even which elements within the shot have changed, using a visual delta and an optional overlay to highlight the changed areas.
“We support reconforming the (Netflix) TTAL files and a few other closed-caption formats,” said Webster. “There is a massive amount of progress that can be made in the foreign dubbing and localization industry, so we are hoping that supporting these closed-caption formats will finally wake that industry up to the possibilities of automated change management.”
Matchbox runs as a standalone application (on Mac or PC) that integrates with Pro Tools, Hiero, and any platform that can export/import timecode-based lists such as ADR Master and VoiceQ. Matchbox also outputs marker lists in the Media Composer format so picture editors can load change information directly into their timeline. Webster revealed, “This year, we are looking to add integrations with several other platforms.”
Matchbox works by comparing two versions of a reel or episode. It generates a damage report of the differences (represented as a “damage” percentage and a list of tracked changes). Users can review the changes in the timeline window and delete or heal cuts, extend the heads or tails of matches to fill gaps, set in and out times for selected matches, force match clips, create user notes anywhere in the timeline, and then ultimately render selected changes in the editing software (Pro Tools, Hiero, etc.), as “Copy/Paste” or “Cut/Paste.”
The beauty of Matchbox is that it can scale from simple to complex as needed. “There are a million different ways you can run your conform. Matchbox can compare anything, basically,” noted Dialogue Supervisor/Sound Editor Ryan Cole, who has worked on films like “Free Guy” and “The Adam Project.” Matchbox offers many different Matchup options, such as “Matchup Anything,” “Matchup Any Video Data,” “Matchup Any Audio Data,” “Matchup Selected Clips,” “Matchup Audio Files in Selected Gaps,” and more.
Cole notes that running the reconform as “Copy/Paste” is helpful for holding onto scenes that have been removed. “So if you’re working on a film for a few months and they decided to cut a scene that wasn’t working, before you reconform, you can move that scene further down the Pro Tools timeline and save it. If they put that scene back in five versions later, you can have Matchbox select just that scene, compare the two versions, and then copy the new version back in. You don’t have to do it manually, which is really handy,” he said.
Cole primarily handles dialogue, so he likes to start by importing the old picture EDL (edit decision list) into Matchbox, importing the new EDL, and hitting the “Matchup” button to compare the two. “The Summary tab in Matchbox’s List View pop-out window shows you which shots have been swapped, rebalanced, added, extended, removed, trimmed, updated, and matched. It even gives you a percentage of how much it has changed. Matchbox will automatically put markers at the changes, and you can drop your own user markers, Ryan Cole. too. Sometimes, Matchbox will flag places where it detected a cut or swapped shot but there was no change. You can just delete that marker and heal the cut so it’s one less thing to worry about when you render the reconform,” said Cole.
In Matchbox, users can focus on specific changes related to their responsibilities (such as just dialogue line changes or added processing like reverb, or music changes, or effects changes, or VFX changes, and so on) or they can look at all changes and create optimized notes for particular departments. User notes created in Matchbox can be dropped into the Pro Tools session during the reconform.
For more complicated reconforms, such as those that Sound Effects Editor Diego Perez was handling on “The Batman,” it’s important to have good coordination with the picture department. The Cargo Cult made it easy for post-production departments to request the necessary files from the picture department that will generate the best results for a Matchbox reconform. They created a “Matchbox – A Letter to the Editor” PDF (downloadable on their Support page) that can be emailed directly to the picture department. It’s an easy-to-follow list of instructions for editors using Media Composer, Premiere, or other NLEs. Perez said, “As long as we have a good AAF from the picture department, then Matchbox does a lot of magic on its own, which is amazing.”
When a good AAF isn’t available, Matchbox can compare just raw video or raw guide audio. “That has (surprisingly) become one of the most commonly used workflows. The ability to compare just the reference media is really important,” said Webster.
Using Matchbox for sound effects on VFX-heavy “The Batman,” Perez was able to quickly see the VFX changes using the “Scan for Visible Diffs” option in the “Compare” dropdown menu. Diffs are notated by purple segments on the Diffs timeline of the main window, making it easy to jump to each shot that changed. There’s a picture comparison window where all changes in the image are highlighted in red. “This was super helpful to scan for little changes, like updates to Batman’s device displays or computer screens. Sometimes things on those screens change and you need a little sound for that. You can make your own markers to note VFX updates for other editors or the re-recording mixer and those carry over to Pro Tools. This is what I usually do for effects,” said Perez.
Reviewing the matches is also helpful for removing unnecessary or unwanted changes in a reconform. “For example, if we’re cutting backgrounds, we can save time in Pro Tools by healing over sections in Matchbox that we don’t need to fix. Or, if the Foley footsteps are a bit out of sync, it’s sometimes easier to manually realign them instead of having Matchbox do it. It’s also great for rebalances, for bringing tracks from one reel into a different reel; Matchbox can simplify that process. The cool thing is you have options to make these choices before going into the effects reel,” Perez said. These examples are just a fraction of what Matchbox can do. Perez, who helped to beta-test Matchbox for The Cargo Cult, noted that “at first, the learning curve is kind of steep. But it’s definitely worth putting in the time to learn it though because you’ll get that time back ten-fold.”
Is Matchbox a good fit for you? Check out page 41 of the Matchbox Manual, accessible on The Cargo Cult’s (thecargocult.nz) Support page. You’ll find info on supported Mac OS and Windows versions (and some info for Linux users), supported versions of Pro Tools, supported file formats for input and output, supported frame rates, and other helpful info. Looking for a preview or how-to videos? The Cargo Cult’s YouTube page has a playlist of Matchbox tutorials.
Ready to try it out? Download a full version (14-day free trial) on The Cargo Cult’s Downloads page.
NUENDO 11 – BUILT-IN RECONFORM
Those considering the jump from Pro Tools to Nuendo (steinberg.net/nuendo) — a DAW that Steinberg dubs “the most advanced audio post-production solution available” — will find that it has a built-in ReConform tool that works by comparing two EDLs (an old and a new) from the picture department.
Selecting “ReConform” in the “Project” dropdown menu opens the ReConform window, which is separated into four sections: Old EDL, New EDL, Change EDL, and Video. The interface is easy to grasp at first glance, yet there are several options for how the reconform is handled. “It’s very predictable and that’s quite a good thing,” said Sound Assistant Simon Carroll, who works along-side supervising sound editor Johnnie Burn at Wave Studios in the UK.
The Old EDL and New EDL sections in the top half of the ReConform window (where the old and new EDLs are imported, respectively) lists information about the picture edit, such as clip ID, transition type (cut, dissolve, or wipe), EDL File name, source timecode for ins and outs, and destination timecode for ins and outs.
Clicking the “Generate” button in the middle of the ReConform window will provide an overview of the differences between the old EDL and the new EDL, displayed in the Change EDL section in the lower half of the window. This section shows a source (old version) timeline and destination (new version) timeline. Clicking through the listed clips in this section will highlight different areas on the two timelines, giving the sound editor a general idea of where a clip in the old cut is now located in the new cut. Empty spaces in the destination timeline indicate new picture has been added.
The Change EDL section has a “Preview” button, allowing the sound editor to see what will happen to their session during the reconform without actually making any of those changes. (Disabling “Preview” and clicking the “Start ReConform” button will render the changes to the session.) There’s the option to preview changes as “SRC” (source) or “DST” (destination). Previewing as “DST” will show the new timeline position for elements after the reconform is rendered.
Previewing allows the sound editor to make fixes to the reconform before rendering. For instance, they can heal two segments together if they don’t want to reconform those segments. Or, they can make cycle markers in their session for added clips, add memos for what’s needed in the new clips (new dialogue, ADR, backgrounds, sound design, etc.) or what’s changed in a particular section that needs reworking, or view/address automation warnings (alerts about changes in automation), and so on. The more drastic the picture changes, the more complex the reconform becomes.
Carroll noted that “with big recuts, you’re looking at making sound design work across a 20-minute reel. It could be incredibly complex. So whatever software you’re using, it’s important to consider two things: how quickly it recuts the material, and what you’re left with at the end of it.”
Since Nuendo 11 is capable of handling all the audio tracks for an entire film (it’s not parsed out onto two or three separate systems as is typically the case with Pro Tools workflows where dialogue will be on one system, and effects, design, and foley are on another), ReConform can run across all tracks, or just selected tracks — whichever the editor prefers.
The “Preferences” icon in the ReConform window opens up a panel of options. In here, the sound editor can change “Apply ReConform to” All Tracks or Selected Tracks, check/uncheck “Create Virgin Territories” (meaning the automation, including volume, in new sections will start from scratch), check/uncheck “Restrict ReConform Range,” set timecode to “Preserve Material Before” and “Preserve Material After,” and check/uncheck the following options: “Include Overhanging Audio Events” (with adjustable frame amounts), “Create Destination Markers,” “Create Dissolve/Wipe Markers,” “Create Insert Markers,” “Create Automation Warning Markers,” “Create Audio Snippet Markers” (with adjustable frame amounts), and “Create Overlap Markers.”
“We like to have the whole mix in one project, to have every element that goes into the soundtrack in one session. So for reconforms on sound design-heavy projects, oftentimes we’re rebuilding it as we go. Nuendo’s ReConform doesn’t track VFX changes like Matchbox, but for our work-flow that’s not a disadvantage. The way we go about re-cutting, it’s not a mathematical process. We’re more hands-on — rebuilding it and thinking about how it’s going to work as we go through,” said Carroll.
MFCHANGENOTE
What happens with reconforms on the dub stage, when it’s not an in-the-box Pro Tools mix on an Avid S6?
Re-Recording Mix Technician Luke Schwarzweller at the Fox Studio Lot — who’s worked on films like “West Side Story (2021),” “Star Wars: Episode IX – The Rise of Skywalker,” and “Jojo Rabbit” has a new go-to called mfChangeNote. “It’s a very small, standalone software. You just load in a change note and it allows you to simplify the changes to get rid of redundancies. You can choose to conform certain parts in the change note and you don’t have to do the whole thing. It runs outside of Pro Tools, but you can allow it to execute the conform within Pro Tools as well,” he said.
mfChangeNote (mfchangenote.com) was originally created by veteran sound editor Malcolm Fife, MPSE, for personal use back in 1990 when films were moving to digital. Fife was handling foley for shows that had video-film hybrid workflows “and conforming could be a tricky mathematical exercise at times,” he said.
It began as a spreadsheet that was essentially a “fancy timecode calculator for working with timecode and footage offsets,” explained Fife. Then in 2014, “mfChangeNote got a big overhaul when I was about to work on a major feature being cut in Premiere Pro, which had no built-in method for producing change notes. I hunkered down for a few months to create many new features and professionalize the application for public consumption.” mfChangeNote is a Mac-only appli-ation (OS X 10.8 or higher, 64-bit) that works for manual reconforms as well as automated conforming. It offers the ability to fix, annotate, format, or simplify a change note, and can be used to create change notes from scratch.
That’s a useful feature for Schwarzweller, who works with several mixers at Fox Studios who prefer to mix on the Neve DFC console. “If you’re conforming the DFC automation, you need to get an actual change note as a text file. Typically, you can get change notes from the picture department but on some movies they don’t provide them and you need to create one,” he explained.
mfChangeNote is also useful for picture assistants as it can generate an accurate change note that simplifies shot moves in an efficient way. Their website provides a “helpful list of tips” in the section on “Creating a Change Note from Adobe Premiere Pro or Final Cut Pro 7” on the “Using mfChangeNote” page.
Once Schwarzweller has the necessary change note, he loads that into Neve’s Encore software that runs the console. “The DFC does its own calculations and gives you the overall changes in the reel. At that point, we check to make sure it’s the same as what we have in Pro Tools. If it’s not, then there’s a problem with the change note and we need to address that, to sift through the changes and figure out where the discrepancy is,” Schwarzweller said.
mfChangeNote can print a logical, readable hard copy or PDF of the change note that shows the expanded/collapsed state of the open change note window, making it easy to scan through and compare changes.
Another feature of mfChangeNote is its ability to work in and convert time formats. Using the “Time Format” menu, users can choose to view time values in five different formats: 35mm Feet and Frame, NTSC timecode, PAL timecode, 24fps timecode, and Frames Only. (There’s also the option to “Convert framerate…” if you need to change the timecode’s counting base in relation to its timebase.)
Schwarzweller pointed out another main difference with a console environment as opposed to a virtual S6 mix. “Sometimes when we’re conforming the stems, we may actually do some repairing on the stems before we DPE (Destructive Punch Enabled) them and bake it back into a new file. We’ll spend some time to crossfade some of the stems and patch them, pull the handles out on the music, and that type of thing to make sure we’re all good. That allows us to not have to re-record over all the changes once we get the new length because we kind of fixed that in the stems,” he said.
mfChangeNote groups adjacent changes in a useful way. For instance, if the tail of one shot has been trimmed and the head of the following shot was extended, it will combine connected changes into a group that can be collapsed into a single summary line, or expanded to show every event.
When running a reconform, mfChangeNote will perform the changes exactly as listed ina the change note. For expanded groups, the changes will happen individually (one row at a time) while collapsed groups will happen in “one big chop,” according to the site, which also offers this bit of interesting advice: “mfChangeNote has the power to run some or all of the changes in your change note automatically in a sequence over your Pro Tools session. With this awesome power comes the ability to run roughshod over all your hard work, maxxing out the undo cue and making Swiss Cheese out of your fine, aged Wisconsin Cheddar. Don’t hand this command over to an intern on their first day. Don’t run this command while drinking Robitussin. You’ve been warned; Now, here’s your magic wand, Harry.”
Before implementing this powerful program into your workflow, it’s worth checking out mfChangeNote’s “Using mfChangeNote” page on their website. Each section under the Contents heading has helpful information about mfChangeNote’s approach to reconforms, tips for using specific features, and suggestions that will provide a better end result.
Jennifer Walden is a freelance writer who specializes in post-production technology.
