Final Cut Motion Tracking Plugins

Tracking an Image Feature


Lyric's Motion Tracking Plugins all employ the same basic motion tracking system, described in detail in this section. You identify an image feature in the clip to be tracked, set some tracking parameters and then ask the plugin to follow that feature's movement through the clip.

The result of a motion tracking is a frame-by-frame record of the movement of that feature throughout the clip being processed, stored internally within the filter. This "track" is then used to drive the final output of the plugin, such as stabilizing the clip or pinning another clip to that feature, and so on.

Unlike most of Final Cut's filters which are modeless, the tracking plugins have a set of fixed steps you work through to get the final processed clip. You select the step you wish to perform in the Step dropdown menu in filter's control panel. The first 3 steps in all the tracking plugins comprise the motion tracking system:

  1. Select the feature to track and set the tracking options. See Feature Selection.
  2. Start the motion track processing. See Motion Tracking.
  3. When the tracking finishes, check it and optionally repair any tracking errors, including retracking portions of the clip. See Track Repair.

Each of the steps has a corresponding section in the filter control panel containing parameters and point selectors for the step.

The Feature Center point control is used to select the center of the feature to be tracked.

Search Range sets the bounds for frame-to-frame feature search. The SubPixel Level dropdown lets you select the motion tracking resolution. CW Threshold sets the feature luminance threshold for center-of-weight based tracking. These parameters are described in detail in the tracking section below.

The Recenter Feature point control is used to reposition a incorrect tracking point during the optional Repair Track step and Repair Mode determines whether the correction is just for the current frame, or if it clears track from that from onwards, requiring a partial re-track. See the track repair section below for more details.



Feature Selection

You prepare for feature selection by applying the tracking filter to a clip in the timeline, opening the clip in the Viewer with the Filter tab showing, and positioning the timeline playhead over the clip at the frame showing the feature in the canvas at the position you want for selection. You should zoom the Canvas in to 100% or 200% or more so that the exact positioning of the point selector can be made.

You set the feature to be tracked by selecting step 1. Select Feature in the Step dropdown menu and then using the Feature Center point control to center the white search range square on the feature. If the square is over a white background and difficult to see, you can change it to a contrasting color by selecting Flip Cursor Colors in the Preferences section.

Note that if you select step 1. again after performing a track, the tracker assumes you want to retrack the feature and so will clear out the current track record.





Motion Tracking

Motion tracking is controlled by parameters in the Track section of the filter panel.

The Search Range slider sets the frame-to-frame bounds for the feature search, so it should be set large enough to cover the feature and its largest single frame-to-frame jump in the clip. This is different to the built-in FCP stabilizer, which requires you to set the search range to cover the feature jumps over the entire clip. The white square shows the current size of the search range. Larger ranges take longer to search and are more likely to contain similar features that might confuse the tracker, so choose a scan area that just covers the feature and its maximum frame-to-frame jump.

In the above example, the ball is being tracked. The selection square is centered on the ball in frame 1, but the movement in frame 2 exceeds the initial search range, so the motion tracking would not succeed. The 3rd image shows the search range increased to cover the frame-to-frame feature jump.

There are 2 tracking algorithms supported: Image Matching and Center-of-Weight.

Image matching scans the search area for the location that is least different, pixel-by-pixel, from the originally-selected feature. Lyric's image matching can search down to 1/8th of a pixel resolution for the best match, selected in the SubPixel Level dropdown menu. The finer resolutions yield smoother and more accurate tracks, but can take much longer to compute. Each level takes roughly 4 times longer than the previous to track. The finer resolutions also sometimes require higher-contrast features; see the section below on temporary image preprocessing for tips on this. You also may need to use finer subpixel levels if the feature being tracked is small (around 4 or 5 pixels). This is because of limitations in the built-in FXScript tracking function used in Image Matching.

Center-of-weight based tracking uses Image Matching to initially find the location of the feature and then looks for the feature's edges, based on luminosity differences, and then computes its center of "luminosity weight". This generates a very-much higher resolution track than image matching and only just a little slower, however it does require a well-defined more-or-less uniform object that has an obvious and consistent center-of-weight. You enable Center-of-weight tracking by setting the CW Threshold slider above zero. The section below on Center-of-weight tracking explains the technique and suggests good candidates features.



To begin the motion tracking, select 2. Track in the Step dropdown menu. The processing can take many 10s of seconds or even minutes, depending on the clip length and tracking settings. The insides of the cursor animate during this processing and when it is complete, the white search range square turns green (or cyan if the cursor colors are flipped) and a track line appears in purple if Show Track is enabled.

You can interrupt the tracking process at any time by hitting the escape key and selecting any step other than 2. Track in the Step dropdown menu.

When tracking is complete, you can play or step through the clip and the bounding square and center dot will follow the feature, showing you how it has been tracked throughout the clip. Any errors can be corrected using the Repair Track step, described below.

Should you need to modify any tracking parameters and retrack the feature, first select step 1. Select Feature in the Step menu, adjust the parameters, then select 2. Track in the Step menu to restart the track.




Center-of-Weight Tracking

If you enable Center-of-Weight tracking by setting the CW Threshold slider above 0, the tracker searches for feature edges around the object in which you place the center point, draws a red line around it, and moves the center dot and range square to the center of the object. This is based on assigning a "weight" to each pixel proportional to its brightness and then computing the center of the object's overall weight. This allows for very accurate subpixel center estimation as the object makes subpixel moves across the frame and natural antialiasing effects dim the pixels proportionately around the edges. You can usually set SubPixel Level to 1 pixel and yet get more accurate results than 1/8 pixel Image Matching. Finer settings are usually only needed for small features.

The object is defined by contiguous areas of similar luminance, which can be any shade of darkness or lightness so long as there is contrasting luminance around the object. You set the luminance similarity threshold using the CW Threshold slider. Increasing it includes more brightness ranges, decreasing it includes less, as seen in the following examples.

The best features for C-o-W tracking have reasonable area, at least 5 to 10 pixels high & wide, and are generally symmetrical so that the center of weight stays constant under perspective changes. Corners and thin objects are often not viable, even though they will usually track satisfactorily in Image Matching mode.

Some other good examples are shown at left. The "5" sign was selected off-center, in the white area at the edge, and the feature detector moved the center dot to the correct center.

The "T" at left is also OK, even though it does not have a distinct edge all around it. The tracker will limit the edge at the search range boundary. It does however, depend on both vertical and horizontal edges within the boundary, so that both vertical and horizontal subpixel positioning will be possible - the pixels at the boundary cutoff edges have no subpixel positioning information. It would not be satisfactory selecting a simple bar area to the right of the T, for example, as this would provide no horizontal positioning information.

Note that the C-o-W tracker uses the Image Matcher at the pixel resolution currently selected to first find the approximate location of the feature and then does its edge detection. Consequently, the feature has to be trackable by the Image Matcher for C-o-W to work. See the section on improving tracking below for advice if you get poor tracking results.





Tracking Repair

When motion tracking completes and the boundary box turns green, a record of the entire track has been made and you can play or step through the clip to see how the tracking has faired. The boundary box and center dot will follow the feature, and enabling Show Track will display the track as a line.

If any tracking errors are present, say because the feature is hidden for some frames, or changes shape enough at some frame to fool the matcher, you have the option of selecting 3. Repair Track in the Step dropdown menu to repair the errors.

When you do this, the boundary box turns yellow, indicating the tracker is waiting for a correction. You position the clip at the frame to be corrected and use the Recenter Feature center control to reposition the track center at that frame. Again, it is useful to zoom in on the Canvas so you can position the center at an exact subpixel position. After you've made the adjustment, the boundary box turns green.

If you have Show Track enabled, the adjusted track will be displayed as you manipulate the recenter point.

You can either repair just the current frame's track, and move onto another frame, or if the track has gone completely off target, start a fresh track from the frame on which that happens. You choose which repair mode to use by selecting either Current Frame Only or Retrack From Here in the Repair Mode dropdown menu. If you've chosen Retrack From Here, the record of the track following the current frame is cleared and it is necessary to choose step 2. Track in the main Step dropdown menu again and wait for the partial retracking to complete, as described in the Motion Tracking section.

Improving Tracking

Because the tracking plugins generate the track recording and render the final output in separate steps, it is possible to temporarily apply preprocessing filters during tracking to improve feature recognition, but turn them off or remove them during rendering.

You apply preprocessing like this by inserting filters above the tracking filter in the Filter tab. This causes the tracker to use the clip filtered by those you have enabled.


Good candidate filters are:

  • De-interlace, which will allow a fast-moving object to be tracked. The ball in the example to the left has been de-interlaced.
  • Gamma Correct or Levels, adjusted to increase contrast around selected features. Even with de-interlacing enabled in the example at left, the underlying FXScript image-tracking function was confused by the gray mesh background. By applying a Gamma Correction filter and setting the Gamma to about 2.0, the mesh darkened, increasing the contrast around the ball, and the track was successful.

The FXScript image-tracking function is also sometimes confused by small features, around 5 pixels or less in size. By selecting a finer SubPixel Level, say 1/2 or 1/4, you can often get a good track, since this internally magnifies the tracking area prior to the search. This is a useful trick in center-of-weight tracking, which would otherwise normally use a 1-pixel SubPixel Level.


Protecting the Track Recording

The Lyric tracking plugins generate a record of the motion track and use it to produce the final composite. This record is stored in a special "static" area in the FXScript plugin that Final Cut generally retains, even across saving and reopening of projects, but that it clears out in certain circumstances. Since this record has often been painstakingly produced, it is important to avoid the circumstances that will cause Final Cut to lose it.

The best way to ensure this is to generate the track and use it to render the final composite in one session within the open Filter tab in the Viewer. If you do have to switch to a different clip, or save & quit Final Cut before completing the rendering, you can reduce the chance of losing a track recording by first switching to the Filter tab in the Viewer before loading the clip under track back into the Viewer again.

You should also protect the final rendering by performing the tracking and rendering on a clip in its own nested sequence. That way, any adjustments in the parent sequence will not invalidate the rendered tracking.


.: Copyright © 2003 Lyric Media :..: Contact Us :..: Software License :.