In many compositing scenarios the ability to reproduce a real-world camera move can be a key factor in achieving believability. The 3D tracking feature in inferno and flame provides us with the functionality but it probably isn’t the most intuitive system. However, with the understanding of a few basic principles you should have success in most cases. This tutorial outlines some fundamental insights and a common working approaches. It is not the manual, but you should have some “luck” using these methods and find the 3D Tracker to be a very useful tool. Discreet’s manual 3D tracker is based on RealViz’s Matchmover. If you’re familiar with that software you may have some insight into how to proceed inside inferno/flame.

For starters, your footage must necessarily come from a moving (not locked-down) camera and the scene must contain some “markers” to track. The markers don’t have to be artificial (put in there specifically for tracking), but ideally they need to be spread out in different planes. For example, having points at various heights and distances from the camera lens will provide the best material for the “solver” to work with. If you have a lot of markers on one wall (all on one plane) that is perpendicular and directly facing the camera lens you’ll likely be in trouble – this means just placing tracking markers just on the back blue/green screen will not track well. It’s necessary to have 6 active trackers in frame at all times. 8 is better – more than that may be overkill as you will possibly spend too much time in the Analysis stage.

To perform a manual 3D Camera Track in flame you enter the Camera menu and hit the 3D Track button (under the Eye column at the bottom of the GUI). This brings up a new menu list specific to the 3D tracking process.

In brief, the buttons are (from top down):

• Camera – provides an area where you can define various real-world camera characteristic such as Lens, Film back, Focal Length.
• Stabilizer – the portal to the Stabilizer module where you’ll do the 2D tracking.
• Coords – where you define the camera orientation to the “world” (what’s up-down, left-right).
• Relations – where you define the relationships between tracker points.
• Frames – where you “Initialize” the tracking
• Analysis – the final stage to produce a solution.

Let me start off by saying that you may not need to do much more than track your scene (in the Stabilizer) and Analyze. The more information you feed the “Solver” (the part of the software that takes your input and then “solves” the artificial camera), the more likely your camera will represent the real one. But it takes more time to set up and solve.

Now some more detail on the steps to perform a track. Along the way we’ll suggest a couple tips that might make things a bit easier …

 

 

 

 

 

 

 

 

 

Side Note Tip #1 : Use the Ruler

It is sometimes helpful to use the Ruler feature. While the FFI camera is initially set up using pixel coordinates, the ruler allows you to work to a certain degree in “real world coordinates”:

1. In the Action->Setup menu, enable the Ruler button (ruler will appear on-screen).
2. Enable the Define button.
3. Adjust the ends of the ruler to some known distance in the scene (preferably, two points perpendicular to the camera).
4. Enter that value in the “Length” entry area.
5. Turn off the Define button.

Manual Tracking Steps

1. Set up your CAMERA

Enter as much information as you have (primarily the Film Back, aspect ratio and focal length). If you don’t have the focal length, it is vital to set the correct film back.

2. TRACK the scene in the STABILIZER

This isn’t intended to be a Stabilizer tutorial, so use the basic guides for using the Stabilizer you find in the manual.

• Track 1 point at a time and then de-activate it when done – then move on.
• If you lose a tracker offscreen or otherwise, delete the errant track points and disable that tracker.
• You can add a tracker on any frame and begin tracking from there forward. It isn’t necessary to have the tracker follow the whole scene through (although it seems to be helpful, the ideal scene to track would have all trackers in-frame throughout the scene, but that is often unrealistic).
• Don’t add a new reference on a tracker other than by the Snap method.
• Do not track backwards.
• Pick points at various heights and depths in the scene.
• Ideally, pick a point on the ground to use as the “Origin” (more on that later)
• Try to look for points which might share the same X, Y, or Z planes to use in the “Coordinate” and “Relations” menus (more on them later).
• It’s important that there be at least 6 valid trackers in the scene at all times – not meaning to say you have to track 6-at-once, but rather, at all times there must be 6 points for which you have gleaned a good track.
• If there is a specific point in the scene to which you need to eventually align an object (for example, you’re placing a 3D object on a table), you should put a tracker (or more) on some fixed point on that surface. It doesn’t have to track for the whole scene (a few frames may be sufficient).
• Save your Stabilizer Setup! You will have invested a good bit of work and it will be useful if you need to re-analyze. Furthermore, elsewhere in your composite you may find the data useful for other modules (tracking GMasks, for example. You can load the setup and then Copy/Paste Tracker channel data into the channels where you need them. The extra channels will be ignored in modules where they don’t apply).

Side Note Tip #2 : Stabilizer/Tracker Hotkeys

You may be dealing with a lot of trackers here. So, if you haven’t done it already, this is a good place to familiarize yourself with the hotkeys in the Stabilizer. Some very useful ones are:

Function	Hotkey	Description
Gang	g	to perform an operation on all trackers simultaneously
Selected	e	deals with the selected tracker
Snap	n	snap reference to current tracker position
Active toggle	`	toggle Active button on/off for current tracker(s)
Delete keyframe	Delete key	Deletes keyframes

I also recommend defining the following hotkeys (use F8 to create):

Function	Hotkey	Description
Analyze	Alt-N	performs tracking analysis
Add Tracker	Alt-`	add a new tracker

I have set up a hotkey for the Analyze button – so successive ‘n’ and ‘alt n’ will Snap Reference to the current position and then proceed with further analysis. The Gang and Selected hotkeys can come in handy ( g and e , respectively) – but use them judiciously. Gang will allow you to use the Active button to toggle ON/OFF all Trackers at once. CAUTIONARY NOTE: Gang will also reset ALL trackers if you hit the Reset button (will also Snap ALL, Analyse ALL, etc, (judiciously, remember?)). That caveat aside, it is good to be able to Gang all trackers to ON to review your existing tracks. I have also set up the hotkey combo Alt ` to Add Tracker.

The up/down arrow keys will scroll through the Tracker list (in the menu only 5 are visible at one time).
Also, note as the number of Trackers (and thus, motion paths) becomes greater, you may experience a general slowdown if you have many displayed at once. It is sometimes useful to turn off Motion Paths in the Setup Menu.

Briefly, my methodology (using the above hotkeys) is:

1. Add Tracker (hotkey: Alt `)
2. Examine the scene for a good reference point to track
3. Track until it gets “lost” (pen-down to stop analysis)
   1. if still viable point but needs a reference update:
      1. delete (hotkey: Delete Key) a few keyframes back from where it got “lost”
      2. Snap (hotkey: n) new reference
      3. continue analysis (hotkey: Alt n)
      4. repeat (1-3) until (3.2 below)
   2. if the point has gone offscreen or has become otherwise non-useable:
      1. delete any errant keyframes (Delete Key)
      2. de-activate the Tracker. (hotkey: ` )
4. Start back at (1) with a new Tracker (at this point I’ll sometimes back up a few frames from (3.2) and attempt to locate a point near the previous – one that will continue further along the sequence in order to provide a continuous overlap of Tracking channel data for that general location within the scene)
5. After you’ve done a number of tracks, it’s easy to forget just what and where you’ve tracked already. Here is another hotkey sequence which is very useful:

1. hit hotkey: g (Gang)
2. hotkey: ` (Active – will activate ALL trackers)
3. visually examine tracks
4. hotkey: ‘ (Active – will de-activate ALL trackers)
5. hotkey: e (Selected – which should re-select your last Tracker if you haven’t selected another)
6. hotkey: ` (Activate Selected Tracker)

3. Set up COORDINATES

Try to determine the most logical way to define the “world” your camera is looking into. For example, if you’ve footage of a room, you would probably want to define the X,Y,Z space of the room in a way that aligned to the geometry of the room. So, X, Y, and Z would follow the orientation of the walls, floor and ceiling (as opposed to some oblique angle).

Set up Origin. This is the 0,0,0 point in the X,Y,Z coordinate space of the scene. I usually like to try to place this on the ground plane but it is not requisite.

Distance from (point to point) – I usually use the same points I’ve selected for my Ruler!

Set up the X (or Y, or Z) direction between 2 points. X is from left to right. Y is from down to up (think as if you were to set an axis on the origin).

 

4. Set up RELATIONS

Parameters set up in this menu put “constraints” on the solver. Again, you are further defining the “world” the camera is seeing. A basic example usage is as follows:

1. Assume you have a room with two intersecting walls on which you have two tracking points
2. Hit the Create New button
3. Select the points in the left column list of trackers (Ctrl tap selects non-contiguous points)
4. Hit the right arrow button in the GUI to move the points into the left column
5. Select the “Share X” from the pulldown selection
6. Repeat (2) through (4)
7. Select the “Share Z” from the pulldown selection. IMPORTANT NOTE: The points you select share their coordinates relative to the ORIGIN, NOT relative to the CAMERA.

You can continue to add a Relation for any two or more points which you know share the same plane(s). For example, if multiple trackers are on the ground, you can create a new Relation which defines all those as sharing a 0.000 Y position (Constant initialized – assuming you’ve selected the ground plane as the Origin)

 

5. INITIALIZE FRAMES

This is where you tell the solver to initialize the Frames. Normally just the automatic ‘initialize Frames’ is enough. There should be enough keyframes to cover all the tracks, however too many keyframes will slow down the solution. As a guide, a good set of 2D tracks provides an excellent 3d track with keyframes set every 5 to 10 frames.

 

 

 

6. ANALYSIS

In Analysis you track the camera, producing a resulting axis tree in the schematic of all the trackers and the camera move. The table shows basically all the trackers and the accuracy between the original 2D track and the 3D track given the current solution. The average column shows the average residual difference over all frames. At the bottom of the table you can set a Threshold amount that will light up bad frames in the timeline, and help point out problem areas. If a tracker is showing as being inaccurate then you can deselect it so it is no longer enabled and contributing the the solution. You can do this for as many trackers as you want, when you see which ones are working well and which are not. As a rule the better the 2D track the better the solution, so spend time on the initial tracks.

The Reconstruction button (on by default) is vital – it creates an axis in action for each tracker.

Once the analysis is finished you should end up with an Axis on screen for each point you tracked. Now you should verify that you have a legitimate track. First just scroll through the move and see if it appears correct from Perspective view. Once that has been verified, you should look at the scene from the Side or Top view and evaluate whether the camera appears to be moving as you would expect. You may find that the top or side views aren’t really oriented perpendicular to the Tracked camera so it may be useful to add a camera from which to view and validate the scene. Are your Axes where you expect them (do the ones on the floor appear to be on the floor and on the same plane, for example?).

Another way is by placing a 3D object or objects in the scene to see if they appear “locked” in the scene and while maintaining the correct perspective. One very good empirical method is to place a grid on the ground plane (or any plane) and check to see if the perspective is correct (in a room, for example). It is entirely possible to have a great looking track where Axis is locked, yet have the incorrect focal length which will be immediately apparent by using the grid method. Fortunately, having the absolute correct lens is not always critical.
Occasionally an Axis may not be visible in the scene. This happens often with trackers near infinity (such as clouds) where the Axis is there, but invisible because it’s outside the camera frustum.

 

Fault fixing : “My track sucks! What do I do now?”

First of all, depending on the nature of the error, it may not matter. For example, if your camera is upside down but all you’re doing is a screen or sign replacement – although it may be disorienting to you, all you really need to do is flip your artwork 180 degrees. If you need to reanalyse the 3D track (after adding more trackers, for example) – you must delete all the axis nodes in the schematic. I find that it is sometimes necessary to Reset the Camera as well (Animation, select camera, Reset) because there seems to be some residual data which remains.

Does the track appear jerky or contain some unexpected bumps? This can almost always be attributed to poor or insufficient number of trackers. You may need to add some trackers (preferably in new planes) to the scene. Examine the Camera axes and take note of the frames where it appears to go awry. Go back into the Stabilizer and check your existing tracks for errant keyframes in those areas. Delete them (the keyframes) in the Stabilizer (you’ll need to re-analyze – don’t forget to delete the axis-tree in the schematic first). Check your minimum number (6-8) and try to make sure they overlap (where you lose one, you’ve backed up a few frames and started another).

Is the camera upside-down or otherwise incorrectly oriented? Check your Coordinates. You can readjust the Coordinates and then hit the “Set” button (in the Coordinates menu) and it will re-orient the Tracker/Axes to the new alignment (no re-tracking necessary!). Try swapping your “From” and “To” trackers and subsequently hitting the “Set” key.

Does the lens appear to be incorrect? (is the perspective of your grid wrong?) Sometimes this is not a critical issue. When it is, it is. Unfortunately you cannot “force” a lens into the solver. It appears to take your “recommendation” (from the Camera menu) but is not governed by it.

 

Submitter: Jake Parker