The Jittering Issue with Damping in Cinemachine and How to Tackle it

If you are familiar with Cinemachine. you probably know there is a knotty problem with Cinemachine' damping if you are using Framing Transposer or some other components to track a follow point. That is, the camera jitters with damping enabled under unstable frame rate. The more unstable frame rate is, the more heavily camera will jitter. This post will discuss this phenomenon and proposes a workaround to solve this issue.

Camera jitters with damping in Cinemachine

Unity's Cinemachine has a notoriously severe problem that may cause the follow object to seemingly jitter when you are using the Framing Transposer component with damping enabled.

To show this, I did a simple experiment. I created a new blank scene and spawned a new attached with the following script:

public class CubeMove : MonoBehaviour
{
    public float speed;
    public int fps;

    public float CurrentSpeed
    {
        get { return currentSpeed; }
    }

    private float elapsedTime = 0.0f;
    private float currentSpeed;

    void Start()
    {
        if (fps != 0)
        {
            Application.targetFrameRate = fps;
        }

        currentSpeed = speed;
    }

    void Update()
    {
        elapsedTime += Time.deltaTime;

        if (elapsedTime >= 5.0f)
        {
            if (currentSpeed > 0.0f)
            {
                currentSpeed = 0.0f;
            }
            else
            {
                currentSpeed = speed;
            }

            elapsedTime = 0.0f;
        }

        transform.position += new Vector3(1, 0, 0) * currentSpeed * Time.deltaTime;

    }
}

This script moves the cube for 5 seconds and then keeps it steady for another 5 seconds and continues moving. The move speed as well as the fps (frames per second) can be set for test under different conditions.

A new virtual camera is then created with a Framing Transposer component following this cube. A default damping of 0.2 is used.

Here is result with speed is 100 and fps is 0 (when set to 0, the real fps is determined by Unity, may but unstable).

The jitters are very clear. You can also notice that the frame rate (presented in the Statistics panel) is very unstable, and we will know soon it is the unstable fps that results in camera jitters.

Cinemachine proposes a workaround to alleviate this problem, that is, to use the revised version of damping where they sub-divide each frame and simulates damping in the consecutive series of sub-frames. To enable this functionality, go to Edit -> Project Settings -> Player -> Script Compilation and add the CINEMACHINE_EXPERIMENTAL_DAMPING marco to it.

OKay, now we have enabled the new damping algorithm and let's see how it will mitigate the jittering issue. Here is result with the same setting we used in our previous experiment, i.e., speed is 100 and fps is 0.

It is astonishing to see the jittering issue becomes even more severe. I conjecture that the variance of fps will significantly amplify camera jitters when this feature is enabled. In other words, the experimental damping algorithm responds to the variance of fps in a NON-linear way: when the variance is small, the experiment damping will reduce the gaps of camera location between contiguous frames; but when the variance is large, it will enlarge the gaps, leading to unacceptable jittering. (Note: I did not validate this conjecture. If you are interested, just review the code and test it yourself.)

What about the expected result if fps is stable? Let's take more experiments!

Here is result with speed is 100 and fps is 120 (very high fps, which is usually prohibitive in shipped games).

Very steady camera! What about setting fps to 60? Here is the result.

An fps of 60 performs equally well with 120, which is anticipated as fps is stable. Okay, let's try a final experiment where fps is set at an extreme value of 20.

Even a low fps of 20 makes our camera stable, only if fps itself is stable.

Now we can conclude that it is the instability of fps that induces camera jitters, regardless of the exact value of fps. But, why?

Why camera jitters

Before answering this question, let us first take a look at the source of damping implemented in Cinemachine.

public static float Damp(float initial, float dampTime, float deltaTime)
{
    if (dampTime < Epsilon || Mathf.Abs(initial) < Epsilon)
        return initial;
    if (deltaTime < Epsilon)
        return 0;

    public const float kNegligibleResidual = 0.01f;
    const float kLogNegligibleResidual = -4.605170186f; // == math.Log(kNegligibleResidual=0.01f);
    float k = -kLogNegligibleResidual / dampTime; //DecayConstant(dampTime, kNegligibleResidual);

#if CINEMACHINE_EXPERIMENTAL_DAMPING
    // Try to reduce damage caused by frametime variability
    float step = Time.fixedDeltaTime;
    if (deltaTime != step)
        step /= 5;
    int numSteps = Mathf.FloorToInt(deltaTime / step);
    float vel = initial * step / deltaTime;
    float decayConstant = Mathf.Exp(-k * step);
    float r = 0;
    for (int i = 0; i < numSteps; ++i)
        r = (r + vel) * decayConstant;
    float d = deltaTime - (step * numSteps);
    if (d > Epsilon)
        r = Mathf.Lerp(r, (r + vel) * decayConstant, d / step);
    return initial - r;
#else
    return initial * (1 - Mathf.Exp(-k * deltaTime));
#endif
}

Translating into mathematics, we have:

where is the damp time parameter and the elapsed time in this frame. This equation decays the input , the distance for the camera to go to the desired position, by an exponential factor . If , the residual will be , meaning that at this frame, the camera will traverse 99% of the desired distance to go, only remaining 1% amount for future frames.

OK, let's assume we've placed a cube in the origin and it moves along the x-axis at a fixed speed, say, m/s. A camera is placed to track the cube with damping where damp time . Let's further denote the delta time for each frame by , where is the -th frame.

Having all variables fully prepared, we can then simulate the object movement and camera track process.

In the beginning of 0-th frame, the camera and the cube are both at the origin, i.e., (0, 0, 0). As the cube only moves along x-axis, we can emit the y and z dimensions and use a one-dimensional coordiante to represent cube and camera positions.

At the 1-th frame, the cube moves to , the distance the camera traverses is , and the residual is . We set for simplicity.

At the 2-th frame, the cube moves to , the distance the camera traverses is , and the residual is .

At the k-th frame, we have , , and .

Without loss of generality, we can set . The following sections will use this settings unless otherwise stated.

For different combinations of , may have different results. Let's dive into and see how it influences the results.

Case 1: Stable FPS, all are equal

When all are equal, say , our equations reduce to:

apparently has a limitation of when since . This explains why a camera with damping always has a maximum distance to its following target. There maximum distance, also the supremum, is exactly . When is larger, will be larger, implying the maximum distance between the camera and its following target will be larger.

What if is mutable? In this case, we can assume there exists an upper bound such that all satisty . Then we are able to derive the same conclusion.

Another question is, why camera does not jitter when FPS is stable? We turn to examine the sign of :

Therefore, when FPS is stable, is always larger than , and jitter will never happen.

Case 2: Unstable FPS, vary

When FPS is unstable, where may mutate, how will the camera move in response to its following target? We can still examine the sign of , but in another way:

This equation uncovers why camera jitters happen with unstable FPS. The residual at the k-th frame is essentially an interpolation between the following target's current position increment and the last frame's negative residual , where the interpolation strength is the decaying factor . As both and are fixed, a change in will incline the resulting residual to different ends, either or .

In our simplified case in which the target moves at a fixed speed in the direction of x-axis, will always be positive (though its magnitude can vary) and will always be negative. A mutating thus has a chance to alter the sign of , which further brings about camera jitters.

So when will camera jitter? From the above equation, we know that camera will jitter when the sign of consistently changes over time, i.e., the value of oscillates around zero. Let's make it equal to zero and see what we can find then.

This equation tells us when is near , camera will have a large chance to jitter. This motivates us to improve damping by filtering out the occasions where is very close to .

What about going deeper? We can treat as variable, and all other as constants. This abstraction gives us a function of :

Taking the derivative of , we know that is monotonically decreasing when and monotonically increasing when , and . Hence, to make the sign of mutable, must be positive and the minimum of must be negative.

The minimum of can be easily computed:

The last inequality holds because .

This reveals the fact that: when , a variant is likely to cause to change its sign, thus resulting in camera jitters. Suppose is large enough, so then the k-th residual gets smaller than while is positive. A smaller pushes to become smaller for the next frame, which further pushes the root of the function to become larger. In this case, even with the same delta time, will have a larger chance wo fall in the negative area, i.e., is more likely to be less than the root.

Solutions

Solution 1: imposing an invalid range

Based on what we've discussed so far, we can immediately come up with a simple solution: enforce to be if they are very close. That is to say, we use a small value , if , we just set to .

Note that can be zero or negative. If this is the case, we keep the original without doing anything. Besides, you should be aware that here is not the time this frame actually takes, instead, it is just the duration used to calculate damping.

Let us explain it more quantitatively. Suppose , where . Then according to our algorithm. We then plug into the original expression of :

This demonstrates that now the camera lags behind its following target more than the previous frame since the residual is larger. After substituting with , would be zero, meaning that the camera now keeps the same frame as last frame. Camera does not jitter.

Here comes the question: what if the following target slows down, or stops, or even turns back to the opposite direction and the camera still remains the same residual to it?

It is quite a good question. But if we look carefully at the function of , we will find this situation will never happen. Let's rewrite here:

This time, we do not constrain the value of , but at last frame, it's positive.

When gets smaller but still positive, we observe the function gradually shifts leftwards, pushing the root towards zero. This implies that the area gets contracted and the probability of remaining the same residual gets smaller.

When is zero where the following target stops, the current residual can be readily calculated as , which closes the distance gap between the camera of the following target. The ratio, which is calculated as , would be devided by zero, outputting an infinite value.

When is negative, will be negative. The ratio now becomes negative, also beyond the range of .

We can implement this algorithm in less than 100 lines of code. You should modify three files in the official Cinemachine source code directory.

First is Predictor.cs. Add a ImprovedDamp function:

public static float ImprovedDamp(float initial, float dampTime, float deltaTime, float bonus)
{

    if (dampTime < Epsilon || Mathf.Abs(initial) < Epsilon)
        return initial;
    if (deltaTime < Epsilon)
        return 0;

    float tolerance = 0.05f;

    float alpha = Mathf.Log(bonus) * dampTime / kLogNegligibleResidual;
    float ratio = deltaTime / alpha;

    if (ratio <= 1.0f + tolerance && ratio >= 1.0f - tolerance)
    {
        deltaTime = alpha;
    }

    float k = -kLogNegligibleResidual / dampTime; //DecayConstant(dampTime, kNegligibleResidual);

    return initial * (1 - Mathf.Exp(-k * deltaTime));
}

The input bonus is . Parameter tolerance is what you should set as we've introduced above.

In file CinemachineVirtualCameraBase.cs, add a new function ImprovedDetachedFollowTargetDamp:

public Vector3 ImprovedDetachedFollowTargetDamp(Vector3 initial, Vector3 dampTime, float deltaTime)
{
    GameObject go = GameObject.Find("Cube");  // Hard find our following target of interest, you should not do like this!
    Vector3 deltaDistance = new Vector3(100, 0, 0) * deltaTime; // Hard set the velocity, you should not do like this!
    Vector3 residual = initial - deltaDistance;
    Vector3 bonus = 
        new Vector3(residual.x / (residual.x + deltaDistance.x + 1e-7f), 
                    residual.y / (residual.y + deltaDistance.y + 1e-7f), 
                    residual.z / (residual.z + deltaDistance.z + 1e-7f));

    dampTime = Vector3.Lerp(Vector3.Max(Vector3.one, dampTime), dampTime, FollowTargetAttachment);
    deltaTime = Mathf.Lerp(0, deltaTime, FollowTargetAttachment);
    return Damper.ImprovedDamp(initial, dampTime, deltaTime, bonus);
}

This piece of code is very informal, and you should never write your code like this. The purpose of this function is to get and . I reckon the correct way to do this is to create a new (or two) variable in the CinemachineVirtualCameraBase class and update it in each tick. The code presented here is only for demonstration.

In file CinemachineFramingTransposer.cs, change the called function for damping:

cameraOffset = VirtualCamera.ImprovedDetachedFollowTargetDamp(  // Original is DetachedFollowTargetDamp
    cameraOffset, new Vector3(m_XDamping, m_YDamping, m_ZDamping), deltaTime);

You could also try other components, not just FramingTransposer here.

With the default tolerance=0.05, the result is shown below.

Camera jitters disappear. Note that the general fps is quite high (around 400~500). This is because our scene is quite simple, containing only a cube and a camera. In order to simulate a more real runtime game situation, I place 20k cubes in the scene and now the fps is around 30, but still unstable.

Below is the result when using the raw damping algorithm.

Camera jitters more severely due to a generally lower FPS. What about using the improved damping algorithm? Here is result with tolerance=0.05.

Just as expected, camera jitters do not show up. Let's try different tolerances. How will a small tolerance help alleviate jitters? Below is the result with tolerance=0.01.

Camera jitters occur again! This suggests that an excessively small value cannot fully filter out actions that can lead to camera jitters. Let's try our final experiment with tolerance=0.1.

Camera jitters disappear, but the camera motion seems a little stiff. These experiments show that an appropriate value of tolerance to ensure the smoothness and robustness of the camera.

Solution 2: adding low-pass filter

Our improved samping perfectly solves camera jitters under unstable fps, but it looks very stiff when it reaches the boundary of max damping distance. Can we make it more realistic so that the object won't just look stolid? Yes of course, we can add low-pass filter, or moving average to our improved damping to achieve more smooth results.

Recall the algorithm of the improved damping: if , we just set to . Instead of hard setting to , we introduce , the smoothed version of the original delta residual . If holds, we calculate as an average of and :

which can be iterated through a recursive form:

Note that gets updated if and only if holds, i.e., when the camera lies in the unstable area. The use of is similar to low-pass filters in the sense that they all filter out high-frequency signals.

Below is s sample code implementation in file Predictor.cs:

CurrentResidual = initial * Mathf.Exp(-k * deltaTime);
ResidualDifference = CurrentResidual - PreviousRedisual;

float result;

float tolerance = 0.1f;
float alpha = Mathf.Log(bonus) * dampTime / kLogNegligibleResidual;
float ratio = deltaTime / alpha;

if (ratio <= 1.0f + tolerance && ratio >= 1.0f - tolerance)
{
    float beta = 0.001f;
    CachedDeltaResidual = (1 - beta) * CachedDeltaResidual + beta * ResidualDifference;
    result = initial - (CachedDeltaResidual + PreviousRedisual);
}
else
{
    result = initial - CurrentResidual;
}

Let's try it out! With damping time and , we can achieve the following damping result with low-pass filter:

Now the camera looks much more smooth and, flexible. What about trying a smaller damp time, say, ? Here is the result:

The result is ok but sometimes it's still jittering. It is because a smaller leads to a larger and thus a larger chance to cause jitters. To solve this issue, we can set a larger tolerance , or we can have a smaller . We adopt a of and see how it performs.

The camera now becomes smooth again.

We can measure this sort of instability more quantitatively. Below is a graph plotting during five seconds of camera trace with damp time . The original curve (in blue) oscillates over time due to an instability of fps. The improved damping method eliminates all the oscillation and makes the curve absolutely plain. Empowered by low-pass filter, the curve becomes smooth without loss of stability.

Below is the graph with damp time . As can be seen, even with improved damping, the camera still has a chance to vibrate, and the original curve, oscillates much more intensely than with . Employing the low-pass filter gives a much smoother and stable camera motion curve, as expected.

Speaking of this, why can't we just soften our improved damping assignment to where is a function of parameterized by .

Assume and , we first calculate ; then calculate ; last, we have . For , we obtain . is a parameter controlling how fast the value of grows from to . The larger is, the larger mass will be concentrated on the side.

Below is the result with and :

Not bad! The soft version of improved damping really makes the camere smoother and less stiff than the vanilla improved damping algorithm. The follow plot also shows that with soft parameterization, the camera trajectory is much more natural with neglectable amount of oscillation.

We also compare it to different and . Beolow is the result with and .

A larger makes the camera more stiff, but is still better than the original improved damping algorithm.

Below is the result with and .

is less effective as the magnitude of attenuation it applies to is not enough to compensate for the osciallation the unstable fps brings about.

Let's try another damp time. The result with and shows as follows.

When , the oscillation is more severe, as we've already stated above. What about ?

Better, but still not sufficient to mitigate the oscillation. Let's try .

Almost perfect. We can conclude that a smaller needs a larger to offset the intense jitters resulted from unstable fps. Besides, you can combine the soft improved damping method and low-pass filters to achieve a smoother transition.

Solution 3: continuous residual

Okay, let's forget all aforementioned solutions and revisit our residual update formula at the very beginning:

Reformulate thie equation to the following form:

where is the speed of the camera's follow target, and is a more generalized form of the damping function . Theoretically, it can represent any function of interest.

Now, regarding as a function with respect to , we can seek to obtain its derivative:

We use the equality because when you plug into , you will get , implying .

What about derivatives with higher orders? We can calculate the second-order derivative as follows:

It is a nice form which bridges the first-order derivative and the second-order derivative . In fact, for any -th order derivative, it can be recursively calculated as:

Having all these derivatives, we can then expand using Taylor series and calculate the difference to :

Note that if we are still choosing as out damping function , the derivative of it with respect to will be and the value at zero will be .

In practice, we first decide how many terms in the coefficient term should be taken in, and then sum them up and multiply with the velocity term, the result of which is denoted by . The residual at the current frame, can be readily computed as . To save computation, we can first cache up to a threshold, say , and then using the formula of geometric series to efficiently compute the coefficient sum.

To estimate its error, we use the Lagrange remainder:

Decompose it:

where and as we assumed. We can see that the error is asymptotically negligible with respect to , especially when is small.

Recall that where and is the damp time. If is large, say 0.5 or even 1.0, the value of will be somewhat small so that a decent precision can be reached within few steps of expansion, i.e., a small say 2 or 3 could satisfy camera stability. However, if is small, say 0.2 or 0.1 or even smaller, the value of would grow larger, and then a larger might be needed to reach our expected precision. This is in accordance with our observation that a smaller generally leads to a more unstable camera trajectory. We will show this soon.

Let's first try and . Recall that is the maximum order of derivatives we use to approximate the residual difference. means that we only use in the coefficient term. Here is the result:

Looks nice! What about setting ?

Not much difference, but a little bit smoother. Let's try respectively with and . First comes .

It's okay but it seems too fast when the cube comes back to stillness. How about ?

Now everything gets worked! Next, let's set smaller, which generally won't be used in actual gameplay but as a test it's worth a try. We set and try different to see how they influence our camera trajectory.

Here is the result with :

Okay... a total mess. Try :

Unfortunately, the cube always stays behind the camera. Now :

Forget about it ... Let's try :

Things are getting better! At least it does not shake anymore and begins to stay at the right position. I bet is better:

It's close! Last, we try :

Finally, the camera disposes everything well. As we can see from the process, a small requires a large to reach the minimum acceptable precision. I hope you never have the chance to use such a small , and if it happens, cache enough orders of derivatives or it would be prohibitively expensive to compute at runtime.

To further understand why this method solves the jittering issue, we take a deeper look at the expression of derived above. This is an ODE and we solve it out (proof left to the readers):

Here I've expanded as . We cannot directly use this explicit expression to calculate because there is no correct time stamp when game is running. What we only have is the previous frame's residual and the elapsed time at this frame . And as the velocity may change over time, a closed-form of cannor serve our purpose well. We can only incrementally calculate camera residuals at each frame based on what we currently have.

is a monotonic increasing function, and of course, it's continuous. The continuity ensures that the camera trajectory is always smooth and never jitters, if fps is sufficiently high (over one thousand I suppose?).

For the original discrete residual, its velocity is:

where is from Lagrange's Mean Value Theorem. Note that I add a tilde symbol over to distinguish it from the one from the continuos version above.

This is another ODE. We can solve it out (proof left to the readers):

Note that we solve the ODE with respect , the increment time rather than the absolute time . So, we introduce an initial value to control what the initial value of residual is at this frame, is now the elapsed time for this frame satisfying and .

The following graph shows that how the function changes with different and . It can be noticed that this function is very sensitive to the input , the elapsed time at this frame. A small change of the input would significantly change the sign of , thus causing camera jitters. We also notice that a smaller , derived from a smaller , pushes the function leftwards, which also makes it more vulnerable to inputs.

Below is a comparison between five damping algorithms introduced in this article, including the original damping. Damp time is set to 0.2. We observe significant stability improvement when using any of the four proposed damping algorithms. You should be careful when choosing the most appropriate algorithm because the situation on which you intend to use damping. How unstable is your fps? What is the damp time ? How is the tracked object moving? You should experiment with these algorithms and choose the one that best suits your needs.