Camera Screen Space Constraints

游戏 - 相机
, , , ,

Most gameplay cameras follow a single world-space pivot, maintaining a fixed offset from it. Sometimes, though, we want the pivot of interest (POI) to appear at specific screen positions. For example, in some lock-on systems, the character's POI is placed on the left side of the screen while the enemy's POI is placed on the right. This post explains how to translate or rotate the camera so the POI lands at the desired screen-space position.

Problem Formulation

Formally, we'd like to translate or rotate the camera so that the world space POI lands at the desired normalized screen-space position where . Coordiante represents screen center and is the top-right corner.

We can achieve this by either translating or rotating the camera, as introduced below.

Camera Translation

Displacing the POI by camera translation is straightforward: keep the camera's rotation fixed, choose a desired forward distance from the camera to the pivot along the local X axis, then slide the camera along its local Y and Z axes until the POI reaches the target screen position. We follow Unreal's coordinate conventions, where the forward unit vector is the X axis, right Y axis and up Z axis. The desired forward distance must be specified because the view volume is a perspective frustum, determines how screen offsets map to camera‑space offsets.

The following figure shows the relationship of the desired screen space position , horizontal field of view and the forward distance , in camera's local space projected onto the XY plane:

In one word, the POI's Y axis coordinate must be in the camera's local space. Given this, we can translate the camera in its local Y axis so that this constraint can be met.

The vertical case is analogous: with normalized screen Y axis position and vertical field of view , the required camera space Z coordinate is , where .

The following figure shows what we want for camera translation to maintain a screen space constraint. Note that the rectangular zone restricts the area where the pivot position can land.

Camera Rotation

Camera rotation is more complex. We start by assuming the desired rotation's forward direction is where is the pitch and is the yaw. Then, the inverse rotation matrix can be represented as:

The inverse rotation matrix maps a world-space vector into the rotation's local space. Hence, it maps the directional vector from camera position to POI, i.e., , to the local direction .

According to what we derived in last section, must satisfy:

However, the above equations do not have a closed-form solution and they are highly non-linear and ill-formed. But the good news is we can still numerically solve them through the Newton's method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
std::pair<float, float> UComposableCameraScreenSpacePivotNode::CalibrateRotationOffsetNewton(float TanHalfHOR,
    float AspectRatio, FVector Direction, FRotator LookAtRotation, float ScreenX, float ScreenY)
{
    using Trig_T = double(*)(double);
    Trig_T Sin = &FMath::Sin;
    Trig_T Cos = &FMath::Cos;
    
    constexpr static int Steps = 10;
    const float TanHalfVOR = TanHalfHOR / AspectRatio;
    const float a = ScreenX;
    const float b = ScreenY;
    const float m = TanHalfHOR;
    const float n = TanHalfVOR;
    const float A = Direction.X;
    const float B = Direction.Y;
    const float C = Direction.Z;
    
    float X = LookAtRotation.Pitch - UKismetMathLibrary::DegAtan(2.0 * ScreenY * TanHalfVOR);
    float Y = LookAtRotation.Yaw   - UKismetMathLibrary::DegAtan(2.0 * ScreenX * TanHalfHOR);
    X = FMath::DegreesToRadians(X);
    Y = FMath::DegreesToRadians(Y);

    // Start with small damping.
    float OldError = FLT_MAX;
    
    uint32 Iteration = 0;
    for (Iteration = 0; Iteration < Steps; ++Iteration)
    {
        // Compute common terms.
        const float SinX = Sin(X);
        const float CosX = Cos(X);
        const float SinY = Sin(Y);
        const float CosY = Cos(Y);

        const float S = A * CosX * CosY + B * CosX * SinY + C * SinX;
        const float F1 = 2.f * a * m * S - (-A * SinY + B * CosY);
        const float F2 = 2.f * b * n * S - (-A * SinX * CosY - B * SinX * SinY + C * CosX);

        // Compute Jacobian.
        const float DSDX = -A * SinX * CosY - B * SinX * SinY + C * CosX;
        const float DSDY = CosX * (-A * SinY + B * CosY);

        const float DF1DX = 2.f * a * m * DSDX;
        const float DF1DY = 2.f * a * m * DSDY - (-A * CosY - B * SinY);
        const float DF2DX = 2.f * b * n * DSDX - (-A * CosX * CosY - B * CosX * SinY - C * SinX);
        const float DF2DY = 2.f * b * n * DSDY - (A * SinX * SinY - B * SinX * CosY);

        float Det = DF1DX * DF2DY - DF1DY * DF2DX;
        if (FMath::Abs(Det) < 1e-3)
        {
            break;
        }

        float DX = (-F1 * DF2DY + F2 * DF1DY) / Det;
        float DY = (-DF1DX * F2 + DF2DX * F1) / Det;

        const float NewX = X + DX;
        const float NewY = Y + DY;

        // Evaluate new residual magnitude.
        const float SinXn = Sin(NewX);
        const float CosXn = Cos(NewX);
        const float SinYn = Sin(NewY);
        const float CosYn = Cos(NewY);

        const float Sn = A * CosXn * CosYn + B * CosXn * SinYn + C * SinXn;
        const float F1n = 2.f * a * m * Sn - (-A * SinYn + B * CosYn);
        const float F2n = 2.f * b * n * Sn - (-A * SinXn * CosYn - B * SinXn * SinYn + C * CosXn);

        const float NewError = FMath::Sqrt(F1n * F1n + F2n * F2n);
        const float OldErrorMag = FMath::Sqrt(F1 * F1 + F2 * F2);

        // Adaptive lambda adjustment.
        if (NewError < OldErrorMag)
        {
            // Accept update and decrease lambda (move toward Gauss–Newton).
            X = NewX;
            Y = NewY;
            OldError = NewError;
        }

        if (NewError < 1e-2f)
        {
            break;
        }
    }
    
    return { FMath::RadiansToDegrees(X), FMath::RadiansToDegrees(Y) };
}

For a more stable alternative, we can use the Levenberg–Marquardt's variant to the solve the equations.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
std::pair<float, float> UComposableCameraScreenSpacePivotNode::CalibrateRotationOffsetLM(float TanHalfHOR,
       float AspectRatio, FVector Direction, FRotator LookAtRotation, float ScreenX, float ScreenY)
{
    using Trig_T = double(*)(double);
    Trig_T Sin = &FMath::Sin;
    Trig_T Cos = &FMath::Cos;
    
    constexpr static int Steps = 10;
    const float TanHalfVOR = TanHalfHOR / AspectRatio;
    const float a = ScreenX;
    const float b = ScreenY;
    const float m = TanHalfHOR;
    const float n = TanHalfVOR;
    const float A = Direction.X;
    const float B = Direction.Y;
    const float C = Direction.Z;
    
    float X = LookAtRotation.Pitch - UKismetMathLibrary::DegAtan(2.0 * ScreenY * TanHalfVOR);
    float Y = LookAtRotation.Yaw   - UKismetMathLibrary::DegAtan(2.0 * ScreenX * TanHalfHOR);
    X = FMath::DegreesToRadians(X);
    Y = FMath::DegreesToRadians(Y);

    // Start with small damping.
    float Lambda = 1e-2f;    
    float OldError = FLT_MAX;
    
    uint32 Iteration = 0;
    for (Iteration = 0; Iteration < Steps; ++Iteration)
    {
        // Compute common terms.
        const float SinX = Sin(X);
        const float CosX = Cos(X);
        const float SinY = Sin(Y);
        const float CosY = Cos(Y);

        const float S = A * CosX * CosY + B * CosX * SinY + C * SinX;
        const float F1 = 2.f * a * m * S - (-A * SinY + B * CosY);
        const float F2 = 2.f * b * n * S - (-A * SinX * CosY - B * SinX * SinY + C * CosX);

        // Compute Jacobian.
        const float DSDX = -A * SinX * CosY - B * SinX * SinY + C * CosX;
        const float DSDY = CosX * (-A * SinY + B * CosY);

        const float DF1DX = 2.f * a * m * DSDX;
        const float DF1DY = 2.f * a * m * DSDY - (-A * CosY - B * SinY);
        const float DF2DX = 2.f * b * n * DSDX - (-A * CosX * CosY - B * CosX * SinY - C * SinX);
        const float DF2DY = 2.f * b * n * DSDY - (A * SinX * SinY - B * SinX * CosY);
        
        // Update using Levenberg–Marquardt (J^T*J+Lambda*I)Delta = -J^T*F.
        // Form J^T * J and J^T * F.
        const float JTJ_00 = DF1DX * DF1DX + DF2DX * DF2DX;
        const float JTJ_01 = DF1DX * DF1DY + DF2DX * DF2DY;
        const float JTJ_11 = DF1DY * DF1DY + DF2DY * DF2DY;

        const float JTF_0 = DF1DX * F1 + DF2DX * F2;
        const float JTF_1 = DF1DY * F1 + DF2DY * F2;

        // Apply Levenberg–Marquardt damping.
        const float JTJ_DAMP_00 = JTJ_00 + Lambda;
        const float JTJ_DAMP_11 = JTJ_11 + Lambda;
        const float Det = JTJ_DAMP_00 * JTJ_DAMP_11 - JTJ_01 * JTJ_01;
        if (FMath::Abs(Det) < 1e-3f)
        {
            break;
        }
        
        // Solve for Δ = -(JTJ + λI)⁻¹ * J^T * F.
        const float DX = (-JTF_0 * JTJ_DAMP_11 + JTF_1 * JTJ_01) / Det;
        const float DY = (-JTF_1 * JTJ_DAMP_00 + JTF_0 * JTJ_01) / Det;

        const float NewX = X + DX;
        const float NewY = Y + DY;

        // Evaluate new residual magnitude.
        const float SinXn = Sin(NewX);
        const float CosXn = Cos(NewX);
        const float SinYn = Sin(NewY);
        const float CosYn = Cos(NewY);

        const float Sn = A * CosXn * CosYn + B * CosXn * SinYn + C * SinXn;
        const float F1n = 2.f * a * m * Sn - (-A * SinYn + B * CosYn);
        const float F2n = 2.f * b * n * Sn - (-A * SinXn * CosYn - B * SinXn * SinYn + C * CosXn);

        const float NewError = FMath::Sqrt(F1n * F1n + F2n * F2n);
        const float OldErrorMag = FMath::Sqrt(F1 * F1 + F2 * F2);

        // Adaptive lambda adjustment.
        if (NewError < OldErrorMag)
        {
            // Accept update and decrease lambda (move toward Gauss–Newton).
            X = NewX;
            Y = NewY;
            Lambda *= 0.5f;
            OldError = NewError;
        }
        else
        {
            // Reject step and increase lambda (move toward gradient descent).
            Lambda *= 2.0f;
        }

        if (NewError < 1e-2f)
        {
            break;
        }
    }
    
    return { FMath::RadiansToDegrees(X), FMath::RadiansToDegrees(Y) };
}

A rough comparison is done examine the efficiency and accuracy of the two methods. Evidence shows that the LM method takes fewer average steps to converge but the vanilla Newton method is slightly faster regarding runtime due to less computation.

Alright, let's see what we can get for camera rotation:

Summary

It took me quite a while to work through this problem. I first tried to derive an analytic (closed‑form) rotation, but the results consistently deviated from the ground truth. Suspecting a closed form might not be attainable, I switched to an iterative solver, which produced exact results. I hope this post is helpful and contact me if there are any problems or suggestions.