Affine Motion Parameters

Homework-3 | Digital Video Processing

Selahaddin HONİ

Derivation of Pan Motion Model

Pan motion is simply a rotation in y-axis. Therefore, the motivation is finding a correspondence of ϴy over affine motion parameters.
Ry matrix rotates the X, Y, Z coordinates to X', Y', Z'
Due to perspective projection, 2D coordinates (x,y) can be written in terms of Z and F.
    
Writing obtained results in matrix form.
There is an assumption that transition in pan motion is much less than Z.
Finally, the pan motion corresponds to a1 parameter of affine motion model under given assumption.

Derivation of Tilt Motion Model

Tilt motion is simply a rotation in x-axis. Therefore, the motivation is finding a correspondence of ϴx over affine motion parameters.
Rx matrix rotates the X, Y, Z coordinates to X', Y', Z'
Due to perspective projection, 2D coordinates (x,y) can be written in terms of Z and F.
    
Writing obtained results in matrix form.
There is an assumption that transition in tilt motion is much less than Z.
Finally, the tilt motion corresponds to a4 parameter of affine motion model under given assumption.

Parameter Extraction via Flow Vectors

Brief Explanation

Optical flow vectors (u,v) were calculated over Horn & Schunck algorithm. Then, least square approximation was applied to estimate 6-parameters of the affine motion model. Significant point here is the selection of appropriate pixels to solve the linear system.

I have used non-intelligent or in other words video-specific method to pick the relavant pixels. The woody region in the scene is static but appears as moving in the flow field due to camera motion. I thought, sampling pixels from this woody region is the simplest way to reach to the camera motion parameters.

Affine Motion Model

Parameters of the 2D affine motion model can be classified into two: angular (a2,a3,a5,a6) and translational (a1,a4). The effect of the pan and tilt motions are pixel displacements that relate to only translational parameters if the scene is far enough. To illustrate, pan and tilt motions can be observed from a1 and a4, respectively. However, zoom required angular motion parameters to monitor the scale changes, (a2+a6)/2; and needs matrix multiplication with pixel coordinates to satisfy the equation.

Result

In the below demonstrations, you will see the optical flow field next to original video and extracted motion parameters at the bottom. The colors in parameter values have an aim to simplify the observation.

  • RED Positive
  • BLACK Zero
  • BLUE Negative

First, I have clipped a short sequence that the transition occurs: PAN (left), TILT (up), PAN (right).
The transition of the camera motion in 5 frame/s
Final Result
Full video sample in 30 frame/s