Distance between rotations

The 3D rotation group $SO(3)$ admits several representations. Unit quaternions (or versors) and rotation matrices are among the most popular ones. Independent of the representation though, there is an intrinsic notion of distance on $SO(3)$, which will be the main hero of this post. At the end of the day, it doesn’t matter whether you compute it using quaternions, rotation matrices, or even—God forbid—the Lie-Cartan coordinates of the second kind: the end result is guaranteed to be the same.

Using quaternions

Let $p$ and $q$ be unit quaternions representing two rotations in the same basis. Let $q^*$ denote the quaternion conjugate.

The difference rotation quaternion that represents the difference rotation is defined as $r \triangleq p q^*$.
The distance between rotations represented by unit quaternions $p$ and $q$ is the angle of the difference rotation represented by the unit quaternion $r = pq^*$.

Let quaternion $r$ have components $\left( \cos\left(\frac{\theta}{2}\right), \,\mathbf{u}\sin\left(\frac{\theta}{2}\right) \right)$. We can extract the angle from the first component

using the $\arccos$ function

$$ \begin{equation} \theta = 2 \arccos \left(| \langle p, q \rangle |\right) \end{equation} $$

with $\langle p,q \rangle = p_1q_1 + p_2q_2 + p_3q_3 + p_4q_4$ and $| \cdot |$ the modulus function.

Note that distance between quaternions is not the same thing as distance between rotations.

Using rotation matrices

Let $P$ and $Q$ be orthogonal matrices representing two rotations in the same basis. Let $Q^*$ denote the matrix transpose (as in Numerical Linear Algebra by Trefethen and Bau).

The difference rotation matrix that represents the difference rotation is defined as $R \triangleq P Q^*$.
The distance between rotations represented by rotation matrices $P$ and $Q$ is the angle of the difference rotation represented by the rotation matrix $R = PQ^*$.

We can retrieve the angle of the difference rotation from the trace of $R$

again using the $\arccos$

$$ \DeclareMathOperator{\tr}{tr} \begin{equation} \theta = \arccos \frac{\tr R - 1}{2}. \end{equation} $$

Note that distance between matrices is not the same thing as distance between rotations.

Why is distance measured by an angle?

In short, it follows from the Rodrigues’ rotation formula. For more details, read about special unitary group and consult these lecture notes on quaternions. To fully understand what is going on, read An Elementary Introduction to Groups and Representations by Brian Hall and Quantum Theory, Groups and Representations: An Introduction by Peter Woit.