Running day 2 backwards
Matches in, camera motion and 3D points out
- Day 2 went cameras → matrix: given $R$ and $t$, build $E$ (and $F$)
- In practice we don't know $R$ or $t$. We only have SIFT matches, some of them wrong
- Day 3 goes matches → matrix → cameras → points
| Stage | In | Out | Steps |
|---|---|---|---|
| Estimate $F$ | pixel matches | $F$, with inliers marked | 2–6 |
| Undo the camera | $F$, $K$ | $E$ | 7 |
| Take $E$ apart | $E$ | four candidate $(R, t)$ | 8–9 |
| Pick the real one | candidates, matches | one $(R, t)$ | 10 |
| Triangulate | $(R, t)$, matches | 3D points | 11–15 |
Day 2 assumed we knew how the two cameras sat: $R = I$, $t = (-1, 0, 0)$. From that it built $E$, and showed every match must lie on its epipolar line. That was useful for understanding, but it's the wrong way round for a real pair of photos, where the camera positions are exactly what we don't know.
The trick is that $p'^\top F p = 0$ works in both directions. Knowing $F$, it tests matches. Knowing matches, it's an equation for $F$. Enough matches pin $F$ down; RANSAC protects that step from the wrong ones; once we have $F$ and the camera's $K$, $E$ follows, and $E = [t]_\times R$ can be taken apart. With the cameras known, each match becomes two rays in space, and where they meet is the 3D point.
Why go through $F$ at all, instead of solving for $R$ and $t$ directly? Because $R$ hides its 3 unknowns inside sines and cosines, and the constraint multiplies $R$ by $t$. Equations like that have no direct solution; you'd have to guess and iterate. The entries of $F$, on the other hand, appear in the constraint in a plain linear way (step 2). So we solve the easy problem first, a linear one with 9 unknowns, and then extract the 5 numbers we actually want from its answer. Swapping a hard problem in a few unknowns for an easy one in more unknowns is a trick this subject uses over and over.
We keep day 2's running example throughout, so every answer can be checked against something we already know: $P = (2, 1, 4)$, $f = 500$, image centre $(320, 240)$, the match $p = (570, 365)$ in image 1 and $p' = (445, 365)$ in image 2. If day 3's machinery is right, it must hand back $R = I$, $t$ pointing along $(-1, 0, 0)$, and the point $(2, 1, 4)$.