AquaCal

computer-vision
3d-reconstruction
Refraction-aware multi-camera calibration that embeds Snell’s law into bundle adjustment, achieving sub-millimeter accuracy where pinhole calibration produces errors that scale with depth.
Published

February 1, 2025

AquaCal is a multi-camera calibration package that models the air–water interface as a physical element of the camera system — not as distortion to absorb — eliminating the depth-dependent reconstruction error that the pinhole model fundamentally cannot.

Standard multi-camera calibration tools (OpenCV, COLMAP, Kalibr) assume straight-line light propagation from object to sensor. For cameras imaging across an air–water interface, this is wrong: rays refract at the surface, and no amount of lens distortion parameter tuning will correct it. A pinhole optimizer will converge — reprojection error looks “good enough” — but the recovered geometry is systematically biased, and the bias grows with depth. Post-hoc correction does not fix it, because the projection model was never right to begin with. Any pipeline imaging through a refractive interface — aquaria, aquaculture, underwater robotics, submerged inspection, semi-aquatic species — inherits this.

Approach

AquaCal replaces the projection function inside bundle adjustment. The optimization jointly recovers camera extrinsics, a single global water-surface height parameter, and calibration board poses, minimizing refractive reprojection error under Snell’s law rather than pinhole reprojection error.

The pipeline runs in four stages. Per-camera intrinsics come from in-air video of a dry ChArUco board (Zhang’s method), isolating lens parameters from refractive effects. Extrinsics are initialized via pose-graph traversal, alternating between solving for board poses and camera poses along high-confidence edges to minimize error accumulation. Joint refractive bundle adjustment is then the core optimization: forward projection reduces to a 1D root-finding problem under the flat-interface symmetry, solved via Newton-Raphson to sub-nanometer precision in 2–4 iterations, and wrapped in scipy.optimize.least_squares with Huber robust loss and Curtis–Powell–Reid column grouping for sparse-Jacobian efficiency. An optional fourth stage refines intrinsics jointly with the refractive geometry.

A deliberate design choice: setting the refractive indices to 1.0 reduces AquaCal exactly to a standard pinhole calibration on the same solver. It is a refractive calibration package, but also a drop-in standard one.

Accuracy

Synthetic validation against a pinhole baseline sharing the same optimizer — so differences are attributable to the projection model, not the optimization methodology — recovered camera parameters with substantially lower error across every metric:

Metric AquaCal (refractive) Pinhole baseline
Focal length error 0.033% 5.7%
Camera Z position error 0.35 mm 9.66 mm
Lateral position error 0.47 mm 4.94 mm
Reprojection RMS 0.498 px (matches 0.5 px noise floor) 1.376 px

The central empirical claim is depth generalization. Across test depths of 1.1–2.5 m, AquaCal’s reconstruction RMSE stays within 0.38–0.42 mm (maximum variation 0.045 mm), while the pinhole baseline climbs to 0.96 mm laterally and 257 mm on the Z axis — more than two orders of magnitude worse than AquaCal’s 1.9 mm at the same point. The refractive anisotropy ratio (Z/XY) is a stable 2.1–2.5, a geometric property of the top-down array; the pinhole ratio varies erratically from 0.4 to 5.8.

On the deployed 13-camera hardware rig, real-world validation hit 0.674 mm RMSE across 7,762 triangulated inter-corner distances on 52 held-out frames (mean absolute error 0.27 mm, median 0.14 mm). That is only modestly higher than the idealized synthetic range, with the difference attributable to lens manufacturing variation, board planarity, and corner detection noise rather than model failure.

Stack: Python, NumPy, SciPy (scipy.optimize.least_squares with the Trust Region Reflective solver and Huber loss), OpenCV, ChArUco board calibration.