How the Haversine formula works
Given two points (φ₁, λ₁) and (φ₂, λ₂) in radians:a = sin²(Δφ/2) + cos φ₁ · cos φ₂ · sin²(Δλ/2),c = 2 · atan2(√a, √(1 − a)),d = R · c. R is the Earth’s mean radius (6371 km). The formula was published by James Inman in his 1835 navigation textbook and is the standard for ship’s navigation calculators.
Why airline distances are usually a bit longer
Real flight paths add 5–15 % over the great-circle distance because of jet streams (eastbound transcontinental flights ride them; westbound avoid them), airspace restrictions (Russia, much of central Africa), and standardised arrival/departure procedures around big airports (Toronto’s Pearson uses ~30 km of arcs to align with runway 06/24). For nautical miles add ~0.5 % from the spherical assumption.
Vincenty for aviation precision
For genuine flight planning, switch to Vincenty’s inverse formula (1975). It treats the Earth as a WGS-84 oblate spheroid (a = 6378.137 km equatorial, b = 6356.752 km polar) and converges to ±0.5 mm. The extra accuracy is wasted on city-pair distances but essential for ICAO-standard flight plans.
Sources
- Inman J. Navigation and Nautical Astronomy. 1835.
- Vincenty T. Direct and inverse solutions of geodesics on the ellipsoid with application of nested equations. Survey Review. 1975;23(176):88-93.
- Natural Resources Canada. Geogratis postal code geographic coordinates.