Distance calculator

Great circle path length around spheres and ellipsoids

The standard first input for the emissions calculation is the origin and destination airport, so that the minimum distance of the journey can be calculated. Later this value is often increased to take into account diversions, holding patterns, non-direct flight paths etc. Nevertheless, how accurately this initial point-to-point distance known?

Behind this page is a script I wrote to calculate the great circle distance around the Earth, modelled in the first instance as a sphere and then better as an ellipsoid. You can input the latitude and longitude of two airports, and see how much the great circle distance changes depending on the model. To achieve this I worked with the Wikipedia page of the Vincenty formula and the nice blog post by John D Cook.

The following table gives some examples and a comparison to the Great Circle Map (GCM) distance calculator. We see that the simple spherical model is at worst 0.4% wrong. The ellipsoidal model is in perfect agreement with the GCM calculator. In conclusion, the point-to-point distance input to the emissions calculation is very well known, and other sources of uncertainty will dominate.

Calculator inputs should be in degrees, N, E positive, S, W negative.




Outputs are in km.

 

 

Example calculations with Airport 1 as ZRH 47.4647,8.5492
Airport 2 Sphere [km] Ellipsoid [km] GCM [km]
SFO 37.6190,-122.3750 9375.727 9399.169 9399
LHR 51.4706,-0.4619 788.067 789.760 790
PVG 31.1434,121.8050 9031.305 9051.366 9051
CPT -33.9648,18.6017 9109.983 9074.490 9075