The standard first input for the emissions calculation is the origin and destination airport, so that the minimum distance of the journey can be calculated. Later this value is often increased to take into account diversions, holding patterns, non-direct flight paths etc. Nevertheless, how accurately this initial point-to-point distance known?
Behind this page is a script I wrote to calculate the great circle distance around the Earth, modelled in the first instance as a sphere and then better as an ellipsoid. You can input the latitude and longitude of two airports, and see how much the great circle distance changes depending on the model. To achieve this I worked with the Wikipedia page of the Vincenty formula and the nice blog post by John D Cook.
The following table gives some examples and a comparison to the Great Circle Map (GCM) distance calculator. We see that the simple spherical model is at worst 0.4% wrong. The ellipsoidal model is in perfect agreement with the GCM calculator. In conclusion, the point-to-point distance input to the emissions calculation is very well known, and other sources of uncertainty will dominate.
Calculator inputs should be in degrees, N, E positive, S, W negative.
Outputs are in km.
Airport 2 | Sphere [km] | Ellipsoid [km] | GCM [km] |
---|---|---|---|
SFO 37.6190,-122.3750 | 9375.727 | 9399.169 | 9399 |
LHR 51.4706,-0.4619 | 788.067 | 789.760 | 790 |
PVG 31.1434,121.8050 | 9031.305 | 9051.366 | 9051 |
CPT -33.9648,18.6017 | 9109.983 | 9074.490 | 9075 |