HP48SX accuracy questioned.  Printable Version + HP Forums (https://archived.hpcalc.org/museumforum) + Forum: HP Museum Forums (https://archived.hpcalc.org/museumforum/forum1.html) + Forum: Old HP Forum Archives (https://archived.hpcalc.org/museumforum/forum2.html) + Thread: HP48SX accuracy questioned. (/thread156122.html) 
HP48SX accuracy questioned.  Pal G.  09092009 A few years ago, my boss suggested the HP48sx was not an accurate instrument based on his misunderstanding the following article: The article defines two equations for calculating great circle distances and states the first equation listed is not very accurate depending on what calculation device or software is used. The author states the second equation is more accurate for all devices. My boss assumed the HP48sx was not accurate due to the error margin in the result of the first equation. I would like to argue any TI calc would give similar results to the 48sx when using equation 1. In a nutshell:
Equation 1 evaluated with Excel is accurate.
Equation 2 evalutated with Excel is accurate. Would equation 1 evaluated with a TIXX be more accurate than an HP48SX? Unfortunately, the author of the article did not include several variables, such as a pair of lat/lons, or earth radius. edit: fixed link.
Edited: 9 Sept 2009, 7:11 p.m.
Re: HP48SX accuracy questioned.  Thomas Chrapkiewicz  09092009 It would be interesting for you to share the actual data that you had used in your testing.
TomC
Re: HP48SX accuracy questioned.  Eric Smith  09092009 The first formula (based on the spherical law of cosines) is illconditioned for small distances and will have accuracy problems on almost any computer; it is simply much more sensitive to limited precision than the haversine formula. This is an example of why numerical analysis is important. Just because you have a mathematically correct formula doesn't mean that it will be generally useful on a computer (or calculator).
See also "Virtues of the Haversine", R.W. Sinnott, Sky and Telescope, vol. 68, no. 2, 1984, p. 159.
Re: HP48SX accuracy questioned.  hugh steers  09102009 the idea of a great circle is somewhat misleading in the context of earth navigation. use the spherical formulae for pure spheres, but for earth distances it's a bad idea. for some time i've been using Meeus' formula. i put some code here, http://www.voidware.com/earthdist.htm but recently, i became aware of a formula by Vincenty, http://en.wikipedia.org/wiki/Vincenty's_formulae. I read the paper and coded it up. it's a bit more complicated, but claims to be super accurate. However, i had problems with it not converging for points almost opposite the globe. in any case, unless you really want to measure distances on WSG84, you're going to have a height difference involved which the, as presented, Vincenty formula doesn't deal with. so, im still happy to use the Meeus approximation for real use.
However, at some point im going to have to try to get something that takes height into account too.
