prime programming / great circle formulae

Hello from an ABSOLUTE newbie to the Prime:

just got back from HHC2013 and of course the focus was on the Prime. Just so the 34S crowd does not get to upset I did pick up three new overlays from Eric: thanks Eric.

Plan on leaving one on the prize desk for next year fully loaded with IR, Clock ...

Back to the Prime; yes I can read the whole manual (I will) but I want to get to the programming and I learn that by looking at others examples.

The program is simple, I have done it in RPN, Basic, RPL, Fortran but not in Prime of course.

This would be a simple Great Circle program using four inputs,


being the departure(1) and the arrival (2) latitudes and longitudes.

the formulae for distance is:


the formulae for initial track (bearing) is:

è =ATAN2(COS(lat1)*SIN(lat2)-SIN(lat1)*COS(lat2)*COS(lon2-lon1), 

*note that the formulae for bearing reverses the arguments to ATAN2 – see notes below

Since atan2 returns values in the range -ð ... +ð (that is, -180° ... +180°), to normalise the result to a compass bearing (in the range 0° ... 360°, with ve values transformed into the range 180° ... 360°), convert to degrees and then use (è+360) % 360, where % is modulo.

For final bearing, simply take the initial bearing from the end point to the start point and reverse it (using è = (è+180) % 360).

I am trying not to be lazy here but any takers?

Cheers, Geoff

Edited: 26 Sept 2013, 3:50 p.m.


the formulae for distance is:


Geoff, I think you mentioned in your HHC2013 talk that your great circle distance calculations sometimes don't match the onboard computer and you attributed it to the large size (10 miles) of airports.

The calculation above appears to assume that the earth is perfectly round. In fact, it's oblong, stretched at the equator by centrifugal force. I know there's a more accurate equation in one of my astronomy books, but I can't find it at the moment.



Will this work?

EXPORT GREATCIR(lat1, lon1, lat2, lon2)
// D=Distance
// B=Bearing
// Sets the calculator to Degrees mode
// HAngle is found: Vars, Home, 6. Settings, 1. HAngle



B:=(B+180) MOD 360;

// Return distance and bearing to the home screen, in a list



This which leads to this lead you to the gory details!


Both Dave's are correct.

The formula used is from the FAA charts and is very accurate over short distances, say between waypoints of 100 miles.

In the talk the gross navigational check between the two distant airports is accurate enough. A few compounding errors enter; is an oblate spheroid correction being used by the FMC, (i will look into that, the manual is 1500 pages) and which datum points are the start and terminus; centre of aerodrome or primary navigational fix?

Eddie, well done, I will enter this and use it as a teaching ( why is part of the word teaching; aching?) exercise. Should I place the calc in exam mode with blinking lights :-)

Thanks Eddie and thanks for the comments at your blog. It is fun to use the calcs at work. The 41CL will make it back to my lab when I get back to school.

Cheers, all


Hi Eddie,

Add another bracket after the last line before the semi colon.


Also modified with HDigits:=0; and 60 instead of my initial 6371 for nm instead of meters.

I like this!

Thanks so much Eddie!




... is an oblate spheroid correction being used by the FMC

Not in ours (Honeywell GNS-XL).

...centre of aerodrome or primary navigational fix?

In ours: ARP (aerodrome reference point). Can be anywhere.



You are welcome, Geoff. I am glad you find it useful. Just curious how where the 6,371 constant came from?


I think this is the assumed radius of the earth in Km (ignoring oblation).

Otherwise, the formula doesn't give a distance.





Will do Hugh. I have the whole weekend off in London during your next meeting. I will be there for the AGM and all day Sunday.


Try the following on your Honeywell:

When flying east to west or vice versus set up three lines of longitude at 90 degrees to your course with varying lengths, in front of your nose.

For example if flying from 60N60W to 60N70W set up the following when passing 60W. Create a disco after your destination airport and insert the following three lines along the same longitude, then create a disco after each line,

65N65W to 55N65W short line @ 600nm

75N65W to 20N65W longer line @ 3300nm

85N65W to 85S65W longest line @ 6000nm

Use Plan mode and 'step' to the new lines in the legs page. You will see three distinct lines! Since it is the same longitude 65W what gives? Interesting mind game with your fellow pilot.

These lines converge and are coincident when the aircraft crosses the 65W point then they diverge again. In fact it looks like the lines bisect the earth, the shortest line sitting near the surface, while the longer lines deeper under the surface. As you fly over the 65W longitude you are looking directly down on all three lines. It would seem that the shortest distance between to distant points is through the earth and not along the surface. For those of you to familiar with FMC/CDU operations we use much shorter distances. The longest distance between two ponts on my flight plans has been 900nm. Therefore the 'burrowing' earth track does not show up. Also, 1 degree of latitude is 60 mm.

Just my explanation, maybe a talk with Honeywell or you may come up with a better one!


Thanks, already learned enough for rudimentary programming inputs, now to fancy things up. A pleasure seeing you again.


Edited: 27 Sept 2013, 1:00 p.m.




Wow, thank you Geoff and Daves! From 1981 through 1990 I was in charge of collecting and reducing the data for the CAFE air races held in California. To compute the distances involved, I used the algorithms in the RAND report "Hand Calculator Programs for Staff Officers" (1976) which used the oblate spheroid model. I added corrections for increased earth radius due to flight altitude and slant range increases due to climbing and descending. But I didn't know about or use the more recent algorithms or spheriod model you've pointed me to.

Fascinating! I'll have to play with the Prime emulator and try getting this to run on same. Should be both fun and informative.

Best to you, Maximilian and all pilots!


Missed you at the meet Jim.

I have always known that the altitude of the flight is not incorporated into the distance ACTUALLY flown. I just never took it to your level (pun intended) with corrections.

Of course I am just checking the ground based but adding 40,000' to the equation does make a difference to the radius!



P.s. playing with the prime is fun! Now if it could only print on my infrared.



for the curious ...

I used the algorithms in the RAND report "Hand Calculator Programs for Staff Officers" (1976) which used the oblate spheroid model

Link to Hand Calculator Programs for Staff Officers

... BEST!


Possibly Related Threads...
Thread Author Replies Views Last Post
  Programming workaround for "prepend" HP PRIME Marek Russ 4 767 11-29-2013, 05:46 AM
Last Post: Marek Russ
  Converting Great Circle Navigation from 41C to 42S Bill Triplett 11 1,410 11-13-2013, 07:24 AM
Last Post: Kimberly Thompson
  hp prime - programming with lists giancarlo 3 722 11-10-2013, 02:13 AM
Last Post: Giancarlo
  HP Prime - Drawing a circle from a program Jean-Michel 6 957 11-07-2013, 09:20 AM
Last Post: Tim Wessman
  HP Prime Programming Tutorial #3: WHILE, INPUT, KILL, REPEAT, GETKEY Eddie W. Shore 5 951 11-07-2013, 12:25 AM
Last Post: Han
  HP Prime - programming bluesun08 2 591 11-04-2013, 04:31 PM
Last Post: bluesun08
  More undocumented programming limitations in the HP Prime Michael de Estrada 3 766 11-03-2013, 08:22 PM
Last Post: Michael de Estrada
  HP Prime questions: I/O and Meta programming Andy Gryc 2 604 10-31-2013, 11:22 PM
Last Post: Andy Gryc
  HP Prime Programming Tutorial #1: LOCAL, RETURN Eddie W. Shore 6 1,142 10-26-2013, 08:42 PM
Last Post: Miguel Toro
  HP Prime Programming Tutorial #2: MSGBOX, IF-THEN-ELSE, PRINT, FOR Eddie W. Shore 0 512 10-26-2013, 03:51 PM
Last Post: Eddie W. Shore

Forum Jump: