HP30S Numerical Anomaly


Recently while looking at the calculator forensics web site,


I noticed that the HP30S was shown to pass the


test with no error.

I wondered if this was really true, so I decided to look at all the intermediate results during the calculation. I discovered that the zero error result is the result of a peculiarity of the HP30S arithmetic.

The HP30S is apparently doing binary arithmetic internally, with 80 bit precision (about 24 digit decimal precision). It can be verified that ordinary operations such as +, -, * and / are giving 24 digit results. Some of the other mathematical functions don't do so well. Joe Horn discovered a while ago that square root only gives about 12 digit precision. The trig functions can give substantially less than 24 digits; sometimes as low as 14 digits.

The full precision of a particular result can be seen using the technique described on the calculator forensics web site to see guard digits: (http://www.rskey.org/~mwsebastian/miscprj/guard_digits.htm).

The idea is to make a calculation and then subtract some of the leading digits of the result and then you will see the rest of the digits. It may be necessary after the subtraction to multiply by some power of 10 to shift the remaining digits left so they can be seen.

The last calculation in the calculator forensics test is an arcsin, and while looking at the details, I discovered the following peculiar behavior of the HP30S.

In degrees mode, type the following into the HP30S (followed by enter):

Type arcsin(.156434465031) and see the result: 8.999999999

Type (arcsin(.156434465031)-8.999999999)*100000000000 and see: 46.45174843

The digits from the second calculation should be appended to the digits from the first calculation to give a multi-digit result of: 8.9999999994645174843; the exactly correct result should be: 8.99999999946451748615554. In this case the HP30S gets 18 correct digits.

But now do the same calculation with a slightly greater argument:

Type arcsin(.156434465032) and see the result: 9

Type (arcsin(.156434465032)-8.999999999)*100000000000 and see: 100

The exact result should be: 8.99999999952252746476729

What happened? After some playing around, it seems that whenever the result of a calculation (of a trig function, at least) is an integer followed by .9999999995+, that is, the digits after the decimal point are nine 9's and a 5 plus just a few more non-zero digits, the HP30S just rounds up to the next integer. So what should have been:


turns into 9.000000000000000000000000.

And the same thing happens on the other side of an integer (slightly greater than some integer). For example:

arcsin(.156434465049) returns 9.000000001:

(arsin(.156434465049)-9)*100000000000 returns 50.86970993

Realizing that the first result was actually 9.0000000005+some more digits (it got rounded up to 9.000000001 in the display), we can construct the full result of 9.0000000005086970993. The exact result should be:

9.00000000050869710116852--the HP30S gets about 16 correct digits.

But now try arcsin(.156434465048) and get 9

Furthermore, (arsin(.156434465048)-9)*100000000000 returns 0
, indicating that there are no more digits after the integer 9.

Again the digits after the decimal point have been dropped, and we don't even get 12 digit accuracy.

It appears that whenever the digits after the decimal point are greater than .9999999995 or less than .0000000005, they get dropped--at least as a result of a trig calculation; I haven't checked all other caclulations, but this may happen always.

Why do they do this? Probably to avoid some of the apparent small errors that happen in simple calculations, which perplex naive calculator users. For example, do (sqrt(2))^2 on a typical calculator and get 1.99999999999...(depending on how many digits the calculator carries) instead of exactly 2. Or do sin(pi) and get something other than zero. Or do (6^-1)^-1 and get 5.99999999999... instead of exactly 6. The HP30S gets the exact expected result, but at the price of reduced accuracy when a result is near an integer.

So finally, this is why the HP30S gets exactly 9.00000000000000000 when running the calculator forensics test. To see the real error, one should subtract the input argument of the test from the final result. That is, do:


On the HP30S, the result is 0, leading one to believe that the HP30S is *really* accurate. But redo the test as:


to foil its rounding near integers and see a result: 4.833288903E-11

Not all that great for a calculator that can do 24 digit calculations, eh?

Possibly Related Threads…
Thread Author Replies Views Last Post
  HP Prime numerical restrictions? Alasdair McAndrew 4 1,802 11-16-2013, 05:32 PM
Last Post: Alasdair McAndrew
  HP Prime numerical precision in CAS and HOME Javier Goizueta 5 2,408 11-16-2013, 03:51 AM
Last Post: Paul Dale
  Help with flashing HP30s John Ioannidis 14 4,233 09-04-2013, 09:37 PM
Last Post: John Ioannidis
  [HP-Prime] AMBIGUITY between Numerical Calculation (HOME) and Numerical/Symbolic Calculation (CAS mode) CompSystems 2 1,428 08-18-2013, 07:06 PM
Last Post: CompSystems
  OT: My brain is failing me again. Help with numerical / mechanical problem required. Harald 4 1,791 07-01-2013, 10:31 AM
Last Post: Harald
  41CL TURBO anomaly? M. Joury 8 2,454 05-15-2012, 12:11 PM
Last Post: M. Joury
  OT: Pioneer anomaly Jeroen Van Nieuwenhove 4 1,527 04-20-2012, 08:35 AM
Last Post: Walter B
  Numerical Integration Namir 8 2,401 11-23-2011, 08:37 PM
Last Post: Allen
  Question about Numerical Integration Namir 8 2,341 11-11-2011, 07:53 AM
Last Post: Dieter
  Numerical Integration: HP, TI, etc. Chuck 30 6,729 05-28-2011, 10:51 PM
Last Post: Gerson W. Barbosa

Forum Jump: