HP 35s. FIX affects precision



#2

Hi all, by accident I have found that FIX mode affects the precision in some calculations.

FIX mode is used to control the number of decimal digits used in the display of numbers. Only to display them. But consider the next:

Insert this function in the equation list:
EXP(-SQ(U)/2)/SQRT(2*pi)

Set FIX to 3 and integrate the function between -5 and 2.88
You will get 0.997993696055

Set FIX to 5 and integrate the function between -5 and 2.88. You will get 0.99801133646

Similar results, but not identical.
Should this be added to the infamous bug list? I don`t know if I am missing something...

Edited: 22 Oct 2010, 4:36 p.m.


#3

Hi Pablo,

This is not actually a bug, but a feature. For more difficult or long integrations, you can specify a lower number of significant digits which will increase the speed of the integration at the cost of accuracy. See page 8-6 of the manual for more information.

Cheers,
Bart


#4

Thanks Bart, I missed this feature. The manual explains it perfectly.

#5

No, this is not a bug but a (quite useful) feature of the HP35s an other calculators with the HP Integrate. It was already implemented in the good old 34C, the first model with this function.


The idea behind this: you set the desired accuracy (by setting the display format) and the calculator provides a result based on that setting. This way you can choose between better precision and faster execution. In Fix 3 the 35s assumes the function it's integrating is given with three valid decimals. In Sci 5 it assumes six significant digits. So, settings like "Fix 9" or even "ALL" will provide a very accurate result, but the 35s may be busy for a few minutes, while Fix 4 returns a result with adequate accuracy within reasonable time.


Of course, all this is documented in the handbook (page 8-2). You will also find a (quite conservative) estimate for the uncertainty of the result in the Y-register.


Dieter


#6

Dieter --

Very good explanation. I'd add that FIX/SCI/ENG as integration-accuracy settings on the HP-35s, HP-33s, and HP-48/49/50 series don't have the same meaning as those on the HP-34C through the HP-32SII. An archived thread:

HP-33S estimated error of integration

A thorough explanation:

HP SOLVE/INTEG on all RPN-based models

-- KS

Edited: 22 Oct 2010, 11:49 p.m.


Possibly Related Threads...
Thread Author Replies Views Last Post
  HP Prime numerical precision in CAS and HOME Javier Goizueta 5 475 11-16-2013, 03:51 AM
Last Post: Paul Dale
  SCIOVR and ENGOVR in FIX mode Andrew Nikitin 5 408 07-02-2013, 04:46 AM
Last Post: Victor Koechli
  [wp34s] Converting to/from IEEE 754 Binary64 (Double Precision) David Maier 1 215 06-15-2013, 09:21 PM
Last Post: Paul Dale
  Precision DC-50 Mic 3 366 05-26-2013, 01:27 PM
Last Post: DavidM
  Estimating accuracy in finite precision computations mpi 17 956 02-22-2013, 09:44 AM
Last Post: mpi
  A temporary "fix", the Casio Prism (fx-CG10) LHH 5 402 08-25-2012, 07:39 PM
Last Post: LHH
  Craigslist find and fix - HP 19C Matt Fegenbush 12 682 08-13-2012, 02:57 PM
Last Post: Matt Fegenbush
  50G precision & accuracy Matt Agajanian 11 625 05-17-2012, 11:15 AM
Last Post: Crawl
  Keyboard fix for the HP 15C LE Jeff Raines 2 229 03-31-2012, 12:49 AM
Last Post: Jeff Raines
  Just a thought I had--a 35S Trig fix Matt Agajanian 62 1,830 03-26-2012, 10:40 AM
Last Post: bill platt

Forum Jump: