calc precision


Which of the HP calculators uses the highest precision?


HP30S (24 digits), except for square root (12 digits)


>>HP30S (24 digits), except for square
>>root (12 digits)

Wow. Why is sqrt(x) lagging? Common square root algorithms work very similarly to division & reciprocal algorithms. Unless they totally ran out of ROM space and picked a reduction in sqrt(X) precision as the best compromise, this looks like an oversight.

I believe - it's been awhile - that there's a CORDIC-like shift/add algorithm for square root too - each iteration would deliver one digit (either binary bit, hex digit or BCD digit, according to base of numeration system and algorithm flavora) of result. Square root is fairly well-behaved too - unlike asymptotic tan(x) near 90deg and log(x), ln(x) near 0.

Bill Wiese
San Jose, CA




i have also noticed square root is less accurate. if you root with logs you can get more accuracy that the built in square root.
that said, its not necessarily a recommendation for the hp30s.

Possibly Related Threads...
Thread Author Replies Views Last Post
  HP Prime numerical precision in CAS and HOME Javier Goizueta 5 1,091 11-16-2013, 03:51 AM
Last Post: Paul Dale
  wp34s calc to calc communication Andrew Nikitin 1 637 07-29-2013, 01:57 AM
Last Post: Marcus von Cube, Germany
  [wp34s] Converting to/from IEEE 754 Binary64 (Double Precision) David Maier 1 563 06-15-2013, 09:21 PM
Last Post: Paul Dale
  Precision DC-50 Mic 3 841 05-26-2013, 01:27 PM
Last Post: DavidM
  Estimating accuracy in finite precision computations mpi 17 2,035 02-22-2013, 09:44 AM
Last Post: mpi
  50G precision & accuracy Matt Agajanian 11 1,557 05-17-2012, 11:15 AM
Last Post: Crawl
  WP 34S: Double precision registers - What are your feelings? Marcus von Cube, Germany 16 2,054 12-26-2011, 10:49 AM
Last Post: Dieter
  [WP34s] Questions about extended precision fhub 7 1,046 10-28-2011, 05:20 PM
Last Post: Marcus von Cube, Germany
  HP 35s. FIX affects precision Pablo P (Spain) 4 757 10-22-2010, 11:48 PM
Last Post: Karl Schneider
  High precision calculator in Windows 7 jotne 41 4,113 08-21-2009, 09:19 PM
Last Post: Eric Smith

Forum Jump: