calc precision
#1

Which of the HP calculators uses the highest precision?

#2

HP30S (24 digits), except for square root (12 digits)

#3

>>HP30S (24 digits), except for square
>>root (12 digits)

Wow. Why is sqrt(x) lagging? Common square root algorithms work very similarly to division & reciprocal algorithms. Unless they totally ran out of ROM space and picked a reduction in sqrt(X) precision as the best compromise, this looks like an oversight.

I believe - it's been awhile - that there's a CORDIC-like shift/add algorithm for square root too - each iteration would deliver one digit (either binary bit, hex digit or BCD digit, according to base of numeration system and algorithm flavora) of result. Square root is fairly well-behaved too - unlike asymptotic tan(x) near 90deg and log(x), ln(x) near 0.

Bill Wiese
San Jose, CA

#4

nt

#5

i have also noticed square root is less accurate. if you root with logs you can get more accuracy that the built in square root.
that said, its not necessarily a recommendation for the hp30s.



Possibly Related Threads…
Thread Author Replies Views Last Post
  HP Prime numerical precision in CAS and HOME Javier Goizueta 5 2,353 11-16-2013, 03:51 AM
Last Post: Paul Dale
  wp34s calc to calc communication Andrew Nikitin 1 1,141 07-29-2013, 01:57 AM
Last Post: Marcus von Cube, Germany
  [wp34s] Converting to/from IEEE 754 Binary64 (Double Precision) David Maier 1 1,102 06-15-2013, 09:21 PM
Last Post: Paul Dale
  Precision DC-50 Mic 3 1,495 05-26-2013, 01:27 PM
Last Post: DavidM
  Estimating accuracy in finite precision computations mpi 17 4,241 02-22-2013, 09:44 AM
Last Post: mpi
  50G precision & accuracy Matt Agajanian 11 2,953 05-17-2012, 11:15 AM
Last Post: Crawl
  WP 34S: Double precision registers - What are your feelings? Marcus von Cube, Germany 16 4,212 12-26-2011, 10:49 AM
Last Post: Dieter
  [WP34s] Questions about extended precision fhub 7 2,111 10-28-2011, 05:20 PM
Last Post: Marcus von Cube, Germany
  HP 35s. FIX affects precision Pablo P (Spain) 4 1,456 10-22-2010, 11:48 PM
Last Post: Karl Schneider
  High precision calculator in Windows 7 jotne 41 8,738 08-21-2009, 09:19 PM
Last Post: Eric Smith

Forum Jump: