calc precision
#1

Which of the HP calculators uses the highest precision?

#2

HP30S (24 digits), except for square root (12 digits)

#3

>>HP30S (24 digits), except for square
>>root (12 digits)

Wow. Why is sqrt(x) lagging? Common square root algorithms work very similarly to division & reciprocal algorithms. Unless they totally ran out of ROM space and picked a reduction in sqrt(X) precision as the best compromise, this looks like an oversight.

I believe - it's been awhile - that there's a CORDIC-like shift/add algorithm for square root too - each iteration would deliver one digit (either binary bit, hex digit or BCD digit, according to base of numeration system and algorithm flavora) of result. Square root is fairly well-behaved too - unlike asymptotic tan(x) near 90deg and log(x), ln(x) near 0.

Bill Wiese
San Jose, CA

#4

nt

#5

i have also noticed square root is less accurate. if you root with logs you can get more accuracy that the built in square root.
that said, its not necessarily a recommendation for the hp30s.



Possibly Related Threads...
Thread Author Replies Views Last Post
  HP Prime numerical precision in CAS and HOME Javier Goizueta 5 591 11-16-2013, 03:51 AM
Last Post: Paul Dale
  wp34s calc to calc communication Andrew Nikitin 1 338 07-29-2013, 01:57 AM
Last Post: Marcus von Cube, Germany
  [wp34s] Converting to/from IEEE 754 Binary64 (Double Precision) David Maier 1 262 06-15-2013, 09:21 PM
Last Post: Paul Dale
  Precision DC-50 Mic 3 430 05-26-2013, 01:27 PM
Last Post: DavidM
  Estimating accuracy in finite precision computations mpi 17 1,131 02-22-2013, 09:44 AM
Last Post: mpi
  50G precision & accuracy Matt Agajanian 11 766 05-17-2012, 11:15 AM
Last Post: Crawl
  WP 34S: Double precision registers - What are your feelings? Marcus von Cube, Germany 16 1,039 12-26-2011, 10:49 AM
Last Post: Dieter
  [WP34s] Questions about extended precision fhub 7 528 10-28-2011, 05:20 PM
Last Post: Marcus von Cube, Germany
  HP 35s. FIX affects precision Pablo P (Spain) 4 373 10-22-2010, 11:48 PM
Last Post: Karl Schneider
  High precision calculator in Windows 7 jotne 41 2,059 08-21-2009, 09:19 PM
Last Post: Eric Smith

Forum Jump: