calc precision - Printable Version +- HP Forums (https://archived.hpcalc.org/museumforum) +-- Forum: HP Museum Forums (https://archived.hpcalc.org/museumforum/forum-1.html) +--- Forum: Old HP Forum Archives (https://archived.hpcalc.org/museumforum/forum-2.html) +--- Thread: calc precision (/thread-27683.html) calc precision - mapet - 02-10-2003 Which of the HP calculators uses the highest precision? Re: calc precision - Ex-PPC member - 02-10-2003 HP30S (24 digits), except for square root (12 digits) Re: calc precision - Bill Wiese - 02-10-2003 >>HP30S (24 digits), except for square >>root (12 digits) Wow. Why is sqrt(x) lagging? Common square root algorithms work very similarly to division & reciprocal algorithms. Unless they totally ran out of ROM space and picked a reduction in sqrt(X) precision as the best compromise, this looks like an oversight. I believe - it's been awhile - that there's a CORDIC-like shift/add algorithm for square root too - each iteration would deliver one digit (either binary bit, hex digit or BCD digit, according to base of numeration system and algorithm flavora) of result. Square root is fairly well-behaved too - unlike asymptotic tan(x) near 90deg and log(x), ln(x) near 0. Bill Wiese San Jose, CA HP 9G? (nt) - mapet - 02-11-2003 nt Re: calc precision - hugh - 02-13-2003 i have also noticed square root is less accurate. if you root with logs you can get more accuracy that the built in square root. that said, its not necessarily a recommendation for the hp30s.