HP Forums

Full Version: HP 35s Standard Deviation
You're currently viewing a stripped down version of our content. View the full version with proper formatting.

I wonder if someone would mind checking the standard deviation calculation on a 35s to see if HP has fixed the algorithm. Enter 1,000,000 , 1,000,001 and 1,000,002 and find the sample standard deviation. The correct answer should be 1 but as has been discussed before the 33s provided an error message.

Thanks,

John

If I'm doing it right, I get: STAT ERROR

...bt

Thanks bt. From my testing it looks like this is done correctly in the 49G+/50G and 17bii+. As discussed previously this problem only shows up when trying to calculate small differences between large numbers.

Regards,

John

I read in some Hp instructions to subtract the large value before doing statistics. It involves the square of the input and clearly the calculator does not have that accuracy. Sam

Please see this previous thread. Besides this, there may be more in earlier discussions.

HTH

48GX gets it right (returns 1) as well.
Regards, Hal

The 19B does it right, too. Maybe because these models do the statistics based on a list?

This is beyond the range of the machine. It doesn't work on a 32sii nor on a 15c either. It is not a flaw.

Hi John,

I don't have a 35S, so can't directly comment on it, but the following is a quote from page 237 of the HP-42S manual that addresses this:

```Limitations on Data Values
The calculator might be unable to perform some statistical
calculations if your data values differ by a relatively
small amount.  To avoid this, you should normalize your
data by entering values as the difference from one value
(such as the mean).  This difference must then be added
back to any calculations of the mean.  For example, if
your x-values were 776999, 777000, and 777001, you should
enter the data as -1, 0, and 1;  then add 777000 back to
the relevant results.
```

Does the HP-35S manual include any paragrah such as this?

Bill

Edited: 19 Sept 2007, 3:22 p.m.

Yes, on page 12-10.

Gene

Most calculators will fail this kind of calculations, unless you shift the values into a more appropriate range.

Namir

The same on the HP-15C manual.

-- Antonio

Free42 correctly calculates it. ;-)

Also, the 17bii and HP-50g calculate it correctly.

thanks,
bruce

Edited: 19 Sept 2007, 4:09 p.m.

Like many HP calculator issues, the different methods used to determine standard deviation have been discussed here before. This long thread started out as a discussion of the 33S polar conversion bug, but digressed into a discussion of standard deviation.

The HP-41 yields the message "OUT OF RANGE" for the given problem.

Page 101 of the HP-41C Owner's Handbook and Programming Guide includes the statement

"Note: If your data {xi} or {yi} contains many redundant leading digits, you should refrain from copying them into the calculator. For example, if your x-data is {999999999, 1000000001, 1000000002}, you should enter the x-data as {-1, 1, 2} and add the redundant digits to any x-related answer produced."

Most will fail but not all. My HP 30s with s/n CN0351 yields the correct answer.

Regards,

John

John Smitherman posted,

Quote:
Enter 1,000,000 , 1,000,001 and 1,000,002 and find the sample standard deviation. The correct answer should be 1 but as has been discussed before the 33s provided an error message.

Bill Platt replied,

Quote:
This is beyond the range of the machine. It doesn't work on a 32sii nor on a 15c either. It is not a flaw.

Bill is right; the "STAT ERROR" or "Error 2" message is attributable to roundoff errors in the statistical summation. What's ironic is that the failure to produce a result was made possible by extended-precision calculation, albeit using standard-precision inputs.

Here are the tabulations:

```Xi         (Xi)^2 (exact)    (Xi)^2 (result retaining 10 or 12 significant digits)
1,000,000  1,000,000,000,000  1,000,000,000,000
1,000,001  1,000,002,000,001  1,000,002,000,000
1,000,002  1,000,004,000,004  1,000,004,000,000
Summations:
S(X)       S(X^2) (exact)     S(X^2) (using 10 or 12-digit results)
3,000,003  3,000,006,000,005  3,000,006,000,000
([S(X)]^2) (exact result obtained using 13- or 15-digit extended precision)
9,000,018,000,009
([S(X)]^2)/3 (exact result obtained using 13- or 15-digit extended precision)
3,000,006,000,003
```

The exact calculation

sqrt ([S(X^2) - ([S(X)]^2)/n] / [n-1]) of sample standard deviation is

sqrt [(3,000,006,000,005 - 3,000,006,000,003) / (3-1)] = 1

The "erroneous" calculation is based on 13- or 15-digit internal computaion that utilizes the S(X^2) summations that were rounded off after each datum entry:

sqrt [(3,000,006,000,000 - 3,000,006,000,003) / (3-1)] = ERROR

(The HP-15C's extended-precision calculations for statistical summations are utilized within an example in the HP-15C Advanced Functions Handbook listed on pages 208-211.)

If extended precision had not been employed for the problem discussed here, both terms would have been rounded to 3,000,006,000,000 -- yielding the incorrect result of zero, instead of an error.

-- KS

Edited: 26 Sept 2007, 2:09 a.m. after one or more responses were posted

This exercise gave me new respect for my 38G, which gave the correct result.

You wrote:

"Use of extended-precision calculations for standard deviation is described in the HP-15C Advanced Functions Handbook on page 208."

Page 208 of my August 1982 copy only says "Program B exploits a tricky property of the Sum- and Sum+ keys whereby certain calculations can be carried out to 13 significant digits before being rounded back to 10."

Is there a later revision which gives specific details for the calculatioin of standard deviation?

Quote:
Most will fail but not all. My HP 30s with s/n CN0351 yields the correct answer.

Actually, it is many HP calculators which will fail because they carry only ten digits. Calculators from other companies such as the Casio fx-7000G, the TI-59, TI-66, TI-68, TI-95 and Durabrand 828 which carry 13 digits will not fail this particular test. They will, of course, fail tests which require more than thireen digits.

I wonder, was this test proposed by a TI Guy?

Hi, Palmer --

No, my version of the HP-15C AFH gives no further details. I realized after posting that "described" was not a very good term, but didn't edit before you responded. I have edited the post now.

I was also well aware that these methods were the basis of your "Cadillac Quadratic Solver for the hp 33s" article from last year:

-- KS

Thanks Palmer. Through brute force I found that most fail testing 10,000,000 , 10,000,001 and 10,000,002 while the HP 50G and 17Bii+ fail at 1,000,000,000,000 , 1,000,000,000,001 and 1,000,000,000,002.

Regards,

John

Thanks Karl. As usual your post was very thorough and insightful. Perfect benchmarks are not easy to find. I believe that users want HP and other calculators manufacturers to provide accurate and robust algorithms that make the best use of the hardware selected. It's interesting how tests keep showing up from various users that highlight where some machines fall short of our expectations, especially near or around the limits of the machine.

Regards,

John

Quote:
No, my version of the HP-15C AFH gives no further details.

I am sorry about that. I was hoping for something more definitive on the "tricky properties" than what I had been able to figure out by trial and error.

Many years ago TI was very close-mouthed about the implementation of the statistcs routines in the TI-59 until the user community figured out how to download the instruction set. Then they published a discussion of the routines. The routines did include an HIR 20 command which was some sort of branching command which could not be made to work in user memory -- or at least, no one ever figured out how to do it. The TI discussion included mention of the HIR 20 command, but their explanation was about as informative as a Milton Friedman dissertation on the state of the economy.

Quote:
... while the HP 50G and 17Bii+ fail (to calculate sample standard deviation of the data) 1,000,000,000,000 , 1,000,000,000,001 and 1,000,000,000,002.

Hi again, John --

The last two of these data require 13 digits, which cannot be represented as non-integer values on these calculators. The calculation will fail even if summation methods are not used. Does this mean that the HP-17Bii+ accepts excess digits, as the RPL-based models have done, and now the HP-35s does?

If so, more ammunition for my case...

-- KS

Hi Karl. In your note you asked:

Quote:

Does this mean that the HP-17Bii+ accepts excess digits, as the RPL-based models have done, and now the HP-35s does?

If so, more ammunition for my case...

To further support your case when I enter 1,000,000,000,002 the 49G+ and 17bii+ display 1E12. I then subtract 2 and they both display 999,999,999,998. So, the excess digits are being dropped.

Regards,

John

Quote:
the HP 50G and 17Bii+ fail at 1,000,000,000,000 , 1,000,000,000,001 and 1,000,000,000,002.

My TI-83+ yields the correct results for that but fails at 1E13, 1E13 + 1 and 1E-13 + 2 . I am fairly sure that it does not calculate SDEV using the summations but rather uses the sum of the squares of the differences from the mean.

My TI-86 yields the correct results for 1E13, 1E13 + 1 and 1E13 + 2 but fails at 1E14, 1E14 + 1 and 1E14 + 2.

Edited: 25 Sept 2007, 12:50 a.m.

Quote:
I was hoping for something more definitive on the "tricky properties" than what I had been able to figure out by trial and error.

Without scrutinizing the program on the HP-15C AFH, pp. 208-211, I do notice a few things:

Only Sigma- is used; each time it is, the contents of register R7 (which contain the XY summation) are recalled.

The quadratic equation is defined on page 205 as c - 2bz + az2, making the discriminant d = b2 - ac

The basic purpose appears to be the calculation of "b2-ac" and similar terms to 13-digit extended precision before being rounded back down to 10 digits. This procedure increases accuracy when the terms in the subtraction are nearly equal.

It's kind of related to the residual matrix calculation (B - AC).

-- KS

Edited: 4 Oct 2007, 2:33 p.m.