Posts: 653
Threads: 26
Joined: Aug 2010
I just noticed that SLVQ may return results with slight errors even in relatively simple cases:
-1 ENTER
-1 ENTER
1 SLVQ
WP34s results correct (16 digits)
0,61803 39887 49895 0 0,61803 39887 49894 8
1,61803 39887 49894 1,61803 39887 49895
With 39 internal digits a correct 16-digit result should be no problem. So what's going on here?
Dieter
Posts: 3,283
Threads: 104
Joined: Jul 2005
If I'm not mistaken, SLVQ is not implemented natively but in user code (XROM). So the normal round-off errors may occur.
Posts: 3,229
Threads: 42
Joined: Jul 2006
Solve is a user code program so much work isn't carried out in high precision (some is still). It also searches at display accuracy (i.e. using the approximately equal test).
Thus, some lower order digits errors are to be expected.
- Pauli