RM mode and its effects on RRM determinant of AM1 matrix
#1

RM Mode       Determinant
------- ---------------------
0 0.99999 711935 38228
1 0.99999 720675 30552
2 0.99999 709507 09280
3 1.00000 009178 43880
4 0.99999 711925 74617
5 1.00000 895891 46320
6 0.99999 718613 18536

Looks like RM 3 is closest to 1. Fairly significantly closer, it appears.

#2

RM 3 is round half up. It being closer is most likely a fluke.

This does show that the algorithm isn't overly sensitive to rounding mode which is goodness.


- Pauli



Possibly Related Threads…
Thread Author Replies Views Last Post
  AFTER HP-Prime update, Shift+Matrix CRASHES Joseph Ec 3 2,002 12-06-2013, 11:06 AM
Last Post: Joseph Ec
  Integration question and "RPN" mode comment Craig Thomas 16 5,713 12-05-2013, 02:32 AM
Last Post: Nick_S
  HP Prime Matrix TERRIBLE bug and question uklo 19 4,963 11-25-2013, 12:10 PM
Last Post: Mic
  HP Prime: editing a matrix Alberto Candel 6 2,300 11-20-2013, 06:26 PM
Last Post: Helge Gabert
  HP Prime: Recommendation for future RPN Program Mode BruceTTT 3 2,103 11-13-2013, 10:03 PM
Last Post: BruceTTT
  Absolute Value and Matrix BruceTTT 5 2,055 11-11-2013, 11:52 PM
Last Post: Walter B
  flood side-effects (not completely OT) Kiyoshi Akima 6 2,580 11-05-2013, 12:06 AM
Last Post: Kiyoshi Akima
  Side effects of the HP Prime Harald 9 2,878 11-04-2013, 12:01 PM
Last Post: Harald
  Program to change entry mode on Prime Michael de Estrada 3 1,840 10-28-2013, 10:13 AM
Last Post: Han
  HP Prime: Rounding error in determinant Stephan Matthys 3 1,607 10-25-2013, 09:29 PM
Last Post: Walter B

Forum Jump: