Rounding conventions « Next Oldest | Next Newest »

 ▼ Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-04-2009, 09:16 AM Dear All, I'm sure this must have been discussed before but I couldn't find it so apologies if it's a repeat. When trained as a scientist many years ago I was taught that when rounding numbers the convention was to round 5's to the nearest even number, thereby averaging out any introduced error when working with data. Everything I've seen and done in science in the last 30 years has reinforced this convention. I've only just found out that rounding away from zero (i.e., rounding up for positive numbers) is also an acceptable convention according to IEEE Standard Arithmetic. While this may be ok for accounting and finance, it introduces a bias when rounding data sets, and would appear to be unsuitable for many scientific applications especially those involving sets of data. I've used numerous calculators (nearly all HP) and they all round away from zero even calculators clearly designed for scientists and engineers like the 41C and 15C. I've also looked through a few of my manuals and I can't find any real discussion on the subject and there doesn't seem to be any setting or flag to allow the user to choose between the rounding methods. This is very inconvenient because it means that the number displayed is not the true rounded number (according the the scientific round to even method). So, my question is, why do all these calculators, even the scientific ones, use this round away from zero method, and is there any way around the problem (other than witting a program)? Are there any calculators on the market that round "properly" for a scientist? Thanks, Wayne ▼ Gerson W. Barbosa Posting Freak Posts: 2,761 Threads: 100 Joined: Jul 2005 07-04-2009, 11:53 AM On the HP-71B you can choose one out of four rounding methods: ```OPTION ROUND NEAR (default) OPTION ROUND ZERO rounds towards zero OPTION ROUND POS rounds up OPTION ROUNT NEG rounds down ``` These can also be selected by clearing or setting the appropriate flags (-11 and -12). See HP-71B Owner's Manual, pages 56 and 198. For more on HP rounding philosophy refer to and Regards, Gerson. Edited: 4 July 2009, 11:54 a.m. ▼ Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-04-2009, 12:22 PM Thanks Gerson, I have a 71B and this still doesn't round the display "properly". The 71B (like other calcs) still appears to use "round away from zero" for the display when the last digit is 5. For example, enter 2.005 with a fixed 2 decimal place format and it will show 2.01 instead of what most scientists would want, 2.00. It seems to me from checking p56 of the manual that "OPTION ROUND NEAR" only dictates how the 71B rounds the final 12th digit, not how it rounds the currently viewed display (eg, fixed 2). Cheers Wayne ▼ Quan Member Posts: 69 Threads: 5 Joined: May 2009 07-04-2009, 01:03 PM Hi Wayne, I am curious as to why you or most scientists would want to round down to 2.00 instead round up to 2.01? If there was a reason, I surely cannot recall as I can only remember to round up. Last year I helped my niece (3rd grade) with her homework, and the school also taught rounding up. Thanks. Quan ▼ Gerson W. Barbosa Posting Freak Posts: 2,761 Threads: 100 Joined: Jul 2005 07-04-2009, 01:45 PM Quote: I am curious as to why you or most scientists would want to round down to 2.00 instead round up to 2.01? Please follow the link below and take a look at Round-to-even method: Regards, Gerson. Don Shepherd Posting Freak Posts: 1,392 Threads: 142 Joined: Jun 2007 07-04-2009, 01:45 PM Schools teach it this way: if the digit to the right is 0, 1, 2, 3, or 4, leave the rounding-to digit alone; if the digit to the right is 5, 6, 7, 8, or 9, add 1 to the rounding-to digit. So, five digits go one way, five digit go the other way. Sounds fair to me. I thought that is how all calculators do it too. ▼ Thomas Radtke Posting Freak Posts: 1,089 Threads: 32 Joined: Dec 2005 07-04-2009, 02:55 PM Don, 0.5 is the average of 0 and 1. If you always round it up, you'll introduce a bias. If your rounding depends on whether the INT is even or odd, you get rid of the bias in most cases. That's how I understand it. I might be wrong though :^). ▼ Walter B Posting Freak Posts: 4,587 Threads: 105 Joined: Jul 2005 07-04-2009, 05:45 PM That's what I like at this forum: There's a good chance I learn something I didn't know before. I admit I had no idea about round-to-even and the reasons for it (or have forgotten that completely), though having a Ph.D. in physics and having worked with large data sets (way back). IMHO, in real life the error introduced by rounding 5 up (as I did it so far all the time) seems neglectable as long as you discard more than one digit in rounding. Don Shepherd Posting Freak Posts: 1,392 Threads: 142 Joined: Jun 2007 07-04-2009, 06:12 PM So, 92.5 would be rounded to 92 (nearest even number). OK, one of you engineers will have to come to my math class and explain to my students why a 92.5 average results in a B now, rather than an A, on their report cards (standard range for A is 93-100). Jeesh, no wonder kids hate math! : ) ▼ Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-04-2009, 07:05 PM Quote: So, 92.5 would be rounded to 92 (nearest even number). OK, one of you engineers will have to come to my math class and explain to my students why a 92.5 average results in a B now, rather than an A, on their report cards (standard range for A is 93-100). Jeesh, no wonder kids hate math! : ) A 92.5 average should get a B, rather than an A, shouldn't it? They don't hate averaging, do they? Rounding is easier than averaging. You can't round to 2 digits before you do the comparison "IF score < 93 THEN grade = B. Katie Wasserman Posting Freak Posts: 1,477 Threads: 71 Joined: Jan 2005 07-04-2009, 03:27 PM Actually, I'm not so sure that it's ok in finance either. I work on wall street and I've never personally seen any programs using the round-to-even method, but they call it "Banker's Rounding" here which must mean that some financial places do that. I can't think of a single hand-held calculator (not even the HP-50g) that has round-to-even as a possible rounding method. But some large math packages have this; Mathematica's "rint" function, for example. Edited: 6 July 2009, 2:25 p.m. after one or more responses were posted ▼ Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-04-2009, 07:08 PM Quote: Actually, I'm not sure sure that it's ok in finance either. I work on wall street and I've never personally seen any programs using the round-to-even method, but they call it "Banker's Rounding" here which must mean that some financial places do that. I can't think of a single hand-held calculator (not even the HP-50g) that has round-to-even as a possible rounding method. But some large math packages have this; Mathematica's "rint" function, for example. All the Saturn based calcs do round-to-even arithmetic (except the HP71 can do something else if selected); it's just the display rounding that isn't round-to-even ▼ Katie Wasserman Posting Freak Posts: 1,477 Threads: 71 Joined: Jan 2005 07-05-2009, 12:25 AM I thought that we're talking about display rounding in this thread. The original post said: Quote: While this may be ok for accounting and finance, it introduces a bias when rounding data sets, and would appear to be unsuitable for many scientific applications especially those involving sets of data. The "data sets" imply that the data was from an experiment or whatever and input into the calculator. So I thought that the discussion has been about display rounding on input not rounding of the internal functions to their maximum accuracy. Edited: 5 July 2009, 12:31 a.m. ▼ Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-05-2009, 03:51 AM But display rounding on input has no effect on the data that ends up in a data set in the calc (to be processed further, presumably). The data could have been captured from an HPIL device, such an HP3468 voltmeter. In that case, clearly display rounding has no effect on the captured data. Even if the data were typed in, the fact that the typed-in data appears different in the display doesn't change what is stored in a variable. For example, if the calc is in FIX 2 mode and you type in 2.005, it doesn't matter that the display shows 2.01; the number upon which arithmetic will be performed is still 2.005 The display rounding only matters when the human operator is looking at the final result. ▼ Katie Wasserman Posting Freak Posts: 1,477 Threads: 71 Joined: Jan 2005 07-05-2009, 12:28 PM How about if assume that the user enters the data via a program (keystroke entry or HP-IL device) and that the program uses the Round Function in the calculator (usually marked as RND on HP calculators) to force the data to N decimal places as set by the display format. I think that this would be a likely scenario for entering data into a calculator and display rounding is important since RND works based on display rounding. Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-06-2009, 08:05 AM Just to put this into the context of my real life situation. I was using my calculator to calculate the purity of a number of samples of a chemical. There are a number measurements (weight, volume, detector response, etc) for each purity calculation which I manually enter into the calculator to produce the answer. The number of significant figures I need (based on errors involved in the measurements) is 4, ie xx.xx% So I set my calculator to FIX 2, and just read off the (rounded) answers. I report these individual purity measurements separately. At this point all the answers have been rounded up which is probably no real issue but then I want to average these purities and this is when the bias introduced by rounding up (rather than to even) becomes an issue because the average purity will be slightly higher than it really should because all the 5's have been rounded up. I realise the error will be small but it could (and should) be removed entirely by simply rounding to even. I hope this makes sense and explains why this is a real problem. You may well ask why didn't I just use a spreadsheet for the whole process but this is another story ....... ▼ Garth Wilson Posting Freak Posts: 887 Threads: 9 Joined: Jul 2007 07-06-2009, 03:08 PM Makes sense; but even if you don't want the calculator to keep all the values simultaneously and then average them at the end, you could still have the calc keep a running average and avoid the problem. One register keeps the total of the results, and another one keeps the number of results added together to get that total, so at any time you can divide one by the other to get the average so far, to as many digits as you want, even if individual purities were dispalyed in only FIX 2. ▼ Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-07-2009, 10:42 AM Thanks Garth That would certainly solve the problem in many cases. But there are a couple of issues that still remain. I should also point out that as well as needing the average, I also needed the standard deviation because I had to multiply this by a factor (based on the number of samples) to give an error to a 95% confidence limit. Firstly, I was using a HP21 which, with only 1 memory register and no Stats, wasn't capable of doing what you suggest. Ok, I could have reached into my calculator cupboard and pulled out any number of calculators which could have handled this easily so it's not really much of an excuse. Secondly, and this is my main concern, I have collaborations with biologists, pharmacologists, computational chemists, and other disciplines. We routinely exchange data sets back and forth for various types of analysis. Naturally, we only send/receive the data with the appropriate number of significant figures, ie rounded data. So now we have (or sent) a biased data set which then gets used in further analysis. I accept that any errors will be small and in most circumstances will not have a material effect on the results. But I think the point is that, if the rounding were done "properly" then there would be no issue at all. Thirdly, and this is the actual event which highlighted the issue to me. I had to report the data and the average result to the client. So the option was to report the average of the rounded data (even though this was slightly higher than the real average) or report the real average based on the unrounded data even though this wasn't exactly the average of the reported results. It sounds rather pedantic but the report was to be presented in court in relation to a patent dispute. If the report had even a minor inconsistency like this, it could impact on our credibility as an expert witness. In the end I just manually rounded the data using the round to even method and that was fine. But it did strike me as odd that scientific calculators couldn't round "properly" and hence my post on this forum. I was hoping that there might be some undocumented trick such as setting a flag but it seems not. I'll be the first in line to buy the first calculator than can round to even! It's true that most of our "number crunching" these days is done in spreadsheets but there's nothing quite like running the numbers through a calculator yourself to really have faith in the answer. ▼ Gerry Schultz Senior Member Posts: 255 Threads: 59 Joined: Jul 2007 07-07-2009, 01:07 PM Wayne: I have been reading this thread for the last couple of days. It has made me think about rounding conventions after many years and this morning I was struck with the thought of why are you using an HP calculator to process this data? I can understand wanting to use an HP calculator to manually process the data sets to confirm what an Excel spreadsheet has done, but in this discussion, you have implied that you are passing data processed in the calculator on to your coworkers. I am the first to admit that I know nothing about what you do or how careful you need to be with your data. But, if this data is going to be used in a legal proceeding and any question is raised by an attorney about how you reached you expert conclusions, I would think you would want a well documented trail to deflect any question of credibility. I lived with an attorney for 5 years and she always looked for any means to discredit an expert witness in a trial. My thought, in my tiny mind, is that an HP calculator should be used as an independent means to confirm the validity of your Excel processed data sets which increases you confidence when on the stand and if asked if you confirmed this data you have that HP calculator ace in your back pocket and have dealt with any potential rounding errors. It's not that HP calculators are not "good" enough to do this work, it's that if your legal opponent catches wind that you used a calculator to process this data and he was technical enough (and some attorneys are) he could ask about rounding errors during cross-examination and place doubt in the eyes of a naive jury. Just ask any criminal attorney about what the CSI TV shows have done to the public's perspective on how crime scenes are processed. Then stand back and be prepared to cover your ears. Others in this thread have covered well the issue of rounding errors and your posts show that you are rightly thinking about this issue. I've enjoyed this thread as it has made me think about things I have taken for granted or forgotten. I always look forward to the next posted question. What else we will all learn? Gerry Jeff Kearns Member Posts: 222 Threads: 21 Joined: Oct 2006 07-07-2009, 01:50 PM I too have been following this thread with interest and I found it odd that the data sets are not analyzed in Excel or even better, a proper statistical analysis package like Minitab or Statistica. On the one hand, there is the issue of expert witness credibility and on the other hand the desire (for whatever reason, albeit not fully explained) to use a time-consuming handheld calculator, with no 'paper trail' to show exactly how the data were processed. For your sake, I hope that there are not more a handful of data pairs... Would you be willing to share examples of the data with the forum in such a fashion that we are unable to determine the units or purpose - just for discussion purposes? Jeff Kearns ▼ Jeff Kearns Member Posts: 222 Threads: 21 Joined: Oct 2006 07-07-2009, 01:59 PM My curiosity is really piqued now... I have analyzed some rather interesting data sets in my day (LeanSigma practitioner) and have always been impressed with the rapidity and reliability of a HP 49G+ or a 50G to calculate summary statistics, confidence intervals and conduct hypothesis testing on relatively small data sets (>20 pairs). Just how many of your data points end in 5's anyway? What is the worst case difference (statistically speaking) in mean values between the two rounding methods for your data sets? It seems to me that the difference would be negligible. I need to see the data!!! Jeff Edited: 7 July 2009, 2:03 p.m. ▼ Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-08-2009, 12:14 PM Dear Jeff and others Thanks for your comments. I was trying to keep my description of the problem and how it arose simple so as not to confuse or bore people but here is a bit more background for those that are interested in the origin. Unfortunately, we don't have any nice stat packages and we probably couldn't justify the cost as most of our number crunching is pretty basic stuff. And no, I'm neither a troglodyte nor masochist, I do most of this sort of work in a spreadsheet, not a calculator :) I use OpenOffice not MS Excel but I don't think there's much difference. OpenOffice, like the calculators, does not round to even but rather rounds all the 5's up. In my "real life" example The actual data to be analysed was relatively small, 6 standards (in triplicate) to calculate a standard response (by linear regression) and then 8 samples (in triplicate) which were fitted to the regression to calculate concentration from which purity was finally calculated. There was several stages involved in the calculations with the early stages being manually recorded in a lab book before being transferred to a spreadsheet for further calculations and linear regression. The final spreadsheet was then brought to me for checking. I like to check results ab initio by an independent process rather than just check what has been done. As the data set was relatively small I thought I'd just check a few selected preliminary calculations manually by calculator (HP21) and then check the regression and line fitting for which I pulled out my 48GX (I wasn't aware that it could calculate confidence limits so thanks for the tip, I'll look into that). This is the point at which the saga began to unfold because my 8 manually calculated purity results did not agree with the spreadsheet. It took some time to find the source of the discrepancy between the two methods which had nothing to do with rounding up versus rounding to even but an inappropriate manual rounding of the slope and intercept in the spreadsheet was part of the problem. However, in tracking down the problem I started to take a closer look at potential sources of error and the rounding issue arose. I think the maximum error is only 1 in the final significant digit. So if you have 4 significant digits the error should only be between 1 in 1,000 to 1 in 9,999 but if you are rounding to only 2 significant figures it could in theory be as high as 1 in 10 (it's just gone midnight here and my mind is getting a little fuzzy so you might want to check that :)) From this simple data set (2.05, 2.15, 2.25, 2.35, 2.45, 2.55) you can see that the error is 0.1 in 2.3 or just over 4%, not insignificant at all. Ok, it does contain a lot of 5s. You ask "how many numbers end in 5 anyway" and this really does depend on where the numbers come from and how many significant digits you drop off a t a time. I guess from normal calculated numbers one would expect only 10% of numbers would end in five but for some visual measurements 5s are very common, for example melting points are normally given to the nearest 1/2 degree so every second measurement might end in 5. I need to look into this a bit more and see what the typical error might be in a more typical data set and significant figure reduction. I suspect it only really becomes significant if one reduces to 1 (not very usual) or 2 (not uncommon) significant figures. Ironically, I think the problem is worse when you only drop one figure because 1 in 10 are 5s whereas if you drop 2 at a time the number of 50's is only 1 in 100. I hope that makes sense. Also, I think I have found one example of round to even....the HP-95LX has a RND function which seems to do this but I need to look at this in more detail as I don't have a manual for this calc. Thanks all ▼ Katie Wasserman Posting Freak Posts: 1,477 Threads: 71 Joined: Jan 2005 07-08-2009, 01:27 PM I tried the RND function on my 95LX and 200LX and I don't think that it works any differently than the RND function on all the other HP calculators that I've tried. Perhaps I'm doing something wrong, is there a particular reason why you think it does a round-to-even on the 95LX? -Katie ▼ Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-09-2009, 08:52 AM Hi Katie I'm pretty sure my 95LX is using "round to even" . I am using the calculator feature of the device and have the format set to FIX 2. If I input 2.005 and presse ENTER the display shows 2.01 as all the other calculators do (of course the number is still 2.005). If I then press the RND function (F2) the display changes to 2.00, ie it has actually rounded down to even. The only downside of this is that it actually changes the number and not just the way it's displayed. It would be nice to keep the original number but just changed the display rounding. When I try the same procedure using the RND feature of the 41CV or 15C the result is 2.01 not 2.00 so it seems that the 95LX is the odd one out here. I wonder why this is? Perhaps it has something to do with it running on a DOS system? Unfortunately, I don't have a manual for the 95LX. Give it a try and let me know if it works for you, perhaps there is an option on the 95LX for this? Regards Wayne ▼ Thomas Radtke Posting Freak Posts: 1,089 Threads: 32 Joined: Dec 2005 07-09-2009, 03:02 PM It appears to be the default method of the 95LX calculator application. Interesting. Edit: I almost forgot I have the manual. The RND description is wrong. It states that the number is rounded to correspond with the number displayed. Edit 2: Serial ABD3128A03721 Edited: 11 July 2009, 3:44 a.m. Katie Wasserman Posting Freak Posts: 1,477 Threads: 71 Joined: Jan 2005 07-10-2009, 09:31 PM Wayne, I just tried your example on my 95LX and it doesn't work as yours does. Setting format Fix 2 and entering 2.005 will show 2.01 certainly. But when I press RND (F2 on the MATH menu) the X register stays at 2.01. I get the same results in ALG mode as I do in RPN mode I don't think that there are any other options that can be set in the calculator that would effect rounding. Could your 95LX and mine be different ROM versions with a changed Calc program? My 95LX is the last release based on the information found here. The serial numbering prefix found on this same page does not agree with the number of my machine, it's SG30300278 which indicates that it was made early in 1993 I think. -Katie ▼ Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-11-2009, 07:15 AM Hi Katie Gee, that's really odd. Mine is a USA made model S/N ABA3139A00067 and is the early 512k version. I understand (from the excllent link in your post) that the calculator applet is based on the 19BII which also rounds up not even. I wonder if mine is an earlier version which came with round to even which was "fixed" in later models. Or perhaps mine is broken or I'm doing something wrong? I'd be interested to hear from other 95lx owners about the behaviour of theirs. Regards Wayne Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-07-2009, 03:31 PM Quote: It's true that most of our "number crunching" these days is done in spreadsheets but there's nothing quite like running the numbers through a calculator yourself to really have faith in the answer. Does Excel round-to-even? The answer to your original question is that HP calcs don't do what you want when display rounding is in effect. However, if you want a properly rounded-to-even result with a fixed 2 digits after the decimal point that will also display properly when the calc is in FIX 2 mode, 6 keystrokes will do it on a Saturn based calc: EEX 9 + EEX 9 - You could assign this as a tiny program to a used defined key, if the calc has that ability. Otherwise, just remember it. And, of course, if you want a different number of digits after the decimal point, adjust the constant 9. ▼ Katie Wasserman Posting Freak Posts: 1,477 Threads: 71 Joined: Jan 2005 07-10-2009, 09:42 PM Quote: Does Excel round-to-even? No. Microsoft calls in banker's rounding. Read this quite extensive writeup about rounding in VBA, Excel and in general. Edited: 10 July 2009, 9:44 p.m. ▼ Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-11-2009, 12:47 AM What I see on that page is: ```Product Implementation ---------------------------------------------------------------------- Visual Basic for Applications 6.0 Banker's Rounding Excel Worksheet Symmetric Arithmetic Rounding SQL Server Either Symmetric Arithmetic Rounding or Symmetric Round Down (Fix) depending on arguments Java Math library Asymmetric Arithmetic Rounding ``` It looks to me like you would only get Banker's rounding in Excel within a VBA application. Do you read it differently? ▼ Katie Wasserman Posting Freak Posts: 1,477 Threads: 71 Joined: Jan 2005 07-11-2009, 02:38 AM Exactly, you'd have to write in VBA to get banker's rounding in Excel. There's no function within Excel itself for that, although it does have ROUNDUP, ROUNDDOWN and regular ROUND. Werner Member Posts: 163 Threads: 7 Joined: Jul 2007 07-13-2009, 07:28 AM or, as I use it: ```2 E9 + LASTX - ``` Cheers, Werner ▼ Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-14-2009, 01:14 PM Why 2 E9 rather than 1 E9? ▼ Werner Member Posts: 163 Threads: 7 Joined: Jul 2007 07-15-2009, 01:33 PM so that it works for negative numbers as well, eg: ```-55.451 + 1e9 - 1e9 => -55.451 but -55.451 + 2e9 - 2e9 => -55.45 ``` Cheers, Werner ▼ Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-16-2009, 02:02 AM Of course. I assumed without good reason that he would be rounding positive data. I should have mentioned that for negative numbers, he should use the sequence EEX 9 - EEX 9 +. I do all my calculator programming on an HP50 these days, and to enter 2E9 takes 1 more keystroke than to enter 1E9; that is, 2 EEX 9 versus EEX 9. And, the HP50 has no LASTX key; LASTARG isn't available as a key on the non-user-shifted keyboard. I happened to look at an old thread on rounding from 2003, and I think it might be fun to revisit it. I'll start a new thread. Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-04-2009, 08:43 PM Thanks Katie I'ver never heard it called Banker's rounding before but now I have a new respect for the financial sector. I thought the banks would always want to claim that extra half cent for themselves :) Regards, Wayne Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-04-2009, 06:48 PM You said: Quote:While this may be ok for accounting and finance, it introduces a bias when rounding data sets, and would appear to be unsuitable for many scientific applications especially those involving sets of data. The HP71 and later Saturn based calcs round-to-even (unless you set the rounding mode in the HP71 to something else) during arithmetic calculations, so this problem only shows up in the display of final results. One way around it without doing any programming is to set your calculator to display all 12 digits in the result and do the rounding to a smaller number of significant digits yourself. Or, if you have a large number of final results to print at some lower number of significant digits, you can cause that number of digits to be rounded-to-even using a technique I described on comp.sys.hp48 in 2004: ------------------------------------------------------------ Imagine that you want to round a number on your calculator to a reduced number of digits, smaller than the calculator's full precision, with proper rounding (round-to-even, that is!). If the calculator does N digit arithmetic and you want a K digit rounded result, R, the formula: R = value - value * 10^(N-K) + value * 10^(N-K) or R = value + value * 10^(N-K) - value * 10^(N-K) gives the K digit properly rounded result where "value" is the original full-precision number you want to round to K digits. These formulas use Dekker's method, and only work correctly if the calculator does proper rounding during arithmetic. For example, on the HP48, if we have the number 123450000001 in the display and want to round this to 4 digits, we compute: 123450000001 - 123450000001 * 1E8 + 123450000001 * 1E8 and we get: 123500000000 But if my starting value is 123450000000 and I compute: 123450000000 - 123450000000 * 1E8 + 123450000000 * 1E8 we get: 123400000000 The result is rounded to even. Now start with 123550000000 and compute: 123550000000 - 123550000000 * 1E8 + 123550000000 * 1E8 we get 12360000000, and we see that the result is rounded to even, as it should be because the discarded digits were 50000000..., that is, a 5 followed by only zeroes. Try reversing the order of the addition and subtraction in the last example: 123550000000 + 123550000000 * 1E8 - 123550000000 * 1E8 we still get: 123600000000 Now try this with a TI86 calculator: 123550000000 - 123550000000 * 1E10 + 123550000000 * 1E10 (use 1E10 as the multiplier because the TI is a "14" digit machine) we get 123500000000 (only 12 digits are shown) But if we try: 123550000000 + 123550000000 * 1E10 - 123550000000 * 1E10 we get 123600000000 on the TI86. So Dekker's method fails on the TI. ▼ Wayne Best Junior Member Posts: 12 Threads: 2 Joined: Oct 2008 07-04-2009, 09:17 PM Thanks Rodger Yes, you're right, it's all about the rounding on the display. As a user, that's all I see. I take it for granted (perhaps naively) that the precision of the calculations are fine and I cannot recall a single real life example where I've needed all the precision of a modern calculator. So I guess most of us set the display for the number of significant figures we need (for me that's rarely more than 4 decimal places). Once I set the display, I expect (would like) that when I average data that the display shows the "properly" rounded average but this is not so. Sure, I can set the calculator to display all the digits and do the rounding myself but this introduces human errors and, to be honest, I expect the calculator to do this for me and not present me with the "wrong" number. Ok, it's not really wrong but it's not really right either. Regards, Wayne ▼ Rodger Rosenbaum Senior Member Posts: 305 Threads: 17 Joined: Jun 2007 07-05-2009, 04:39 AM You asked in the first post "... is there any way around the problem (other than writing a program)?". If you don't want to look at all 12 digits and do the rounding-to-even yourself, then the answer is no. Presumably, you've been doing some arithmetic on a data set. This probably was the result of a program. It doesn't take much extra programming to use Dekker's method to round the final result(s) to K digits. Then with the display set to show K digits, you'll see the result rounded-to-even.

 Possibly Related Threads... Thread Author Replies Views Last Post HP Prime program: rounding to a fraction Patrice 3 1,117 10-31-2013, 06:16 AM Last Post: Joe Horn HP Prime: Rounding error in determinant Stephan Matthys 3 1,086 10-25-2013, 09:29 PM Last Post: Walter B Rounding of 10^x Olivier De Smet 8 1,779 08-28-2013, 06:33 AM Last Post: Dieter Estimating Accumulated Rounding Errors Egan Ford 13 2,161 08-16-2012, 01:49 PM Last Post: Egan Ford WP34s: number display and rounding Dieter 4 1,023 10-14-2011, 05:11 PM Last Post: Marcus von Cube, Germany rounding, from another posting, a question... Geoff Quickfall 6 1,463 08-13-2011, 06:08 AM Last Post: Paul Dale Rounding errors BruceH 4 1,204 11-22-2009, 11:24 AM Last Post: Vieira, Luiz C. (Brazil) Rounding Revisited Rodger Rosenbaum 5 1,244 07-22-2009, 02:29 PM Last Post: Rodger Rosenbaum HP48g - Arithmetic Rounding querry caribet 10 1,854 11-07-2005, 08:14 PM Last Post: Caribe T Unexpected Rounding D Davis 4 862 03-17-2005, 08:27 PM Last Post: Tony

Forum Jump: