http://www.wiki4hp.com/doku.php?id=benchmarks:savage
I'm gathering the results from MoHPC and comp.sys.48 (from the entire web it is hard) thanks to google search. If anyone wants to help, he is welcome! The wiki is of all of us.
:)
Yet another benchmark port on the wiki: Savage


« Next Oldest  Next Newest »

▼
09132013, 06:43 AM
http://www.wiki4hp.com/doku.php?id=benchmarks:savage I'm gathering the results from MoHPC and comp.sys.48 (from the entire web it is hard) thanks to google search. If anyone wants to help, he is welcome! The wiki is of all of us. :) ▼
09132013, 07:06 AM
The 34S gives:
single precision 2500.000000001578
▼
09132013, 07:15 AM
Nooooooooooooooooooooo i want to know even the time! :( Ok, just use the accuracy :( ▼
09132013, 07:31 AM
I obtained these results via the console mode emulator, so the timings aren't relevant. On real hardware, timing will vary depending on firmware version  I went through three or four different implementations of LN e.g. The numeric results will be identical. I believe that the 34S single precision results are correctly rounded. At least I think I checked this benchmark ages back.
▼
09132013, 07:37 AM
Ok but i added another 34s result (check the page) that shows a different result. So other information (code, fw version, times, and so on) can help a lot. Anyway, i'll add them only as accuracy :) ▼
09132013, 08:03 AM
I think your program is wrong :) It is incrementing before the transcendentals rather than after. This means one more time through the loop I think  the iteration that gives 2500 as its answer doesn't apply the six functions to the almost 2500 result. The program in note 1 & 2 also has the square root and x^{2} operations reversed, the x^{2} should come first. This doesn't make a difference on the 34S but might on other devices.
LBL A
▼
09132013, 08:12 AM
many thanks! So i get rid of that ;) .
09172013, 03:55 AM
Reusing Paul Dale's program for the HP15CLE:
43,22,11 @ LBL A
I get 2499.970322, the same result reported for the HP41CX in Wlodek's article, according to message #3 in My runtime is 51.5 seconds, consistent over three runs. T
09132013, 08:10 AM
Oh i forgot that: thanks!
09132013, 10:17 AM
I found two interesting remarks in my searches: http://www.hpmuseum.org/cgisys/cgiwrap/hpmuseum/archv020.cgi?read=193816#194289
http://www.technicalc.org/tiplist/en/files/pdf/tips/tip6_50.pdf
So, since accuracy change a lot given a computation, maybe it's better to use the savage benchmark as benchmark to see (in general) how fast some non trivial but common operations are executed (instead of nqueens, addloop and similar). Edited: 13 Sept 2013, 10:31 a.m.
09162013, 04:34 AM
Quote: On my WP34S, rev. 3450, with crystal and fresh batteries: Single prec: 283.2 sec, result2500: 6.49e9 Double prec: 312.0 sec, result2500: 5.248e26 (The code by P. Dale in message #6, below)
09132013, 06:17 PM
Running it on my MK61 (and using my 41CX to time it) using this code:
00 1 We'll see how bad this is. Looks like it'll be about a 6 hour run, based on the loop counter that I can see being flashed at me. ▼
09142013, 07:40 AM
So, I fell asleep before the run completed. But, every extrapolation I made, including one at 5h15m completed, had it in the 6h15m6h18m range, so we'll go with that as being the time. And, I realized that I didn't actually RCL 0 at the end. But, RCL 0 returns 2269.0879. That's... wildly off. (I don't doubt that it's really how much error the MK61 has, this really is an 8 digit calculator, though.)
09172013, 06:14 PM
Quote: Adapting Paul Dale's program for the HP30b (RPN mode). This is the version which starts at 1, and runs the transcendental functions 2499 times, adding 1 after each iteration.
Checksum: 27.154 00  SH* @ assign to shiftmultiply button
Result: 2499.99948647 Timing (using external stopwatch): 7.5 seconds, consistent across three runs. (wow, the 30b is FAST, though precision is not up to WP34S standards) Tom ▼
09172013, 06:19 PM
Quote: The 34S is carrying almost three times the number of digits internally, has a few more functions and wasn't coded for performance (rather, for accuracy then space).
▼
09182013, 02:40 AM
Quote: I know, I know. I have been reading the discussions here. Most days, I'll choose the accuracy over the blazing speed, and for real numbercrunching I have a desktop computer.
Quote: That has got to be candidate for understatement of the year! :) T
09182013, 03:57 AM
Redoing the HP32S, HP42S and RPL version to match Dale's version exactly (i.e. loop up to 2499, square before square root, increment after transcendental functions) I also get identical numerical results between different saturnbased (or emulating) models: Result: 2499.99948647; 2500result = 5.13530e4 Timings: HP42S (external stopwatch): 9 min, 56 sec, or 596 sec HP32S (external stopwatch): 6 min, 37 sec or 397 sec (best of 2) HP48G: 117.475 secs HP50g: 65.0018 secs I'm somewhat surprised the 32S is that much faster than the 42S. Listings: RPL (HP50g, HP48G):
\<< RAD TICKS 1. 1. 2499. START SQ [sqrt] LN EXP ATAN TAN 1. + NEXT SWAP TICKS SWAP  B>R 8192. / 800. 2. BEEP \>> HP32S:
LBL S @ 015.5 bytes; CHKSUM=C573 HP42S:
{32Byte Prgm} T
09182013, 11:57 AM
An interesting update: the nspire CX seems to have a lot of problems with "standard" trig/exp functions:
09192013, 07:27 AM
The benchmark again in the format by Dale: 2499 times through the loop, increment after. Timing: HP33s (external stopwatch): 7 min 52 sec, or 472 sec HP41CX (internal, crosschecked with external stopwatch): 48 min, 52.8 sec or 2932.8 sec (consistent within 2 secs on 2 runs) Results: HP33s: 2499.99942403, 2500result = 5.7597e4 HP41CX: 2499.970322, 2500result = 2.9678e2 Listing HP33S is identical to the '32S reported previously Listing HP41CX
LBL 'SAVAGE T ▼
09192013, 09:09 AM
added, thanks!
Edited: 19 Sept 2013, 9:11 a.m. ▼
09232013, 03:25 PM
Quote: Fine. Since there are many almostequal results from almostequal versions of the benchmark I thought it useful to have them all for exactly equivalent programs. I've just got a Prime, and it is too fast for me to time externally: less than a second on the 2499iteration Savage. Result: 2499.99948647.
code: EXPORT SAVAGE() ▼
09252013, 04:59 AM
Added, thanks. About the speed of prime, you can do the savage extended, that should be harder. ▼
09262013, 06:09 AM
Quote: Sorry, but I don't like the savage extended. A benchmark, IMO, should give some information in addition to the timing. Savage does that (some sort of measure of numerical accuracy for transcendental functions). The extended savage, since it requires you starting over and repeating the N1 iterations from the previous savage every time through the outer loop, means arriving at the answer in a silly and extremely roundabout way. T ▼
09262013, 12:30 PM
Quote: I agree with you! indeed the savageextended wants to return more information about the accuracy of the calculator. Just read the part about "the idea" on the wiki page or the analysis done here http://www.technicalc.org/tiplist/en/files/pdf/tips/tip6_50.pdf. To be clear we should do a batch of accuracy tests over an interval of values (like the work done in the article above), to plot how the accuracy changes for each value. Since it's a quick benchmark, we can't do comparisons between plots, so at least we measure the sum of absolute errors for each value. I don't get why the Savage extended is silly in terms of information about accuracy, if it is silly, the Savage standard is definitely useless in comparison. ▼
09272013, 05:57 PM
Quote: That's not why it's silly. Every iteration in the outer loop of the extended savage benchmark could be computed by a single iteration from the last result, but instead it starts from one every time. ▼
09282013, 03:31 AM
You are right! I didn't think about that! (i just thought "nest the savage benchmark in a loop that tests the accuracy for several values". That is inefficient (1)). So it is quite easy to fix, just delete the outer loop and move the sum of absolute errors to add the error after each iteration.
input:
(1) I have even done personal tests and plots to check how the accuracy changes with a similar, inefficient, code! Poor me, i "wasted" a lot of energy. Edited: 28 Sept 2013, 3:31 a.m.
09252013, 05:08 AM
Fun the benchmark a couple of dozen times and divide:
EXPORT SAVAGE()
▼
09262013, 05:54 AM
Quote: Okay, about 24 seconds for 50 times, should mean 0.48 seconds per 2499iteration Savage. There is a TICKS function in the Prime, but I haven't been able to get useful results from it in programs (it works fine in the Home screen). T ▼
09272013, 06:22 PM
TIME(SAVAGE()) There you go. TW ▼
09272013, 09:37 PM
Quote: TIME(whatever) returns the elapsed time with the "_s" unit attached. If you don't want the unit attached to the result, type "time" in lowercase: time(SAVAGE()) This returns the number of elapsed seconds as a real, not a unit. Joe
09282013, 03:42 AM
Indeed. Thanks. I'm getting timings between 0.498 and 0.513 seconds for SAVAGE. T
09252013, 07:15 PM
Quote:
I really really dislike that version of the Savage Benchmark: Those two imperfections stem from the common implementation of the Savage Benchmark on early PCs using some version of the BASIC language. In BASIC it's slightly more complex to do the job right, i.e., without the objectionable aspects I listed above. In common calculator syntax, it's just as easy to do the job right as not. I use the Savage Benchmark version that corrects those issues.
Example of RPN version on HP 42S: (Put 2500 in X and execute SB) LBL "SB" Results: 2499.99946106 in 605 seconds
Example of RPL version on HP 50G: (Execute SB) << RAD TICKS 0. 1. 2500. Results: 2499.99946106 in 64.9 seconds
Example of Algebraic version on HP 20S: (Put 2500 in display and execute B) LBL B Results: 2499.99946106 in 350 seconds One of the fastest HP machines is the original HP 30b, programmed in RPN. Its results are 2499.99946106 in 6.5 seconds.
The more sophisticated algorithms found the the WP 34S cause its results from the same hardware to be very accurate, but also rather slow even with external DC power: 2500.00000001 (actually, 2500.000000006559) in about 300 seconds.
Performing this on the WP 34S without external power will give variously longer run times. With fresh CR 2032 batteries I get about 450 seconds as the WP 34S automatically slows its speed when low battery voltage is sensed, even with new batteries, during the later portion of the run. Edited: 25 Sept 2013, 7:18 p.m. ▼
09262013, 03:22 AM
Quote: It's due to the first value, anyway is just aesthetic. Put 2500 on the stack, and then subtract from it (in the program) one. As you don't compute the single iterations involving tan, atan, log, exp, roots and squares, you can even leave the calculator to compute 2499 for you.
Quote:
I disagree. Because if you do 2*2 and the square root, you have a chance to get 2. If you do square root of 2 first you have no chances to get 2, due to the fact that sqrt(2) is irrational and the calculators has a limited memory. But even you can't get 2 back if you do computations numerically, and not symbolically, on a paper. To get the perfect sqrt(2)*sqrt(2)=2 handling sqrt(2) as 1.414.... you need infinite time just to compute sqrt(2). Moreover i found some remarks about "accuracy by chance" here on MoHPC (i hope to find more discussion about that topic on comp.sys.48 when i will have time) and i reported them on the wiki.
In few words they say: the accuracy evaluation done on only a specific value of the test (in this case, 2500) has little meaning. The best is: do a batch of computations over consecutive values and check how the accuracy changes. Edited: 26 Sept 2013, 3:29 a.m. 