Good (simple?) calculation for benchmarking?



#15

Hi, this is probably way elementary for all of you, but I was wondering if anyone has a good suggestion for a *simple* yet good calculation I could type in to HP 48 / 49 series calculators to do a general benchmark comparison? I have 3 calcs at the moment that I would like to do this with, just for fun to see the differences in speed: HP 48GX, HP 49G, & HP 49G+ (w/50G ROM).

Thanks for your time!


#16

I doubt that this will work as a benchmark. All of the mentioned calculators are probably too fast to even press start/stop on your stop watch.

So you would need a loop of calculations. Then you would have to factor different things into it: int performance, float performance, how well are functions implemented, gfx performance and so on.

No wonder a PC benchmark suite is quite complex :)

Easiest thing would be a +1 loop, like 10000 times. That should be hand-stoppable, this is more or less raw CPU speed dependent.


#17

And, remember to benchmark an empty loop, so you understand the overhead of the loop, and subract that from benchmarks of other operations.


#18

Benchmark for lots of calculators

#19

I see it is not so simple. :-)

So would a better way to do it be to write a small program to say, find pi to the 1000th digit or something like that, and then just time each one?

I've (obviously) never done any programming before, so I just thought it might be a fun exercise, but sounds like it's more complex than I thought!

**EDIT** thanks for the link above, it came in while I was typing. :-) I will check that out and see if I can reproduce it on these calcs.


Edited: 26 Feb 2013, 12:12 p.m.

#20

Use this. It does useful work and solves a real problem and all the code for everything you want to test is here:

http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/articles.cgi?read=700

Another benchmark (used for super computers (http://top500.org)) would be to solve a dense system of liner equations. Also a real problem. I've use this compare my 71b, 48GX, and 50g. IIRC, the random number generator in all 3 are the same and thus generated the exact same problem to solve. Memory limits the size of the matrix.

As you mention above Pi to 1000 digits. I like to use that to compare some calculators, but mostly 8-bit processors.

For all three you need to use the same algorithm if you want to reduce the number of variables. E.g.:

  1. See [url:http://sense.net/~egan/hpgcc/#Example: π Shootout] for an example of how different Pi computations on the same calculator vary widely with speed.
  2. The Top500 rules are very clear that the algorithm used must conform to the standard operation count for LU factorization with partial pivoting. In particular, the operation count for the algorithm must be (2/3)n3 + O(n2) floating point operations.
  3. For all my 8-bit Pi benchmarks I always use assembly (compilers and interpreters introduce too much variable) and I use the same Machin Arctan O(n2) algorithm.
#21

Quote:
*simple* yet good

Go with the largest dense matrix and vector you can fit into memory/2 (48/49/50 matrix operations use a nondestructive copy doubling the amount of RAM required), then solve it. Use random numbers between -0.5 and 0.5. You can measure the time in seconds or use the internal 48/49/50 timer (ticks) to get higher resolution times.
#22

An iterative calculation of some sort is the only usable type of benchmark, preferably one that uses the results of the previous iteration in the next to produce a final result that is useful to gauge the propagation of truncation and accuracy errors. For scientific applications that likely make up a large portion of the applications for our HPs, it is logical to include various standard mathematical functions in the loop, in preference to some other benchmarks that spend most of the execution time moving data to and fro, and operating on them with basic arithmetic. However, that type of benchmark would provide better results for applications that use those capabilities, such as matrix manipulation intensive applications.

For several decades one of the benchmarks often utilized has been
the Savage Benchmark. It was useful in the early personal computing
era when desktops were much slower than many handheld devices are
today.

One common version of this benchmark in RPL follows that is based
on the exact series of operations that were easily programmed using
the typical syntax of BASIC on early personal computers for the
statement 10 A=TAN(ATN(EXP(LOG(SQR(A*A)))))+1 .
Note: "sqrt" below is the square root symbol.
<<
RAD
1.
1. 2499. START
SQ sqrt LN EXP ATAN TAN 1. +
NEXT
>>
The HP 50G (version 2.15) completes the above in about 65 seconds
with this result: 2499.99948647 .

I prefer a slight modification that produces more meaningful results
that takes advantage of the versatility of RPL syntax by:
(1) Arranging for exactly 2500 iterations (first version does 2499),
(2) Taking the square root, followed by the square (first version
reverses this order and thus operates only on integer values),
(3) Add steps to automatically indicate the execution time in
seconds, after iterations are complete,
(4) Add step to return calculator to DEG mode, and
(5) Add 800 Hz two-second beep at end.
<<
RAD TICKS
0.
1. 2500. START
1. + sqrt SQ LN EXP ATAN TAN
NEXT
SWAP TICKS SWAP - B->R 8192. /
DEG
800. 2. BEEP
>>
The HP 50G (version 2.15) completes the above in about 65 seconds
with this result: 2499.99946106 (and 64.8814697266 seconds for
this run).

Note: For the 49G+ and 50G, one will not want the calculations done
using the binary integer types that exist on those machines. That's
the reason for the decimal point appearing with each value in the
programs above. Remove those decimal points, and execution times
increase very significantly.

Very simple benchmarks, such as a loop that just increments a value by 1 each iteration for a specified interval, produce very inaccurate estimates of the practical or typical speeds of two machines relative to each other.

For example, Free42 completes the Savage Benchmark in about 1.5 seconds on my Android smartphone, 400 times faster than a real HP 42S. But just counting the iterations of a X=X+1 type of loop for 60 seconds on these two machines indicates that Free42 on my Android is a whopping 7800 times faster than a real HP 42S. That's a factor of almost 20 in the comparative results caused simply by the choice of benchmark method!

There is no "universal" benchmark that is meaningful for every application.


Edited: 26 Feb 2013, 4:40 p.m.


#23

Oh man, thanks guys. We are already so far over my head with that last post. :-) It will take me a while to decipher that and figure out how to do it.
I get it that the simple calculation won't work (like I first thought), so I will try to figure out how to do the more complex one as I have some time.


#24

To cut to the essential, the Savage Benchmark may be quickly entered into any of your three calculators by storing the program object

<< RAD TICKS 0. 1. 2500. START 1. + sqrt SQ LN EXP ATAN TAN NEXT SWAP TICKS SWAP - B->R 8192. / DEG 800. 2. BEEP >>

("sqrt" stands for the "square root symbol")

and storing that as, for example, SBM.

Executing SBM will result in a display of the results of execution and the execution time. A result of 2500.00000000 would be perfect, but all of your machines will produce the same result (2499.99946106), at an execution time of about 65 seconds (for the 49G+) and greater (for the other two machines).


#25

Does the TICKS command produce more accurate results then the TIME command? If not then it is simpler to use the TIME command (no conversions required).


#26

All of the elapsed time display overhead takes place after the looping has been completed, after the begin and end TICKS values have been stored. There's no substantive advantage to using TIME, HMS-, and HMS-> compared to the TICKS process, and one situation (albeit unlikely) where TIME will fail.

<< RAD TICKS 0. 1. 2500. START 1. + sqrt SQ LN EXP ATAN TAN NEXT SWAP TICKS SWAP - B->R 8192. / DEG 800. 2. BEEP >>

versus

<< RAD TIME 0. 1. 2500. START 1. + sqrt SQ LN EXP ATAN TAN NEXT SWAP TIME SWAP HMS- HMS-> 3600. * DEG 800. 2. BEEP >>

The TICKS method does not develop a nonsense result such as the TIME method does if the loop begins on one side of 00.00 (midnight) and ends on the other side. :-)

#27

Quote:
all of your machines will produce the same result (2499.99946106),

which happens to be exactly the same result produced by the HP-71B, where they inherited lots of math algorithms and Saturn code from:

>RADIANS
>1 T=TIME@A=0@FORI=1TO2500@A=TAN(ATAN(EXP(LN(SQR(A+1)*SQR(A+1)))))@NEXTI@DISPA;TIME-T
>RUN
2499.99946106 387.53

#28

Yes, that result is produced for all machines that are Saturn or Saturn-like...for example: The HP 42S, HP 32SII, HP 20S (lowly old algebraic Pioneer), and the recent non-Saturn HP 30b.

The HP 30b is the speed champion among all HPs that I've tried. It completes the 2500-iteration Savage Benchmark in 6.5 seconds. That's ten times faster than the HP 50G, and eight times faster than the HP 15C LE (which is not Saturn-like and produces a result with considerably less precision and accuracy).

I have yet to program my HP39GII for this. Unfortunately it has no real time clock implementation to make timing easier.


#29

Quote:
The HP 30b is the speed champion among all HPs that I've tried. It completes the 2500-iteration Savage Benchmark in 6.5 seconds.

It takes quite longer on my wp34s, but the results are outstanding:

001 LBL A
002 TICKS
003 # 250
004 SDL 001
005 0
006 INC X
007 SQRT
008 x^2
009 LN
010 e^X
011 ATAN
012 TAN
013 DSE Y
014 BACK 008
015 TICKS
016 RCL- T
017 SDR 001
018 x<> Y
019 END

DBLON
XEQ A --> 2500. (2.500 000 000 000 000 000 000 000 000 053 091 E 0003)
x<>y --> 319.3 (5m 19.3s)

DBLOFF
XEQ A --> 2500.00000001 (2.500 000 000 006 559 E 003)
x<>y --> 287.1 (4m 47.1s)

-----------------------------------------

P.S.: With the latest firmware, 3.2r3371:

V3.2 is slightly faster:

    306.0              (5m 06.0s)

275.9 (4m 35.9s)


Edited: 3 Mar 2013, 1:14 a.m. after one or more responses were posted


#30

The logarithm will be the killer here I suspect.


- Pauli


#31

Or TAN, according to Table 3 in this paper. The next ones in order would be ATAN, LN and EXP. The table applies to minimax polynomials, but might apply to Taylor series as well.

I was expecting faster results, but I had forgotten about this post by W. Bruce Maguire II last year:

WP-34s: Speed benchmark

Gerson.


#32

I'm almost certain it is the logarithm. The 34S uses taylor expansion for the trig functions and this is quite fast. For exponential and logarithm it uses other series and I know the logarithm is very slow.


- Pauli


#33

You are right! LN and EXP are indeed slower than TAN and ATAN:

...
009 ATAN
010 TAN
011 ATAN
012 TAN
... --> 215.7 s (2500.000000006559)


...
009 LN
010 e^x
011 LN
012 e^X
... --> 358.2 s (2499.999999999975)

Gerson.
#34

Very impressive!!!

#35

6.5 seconds? Wow, how slow... ;-)

EXPORT SAVAGE()
BEGIN
A:=0
FOR I FROM 1 TO 2500 DO
A:=TAN(ATAN(EXP(LN(sqrt(A+1)²))));
END:
END;

Run on the home screen with SAVAGE().

Res: 2499.99946106

Run on the home screen with Time(SAVAGE())

Res: 3.6_s

Use the Time(<argument>) function to time execution speed. There is an internal clock. Internally, it keeps time as ms since boot. If i remember correctly, it will roll over at about 47 days of running. No issues with midnight though! :-)

TW

Edited: 28 Feb 2013, 11:22 a.m.


#36

Thanks much, Tim. I haven't had as much time as I'd like to explore the capabilities of the HP 39GII, so I find your information very interesting. I had little doubt that the remarkable HP 39GII would be faster than the already impressive HP 30b. Now I know that it's twice as fast, for this benchmark. And 20 times faster than the HP 50G. The HP 39GII is the new HP Savage Benchmark speed champion.

The HP 30b and HP 39GII show all the good things that come from not having an emulation layer that the calculator hardware must muddle through.

#37

Got a different result at first, but then realised you are doing this in radians. No I get the same result and a very similar time (387.87).


#38

Quote:
I get the same result and a very similar time (387.87).

This means the CPU clock frequency of your HP-71B is 621200 Hz, or very close to (per HP-71B CPU frequency?, a recent thread).
Notice SQR(A+1) is computed twice every iteration. The following saves about 22.5 seconds:

>RADIANS
>1 T=TIME @ A=0 @ FOR I=1 TO 2500 @ B=SQR(A+1) @ A=TAN(ATAN(EXP(LN(B*B)))) @ NEXT I
>2 DISP A;TIME-T
>RUN
2499.99946106 365.09

#39

Quote:
Notice SQR(A+1) is computed twice every iteration. The following saves about 22.5 seconds:

Yes, I did notice that. I tried to substitute "SQRT(A+1) * SQRT(A+1)" with "SQRT(A+1)^2". To my surprise this was even slower. I appreciate that squaring takes longer than multiplying. But I didn't think it would take longer than calculating the extra square root.


#40

A user-defined function takes longer (about 460 seconds). Using a one-element vector instead of a variable and the DOT function (available in the MATH ROM) might work, but this would probably be even slower.


#41

Yes, 460 seconds is roughly what I got. What do you mean by user-defined? There is nothing like x² (square function) on the 71B, is there?


#42

It appears SQ was not a standard BASIC function. My old 8-bit computer didn't have one either, as far as I can remember.

>10 DEF FNS(X)=X*X
>20 DISP FNS(5)
>RUN
25

#43

I see, thanks!

I really need to play with the 71B a lot more. It is a fantastic machine and unfortunately I still know so little about it.

Cheers,
Harald

Edited: 1 Mar 2013, 5:14 p.m.


Possibly Related Threads…
Thread Author Replies Views Last Post
  HP35s Program Four Slings Lift Calculation Jean-Marc Biram (Australia) 2 2,296 12-16-2013, 07:21 PM
Last Post: Jean-Marc Biram (Australia)
  Simple Tetris. free for you to improve on cyrille de Brébisson 3 1,979 11-20-2013, 05:43 PM
Last Post: Erwin Ried
  [HP-Prime] Simple Game (Bugs) CompSystems 1 1,464 11-01-2013, 10:18 AM
Last Post: Han
  Good puzzle for kids to solve on 35s? snaggs 11 3,516 09-18-2013, 10:40 PM
Last Post: David Hayden
  [HP-Prime] AMBIGUITY between Numerical Calculation (HOME) and Numerical/Symbolic Calculation (CAS mode) CompSystems 2 1,491 08-18-2013, 07:06 PM
Last Post: CompSystems
  slow calculation Bill Plaxton 3 1,660 08-15-2013, 11:57 AM
Last Post: Bill Plaxton
  Simple Math Question Namir 2 1,450 08-09-2013, 06:13 PM
Last Post: Eddie W. Shore
  hp 17bii+ dates calculation... JoePaul 6 2,483 07-14-2013, 11:32 AM
Last Post: Katie Wasserman
  HP35 Ad. from 1972 with calculation error reused in 2009? Michael Kathke 46 11,942 05-14-2013, 03:52 PM
Last Post: Dieter
  A good general mathematics reference book? Chris Smith 11 3,521 04-29-2013, 09:44 AM
Last Post: Chris Smith

Forum Jump: