Yet another benchmark port on the wiki: Savage

[Pier Aiello]

http://www.wiki4hp.com/doku.php?id=benchmarks:savage

I'm gathering the results from MoHPC and comp.sys.48 (from the entire web it is hard) thanks to google search. If anyone wants to help, he is welcome! The wiki is of all of us.

:)

[Paul Dale]

The 34S gives:

single precision   2500.000000001578

double precision   2499.999999999999999999999999999661

- Pauli

[Pier Aiello]

Nooooooooooooooooooooo i want to know even the time! :(

Ok, just use the accuracy :(

[Paul Dale]

I obtained these results via the console mode emulator, so the timings aren't relevant. On real hardware, timing will vary depending on firmware version -- I went through three or four different implementations of LN e.g. The numeric results will be identical.

I believe that the 34S single precision results are correctly rounded. At least I think I checked this benchmark ages back.

As an aside, the 34S can be used to verify correct rounding for other calculators. Set double precision and after each operation, force a rounding step to the appropriate precision.

- Pauli

[Pier Aiello]

Ok but i added another 34s result (check the page) that shows a different result. So other information (code, fw version, times, and so on) can help a lot.

Anyway, i'll add them only as accuracy :)

[Paul Dale]

I think your program is wrong :-) It is incrementing before the transcendentals rather than after. This means one more time through the loop I think -- the iteration that gives 2500 as its answer doesn't apply the six functions to the almost 2500 result.

The program in note 1 & 2 also has the square root and x² operations reversed, the x² should come first. This doesn't make a difference on the 34S but might on other devices.

	LBL A

	TICKS

	2

	4

	9

	9

	# 001

	LBL 00

		x[^2]

		[sqrt]

		LN

		e[^x]

		ATAN

		TAN

		INC X

		DSE Y

			GTO 00

	TICKS

	RCL- T

	SDR 001

	x[<->] Y

	END

- Pauli

[Pier Aiello]

many thanks! So i get rid of that ;) .

[Tom Grydeland]

Reusing Paul Dale's program for the HP-15CLE:

43,22,11 @ LBL A

2

4

9

9

44 1     @ STO 1

1

43,22, 0 @ LBL 0

43 11    @ x^2

11       @ SQRT

43 12    @ LN

12       @ e^x

43 25    @ ATAN

25       @ TAN

1

40       @ +

42, 5, 1 @ DSE 1

22 0     @ GTO 0

43 32    @ RTN

I get 2499.970322, the same result reported for the HP-41CX in Wlodek's article, according to message #3 in
this thread. Note that this is _not_ the same result reported for HP-15CLE in the benchmark page on the wiki.

My runtime is 51.5 seconds, consistent over three runs.

--T

[Pier Aiello]

As you probably saw, i reported it from the source cited in the wiki page.

I guess that someone has mixed the order of operations, like first x^2 then sqrt, or first sqrt then x^2 and so on. Anyway thanks i'll report your code asap.

[Pier Aiello]

Oh i forgot that: thanks!

[Pier Aiello]

I found two interesting remarks in my searches:

http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/archv020.cgi?read=193816#194289

http://www.technicalc.org/tiplist/en/files/pdf/tips/tip6_50.pdf
(page 5)

So, since accuracy change a lot given a computation, maybe it's better to use the savage benchmark as benchmark to see (in general) how fast some non trivial but common operations are executed (instead of nqueens, addloop and similar).

Edited: 13 Sept 2013, 10:31 a.m.

[Tom Grydeland]

Quote:

---

i want to know even the time!

---

On my WP-34S, rev. 3450, with crystal and fresh batteries:

Single prec: 283.2 sec, result-2500: 6.49e-9

Double prec: 312.0 sec, result-2500: 5.248e-26

(The code by P. Dale in message #6, below)

[bhtooefr]

Running it on my MK-61 (and using my 41CX to time it) using this code:

00	1

01	STO 0

02	STO 1

03	2499

07	STO 2

08	RCL 2

09	RCL 1

10	-

11	x!=0 29

13	RCL 0

14	x^2

15	SQRT

16	LN

17	e^x

18	ATAN

19	TAN

20	1

21	+

22	STO 0

23	RCL 1

24	1

25	+

26	STO 1

27	GTO 08

29	R/S

We'll see how bad this is. Looks like it'll be about a 6 hour run, based on the loop counter that I can see being flashed at me.

[bhtooefr]

So, I fell asleep before the run completed. But, every extrapolation I made, including one at 5h15m completed, had it in the 6h15m-6h18m range, so we'll go with that as being the time.

And, I realized that I didn't actually RCL 0 at the end.

But, RCL 0 returns 2269.0879. That's... wildly off. (I don't doubt that it's really how much error the MK-61 has, this really is an 8 digit calculator, though.)

[Pier Aiello]

i'll report it :)

Done :)

Edited: 14 Sept 2013, 9:53 a.m.

[Tom Grydeland]

Quote:

---

If anyone wants to help, he is welcome!

---

Adapting Paul Dale's program for the HP-30b (RPN mode). This is the version which starts at 1, and runs the transcendental functions 2499 times, adding 1 after each iteration.

Checksum: 27.154

00 - SH*   @ assign to shift-multiply button

01 - 2

02 - 4

03 - 9

04 - 9

05 - STO 0

06 - 1

07 - LBL 00

08 - X^2

09 - [sqrt]

10 - Ln

11 - e^X

12 - Math

13 - Input

14 - Down

15 - Down

16 - Down

17 - Input   @ ATAN

18 - Tan

19 - 1

20 - +

21 - DSE 0

22 - Gto 00

Result: 2499.99948647
(2500 - result = 5.135300e-4)

Timing (using external stopwatch): 7.5 seconds, consistent across three runs.

(wow, the 30b is FAST, though precision is not up to WP-34S standards)

--Tom

[Paul Dale]

Quote:

---

(wow, the 30b is FAST, though precision is not up to WP-34S standards)

---

The 34S is carrying almost three times the number of digits internally, has a few more functions and wasn't coded for performance (rather, for accuracy then space).

- Pauli

[Tom Grydeland]

Quote:

---

The 34S is carrying almost three times the number of digits internally, has a few more functions and wasn't coded for performance (rather, for accuracy then space).

---

I know, I know. I have been reading the discussions here. Most days, I'll choose the accuracy over the blazing speed, and for real number-crunching I have a desktop computer.

Quote:

---

The 34S [...] has a few more functions

---

That has got to be candidate for understatement of the year! :-)

--T

[Tom Grydeland]

Redoing the HP-32S, HP-42S and RPL version to match Dale's version exactly (i.e. loop up to 2499, square before square root, increment after transcendental functions) I also get identical numerical results between different saturn-based (or emulating) models:

Result: 2499.99948647; 2500-result = 5.13530e-4

Timings:

HP-42S (external stopwatch): 9 min, 56 sec, or 596 sec

HP-32S (external stopwatch): 6 min, 37 sec or 397 sec (best of 2)

HP-48G: 117.475 secs

HP-50g: 65.0018 secs

I'm somewhat surprised the 32S is that much faster than the 42S.

Listings:

RPL (HP-50g, HP-48G):

\<< RAD TICKS 1. 1. 2499. START SQ [sqrt] LN EXP ATAN TAN 1. + NEXT SWAP TICKS SWAP - B->R 8192. / 800. 2. BEEP \>>

HP-32S:

LBL S  @ 015.5 bytes; CHKSUM=C573

RAD

2499

STO U

1

LBL U  @ 016.5 bytes; CHKSUM=EC87

x^2

SQRT

LN

e^x

ATAN

TAN

1

+

DSE U

GTO U

HP-42S:

{32-Byte Prgm}

LBL "SAVAGE"

RAD

2499

1

LBL 00

X^2

SQRT

LN

EXP

ATAN

TAN

1

+

DSE ST Y

GTO 00

END

--T

[Pier Aiello]

Thanks, i'll report them asap (both #16 and #19)

[Pier Aiello]

An interesting update: the nspire CX seems to have a lot of problems with "standard" trig/exp functions:

http://tiplanet.org/forum/viewtopic.php?p=148292#p148292

[Tom Grydeland]

The benchmark again in the format by Dale: 2499 times through the loop, increment after.

Timing:

HP-33s (external stopwatch): 7 min 52 sec, or 472 sec

HP-41CX (internal, cross-checked with external stopwatch): 48 min, 52.8 sec or 2932.8 sec (consistent within 2 secs on 2 runs)

Results:

HP-33s: 2499.99942403, 2500-result = 5.7597e-4

HP-41CX: 2499.970322, 2500-result = 2.9678e-2

Listing HP-33S is identical to the '32S reported previously

Listing HP-41CX

LBL 'SAVAGE

TIME

RAD

2499

1

LBL 00

X^2

SQRT

LN

E^X

ATAN

TAN

1

+

DSE Y

GTO 00

TIME

R^

HMS-

BEEP

--T

[Pier Aiello]

added, thanks!

Two notes:

- I don't report your result if they match with the score of the same calculator already present in the list (even if i could report it as confirmation :/ ). If you don't like it, feel free to modify the page, it's the wiki of hp calculators users!

- Since you seems enjoyed by this benchmark, could you do also the savage extended? (careful, it takes a lot of time)

http://www.wiki4hp.com/doku.php?id=benchmarks:savage-extended

Edited: 19 Sept 2013, 9:11 a.m.

[Tom Grydeland]

Quote:

---

I don't report your result if they match with the score of the same calculator already present in the list (even if i could report it as confirmation :/ )

---

Fine. Since there are many almost-equal results from almost-equal versions of the benchmark I thought it useful to have them all for exactly equivalent programs.

I've just got a Prime, and it is too fast for me to time externally: less than a second on the 2499-iteration Savage. Result: 2499.99948647.

code:

EXPORT SAVAGE()

BEGIN

A:=1;

FOR I FROM 1 TO 2499 DO

  A:=TAN(ATAN(e^(LN(\|(A*A)))))+1

END;

END;

[Pier Aiello]

Added, thanks. About the speed of prime, you can do the savage extended, that should be harder.

[Tom Grydeland]

Quote:

---

About the speed of prime, you can do the savage extended, that should be harder

---

Sorry, but I don't like the savage extended.

A benchmark, IMO, should give some information in addition to the timing. Savage does that (some sort of measure of numerical accuracy for transcendental functions). The extended savage, since it requires you starting over and repeating the N-1 iterations from the previous savage every time through the outer loop, means arriving at the answer in a silly and extremely roundabout way.

--T

[Pier Aiello]

Quote:

---

A benchmark, IMO, should give some information in addition to the timing. Savage does that (some sort of measure of numerical accuracy for transcendental functions)

---

I agree with you! indeed the savage-extended wants to return more information about the accuracy of the calculator. Just read the part about "the idea" on the wiki page or the analysis done here http://www.technicalc.org/tiplist/en/files/pdf/tips/tip6_50.pdf. To be clear we should do a batch of accuracy tests over an interval of values (like the work done in the article above), to plot how the accuracy changes for each value.

Since it's a quick benchmark, we can't do comparisons between plots, so at least we measure the sum of absolute errors for each value.

I don't get why the Savage extended is silly in terms of information about accuracy, if it is silly, the Savage standard is definitely useless in comparison.

[Tom Grydeland]

Quote:

---

I don't get why the Savage extended is silly in terms of information about accuracy

---

That's not why it's silly.

Every iteration in the outer loop of the extended savage benchmark could be computed by a single iteration from the last result, but instead it starts from one every time.

[Pier Aiello]

You are right! I didn't think about that! (i just thought "nest the savage benchmark in a loop that tests the accuracy for several values". That is inefficient (1)). So it is quite easy to fix, just delete the outer loop and move the sum of absolute errors to add the error after each iteration.

input:

  n: upper bound value to test the savage benchmark

---

globalSum = 0

sum = 1

for k=1 to (n-1) { //note the "n-1"

  sum = sum + tan ( atan ( exp ( log ( sqrt ( sum*sum) ) ) ) )

    //CAREFUL here!

    //for the original savage benchmark many have done sqrt first and square then,

    //obtaining different results.

    //The algorithm must be the same in terms of main operations:

    //Square first

    //then square root

    //then log

    //then exp

    //then atan

    //then tan

  globalSum = globalSum + abs((k+1)-sum)

}

print globalSum //the global sum obtained

(1) I have even done personal tests and plots to check how the accuracy changes with a similar, inefficient, code! Poor me, i "wasted" a lot of energy.

Edited: 28 Sept 2013, 3:31 a.m.

[Paul Dale]

Fun the benchmark a couple of dozen times and divide:

EXPORT SAVAGE()

BEGIN

A:=1;

FOR J FROM 1 TO 50 DO

  FOR I FROM 1 TO 2499 DO

    A:=TAN(ATAN(e^(LN(\|(A*A)))))+1

END;

- Pauli

[Tom Grydeland]

Quote:

---

Fun the benchmark a couple of dozen times and divide:

---

Okay, about 24 seconds for 50 times, should mean 0.48 seconds per 2499-iteration Savage.

There is a TICKS function in the Prime, but I haven't been able to get useful results from it in programs (it works fine in the Home screen).

--T

[Tim Wessman]

TIME(SAVAGE())

There you go.

TW

[Joe Horn]

Quote:

---

TIME(SAVAGE())

---

TIME(whatever) returns the elapsed time with the "_s" unit attached. If you don't want the unit attached to the result, type "time" in lowercase:

time(SAVAGE())

This returns the number of elapsed seconds as a real, not a unit.

-Joe-

[Tom Grydeland]

Indeed. Thanks.

I'm getting timings between 0.498 and 0.513 seconds for SAVAGE.

--T

[Mike Morrow]

Quote:

---

LBL 'SAVAGE

TIME

RAD

2499

1

LBL 00

X^2

SQRT

LN

E^X

ATAN

TAN

1

+

DSE Y

GTO 00

TIME

R^

HMS-

BEEP

---

I really really dislike that version of the Savage Benchmark:

1. It only iterates 2499 times to produce the near 2500 result. It would be more elegant for it to do a full 2500 iterations to produce that near 2500 result!

2. It first squares the number, then takes the square root. That's a little cheesy, because if every other result were perfect the "square, then square root" sequence would be operating only on whole numbers. Better to take square root first, then do square.

Those two imperfections stem from the common implementation of the Savage Benchmark on early PCs using some version of the BASIC language. In BASIC it's slightly more complex to do the job right, i.e., without the objectionable aspects I listed above. In common calculator syntax, it's just as easy to do the job right as not.

I use the Savage Benchmark version that corrects those issues.

Example of RPN version on HP 42S: (Put 2500 in X and execute SB)

LBL "SB"

RAD

0

LBL 01

1

+

SQRT

X^2

LN

E^X

ATAN

TAN

DSE ST Y

GTO 01

BEEP

DEG

END

Results: 2499.99946106 in 605 seconds

Example of RPL version on HP 50G: (Execute SB)

<< RAD TICKS 0. 1. 2500. 

START 1. + sqrt SQ LN EXP ATAN TAN NEXT

SWAP TICKS SWAP - B->R 8192. /

DEG 800. 2. BEEP >>

'SB' STO

Results: 2499.99946106 in 64.9 seconds

Example of Algebraic version on HP 20S: (Put 2500 in display and execute B)

LBL B

RAD

STO 0

0

LBL 1

+

1

=

sqrt

X^2

LN

E^X

ATAN

TAN

STO 1

1

STO-0

RCL 0

X=0?

GTO 2

RCL 1

GTO 1

LBL 2

STO 1

DEG

RTN

Results: 2499.99946106 in 350 seconds

One of the fastest HP machines is the original HP 30b, programmed in RPN. Its results are 2499.99946106 in 6.5 seconds.

The more sophisticated algorithms found the the WP 34S cause its results from the same hardware to be very accurate, but also rather slow even with external DC power: 2500.00000001 (actually, 2500.000000006559) in about 300 seconds.

Performing this on the WP 34S without external power will give variously longer run times. With fresh CR 2032 batteries I get about 450 seconds as the WP 34S automatically slows its speed when low battery voltage is sensed, even with new batteries, during the later portion of the run.

Edited: 25 Sept 2013, 7:18 p.m.

[Pier Aiello]

Quote:

---

It only iterates 2499 times to produce the near 2500 result. It would be more elegant for it to do a full 2500 iterations to produce that near 2500 result!

---

It's due to the first value, anyway is just aesthetic. Put 2500 on the stack, and then subtract from it (in the program) one. As you don't compute the single iterations involving tan, atan, log, exp, roots and squares, you can even leave the calculator to compute 2499 for you.

Quote:

---

It first squares the number, then takes the square root. That's a little cheesy, because if every other result were perfect the "square, then square root" sequence would be operating only on whole numbers. Better to take square root first, then do square

---

I disagree. Because if you do 2*2 and the square root, you have a chance to get 2. If you do square root of 2 first you have no chances to get 2, due to the fact that sqrt(2) is irrational and the calculators has a limited memory. But even you can't get 2 back if you do computations numerically, and not symbolically, on a paper. To get the perfect sqrt(2)*sqrt(2)=2 handling sqrt(2) as 1.414.... you need infinite time just to compute sqrt(2).
So, imo, it's easy to get a false value if you do square root first. Then, accordingly to what i said, is better the version with square first for me.

Moreover i found some remarks about "accuracy by chance" here on MoHPC (i hope to find more discussion about that topic on comp.sys.48 when i will have time) and i reported them on the wiki.

In few words they say: the accuracy evaluation done on only a specific value of the test (in this case, 2500) has little meaning. The best is: do a batch of computations over consecutive values and check how the accuracy changes.

Edited: 26 Sept 2013, 3:29 a.m.