[WP 34s] Pressure Conversion Factors



#2

WP 34s (3.2T 3375)

Pressure conversion factors implemented in the WP 34s calculator seem to be based on the published values found in the NIST Guide to SI Units. (So are all the others, for that matter. I think.) This is a good source for conversion factors (reliable and free), particularly the exact conversion factors highlighted in bold face in the document., There are, however, many listed conversion factors which are approximations in the NIST Guide; results from conversions carried out using these approximated factors must be restricted to the number of given significant figures in the value from the table. As the given approximate conversion factors contain six or seven significant figures (usually) and most people are looking for answers with two, three, or four sig-figs (at least in engineering calculations in my experience), these conversion factors are more than adequate for the job.

EG: The NIST Guide gives the conversion factor from units of mmHg to units of Pa as 1.333 224 E+02. So, given a pressure 760 mmHg the converted result (760 * 133.3224) would be 101 325.024 Pa. Because the factor has seven significant digits the answer should be rounded to no more than seven significant figures: therefore, our rounded answer to seven significant digits is 101 325.0 Pa. You could argue that the answer should not have more than two significant digits based on the given value 760 mmHg, which is true if you did not implicitly mean the number 760 exactly.

Now to my point: Some of the included approximate conversion factors in the NIST Guide have exact conversions which were not included in the publication (NIST is emphasizing the uncertainty of certain measured values like barometric pressures). In particular, the conversions for many of the pressure units: those listed as conventional (mmHg, cmHg, ftHg, mmH2O, cmH2O, inH2O, ftH2O) and those that are related to these conventional pressure units (torr). All these conventional units are based on the pressure unit of atmosphere and the equivalent in these other units. For instance, mmHg-to-Pa factor is based on the exact definition 1 atm = 101 325 Pa and 1 atm = 760 mmHg (conventional); so the listed value in the NIST Guide 1.333 224 E+02 is the approximation of (101 325/760). I think that the pressure conversion factors implemented in the WP 34s should use the exact definitions of these conventional pressure units. This would eliminate unexpected and slightly wrong conversions displayed on the calculator such as
760 mmHg = 101 325.024 Pa?!

Here are a list of suggested adjusted pressure conversion factors.

760 mmHg = 101 325 Pa, therefore:

1 mmHg = (101 325/760) Pa = 133.322 368 421 0526 Pa

1 torr = 133.322 368 421 0526 Pa

29.921 259 842 519 69 inHg = 101 325 Pa, therefore:

1 inHg = 3 386.388 157 894 736 Pa

1 lbm = 0.453 592 37 kg (exact)

1 lbf = (0.453 592 37 kg)(9.806 65 m s^-2) = 4.448 221 615 260 500 N

1 (lbf/in2) = 1 psi = (1 lbf in^-2)(4.448 221 615 260 500 N lbf^-1)[(39.370 078 740 157 48)^2 in^2 m^-2]

1 psi = 6 894.757 293 168 361 Pa

Tim

Edited: 3 Nov 2013, 11:52 p.m.


#3

Tim,

Thanks for your elaborate post. AFAICS, you're right. Now let's go to the consequences.

Quote:
As the given approximate conversion factors contain six or seven significant figures (usually) and most people are looking for answers with two, three, or four sig-figs (at least in engineering calculations in my experience), these conversion factors are more than adequate for the job.

Concurs with my experience :-) BTW, I don't know any pressure-meter (?) being accurate to five figures or more. And please see the very first page about CONV in the manual. Summing up, 90% of CONV is a crutch for those folks who still can't get accustomed to SI - and to my experience, those folks don't work in precision businesses to any extent coming close to the precision you request.

d:-)


#4

Understood.

Your calculator has the highest precision on the market (one of the reasons I purchased the physical unit from Eric and donated to the project). The answers it displays ought to reflect that precision (I think); even if the extent of that precision is not required. When I use the CONV function I expect it to work to the same standard as the rest of the calculator. Should it not give the right answer to a conversion calculation?

I really like the WP 34s and plan to continue using it -- whether you choose to fix the approximated conversion factors or not. It is of minor significance.

All the best,

Tim


#5

Thanks. From a "real world" point of view, I vote for leaving CONV as is.

d:-)

#6

Somewhat off-topic:

> So, given a pressure 760 mmHg the converted result (760 * 133.3224) would be 101 325.024 Pa

Using only 3 digits as input value (760), only 3 digits are relevant in the answer...
101 *10^3 Pa

:-)

Edited: 4 Nov 2013, 12:08 p.m.


#7

Quote:
Quote:
So, given a pressure 760 mmHg the converted result (760 * 133.3224) would be 101 325.024 Pa

Using only 3 digits as input value (760), only 3 digits are relevant in the answer...
101 *10^3 Pa

Usually true though not here: 760 is an exact value, i.e. 760.000000... Please see the OP. HTH.

d:-)


Edited: 4 Nov 2013, 12:16 p.m.


#8

Just curious:

Is there any know conventiont to indicate an exact value, in contrast to a rounded value?

Eg 760=760 in contrast to 759,5.....760,4 = 760


#9

I'm not aware of any mathematical symbol indicating such an exactitude. IIRC, the torr scale is simply defined by the fact that 760 torr = p0, like the Celsius scale by 0°C = T0, with 760, p0, 0 and T0 being exact values. Without knowing the definition, you can't know the precision.

d:-/

#10

As we told the students in physics class, 760. would be the proper way to express 760 itself to 3 significant figures.

For infinite precision, you could use an overscore/overline:

Per Wikipedia, the overline can be used to represent a repeating decimal:


3.¯3 = 3.333333333333...

(I couldn't copy, nor figure out how to put in, the overscore directly above the number!)

Thus, in this notation 760.¯0 would indicate infinite precision of 760


Possibly Related Threads...
Thread Author Replies Views Last Post
  [WP-34S] Unfortunate key damage with update to V3 :( svisvanatha 5 387 12-10-2013, 11:37 PM
Last Post: Les Bell
  WP-34S (Emulator Program Load/Save) Barry Mead 1 210 12-09-2013, 05:29 PM
Last Post: Marcus von Cube, Germany
  DIY HP 30b WP 34s serial flash/programming cable Richard Wahl 2 292 12-04-2013, 11:14 AM
Last Post: Barry Mead
  HP 35s polar/rectangular conversion CD Dodds 6 326 11-28-2013, 02:39 PM
Last Post: Dieter
  WP 34S/43 ?'s Richard Berler 3 301 11-10-2013, 02:27 AM
Last Post: Walter B
  My FrankenCulator (wp-34s) FORTIN Pascal 4 315 11-09-2013, 06:18 PM
Last Post: FORTIN Pascal
  WP 34S Owner's Handbook Walter B 5 457 11-09-2013, 05:34 PM
Last Post: Harald
  wp 34s overlay and programming. FORTIN Pascal 6 398 11-08-2013, 01:28 PM
Last Post: Nick_S
  m.dy in display of WP-34S Harold A Climer 3 227 11-05-2013, 11:28 AM
Last Post: Andrew Nikitin
  WP 34s : An old problem coming back Miguel Toro 2 228 11-05-2013, 03:00 AM
Last Post: Marcus von Cube, Germany

Forum Jump: