Rounding errors  Printable Version + HP Forums (https://archived.hpcalc.org/museumforum) + Forum: HP Museum Forums (https://archived.hpcalc.org/museumforum/forum1.html) + Forum: Old HP Forum Archives (https://archived.hpcalc.org/museumforum/forum2.html) + Thread: Rounding errors (/thread159883.html) 
Rounding errors  BruceH  11212009 Just to emphasise that rounding errors aren't just limited to calculators here's something culled from the Interweb...
Quote: Re: Rounding errors  Lyuka  11212009 24.5 days means 2^31 [ms] It is probably an old singed and unsigned or wrapround bug but not rounding errors.
The source or related article is Slashdot IT Story
Re: Rounding errors  Paul Dale  11212009 So it isn't so much a rounding error as an integer overflow.
Still, I cannot fathom a reason why the autofocus code would care about the time.
 Pauli
Re: Rounding errors  David Hayden  11222009 Quote: I have minimal knowledge of how autofocus works, but that never stopped me from speculating before.... :) The routine might do something like this:
 Check to focusNow if you get the time from a millisecond counter and mistakenly interpret it as a signed value, you'll think that the time interval was negative and all hell breaks loose.
Again, this is purely speculation, but I think it illustrates how time might factor into their calculations.
Re: Rounding errors  Vieira, Luiz C. (Brazil)  11222009 Quote:In this case, we actually have an algorithmic error.
Luiz (Brazil)
