Do other brands also have buggy calcs?



#32

Hello all,

whereas I have the complete list of the bugs of the HP-35s, since it is well documented in this forum, I have no idea of the fact that programmable scientific calculators from other brands (TI and CASIO for the most well-known of them) could also have some bugs, and what kind of bugs they could be.

For instance, I wonder if the top of the range TI-89 and CASIO fx-9860g have bugs or not. I already hear some of you warning me that these calculators should not be compared to an HP-35s, but rather to an HP-50g, but I simply do not have informations of the bugs an HP-50g may have, that's why I mention the 35s.


Sorry for asking such a question here, but where else could I do it, and thank you advance for your answers.


Kind regards.

Jean-Michel.


#33

Seems like a very fair question to post here. But the answer looks pretty clear: just search the internet and you'll find a lot of sites with bugs and bug lists for those models.

#34

Last year my son called me from AZ and mentioned that his TI inspire failed to solve an Euler ODE!!! He mentioned that his TI-89 Titanium DID solve the same problem correctly. When he arrived for Xmas, I updated his OS for the nspire and we tried again. It failed again. We tried it on my TI nspire and that did not work too. We contacted TI and explained the problem. We heard nothing back from them.

It shows that even the best machines have bugs!

Namir


#35

I don't think I'd say that finding a bug in a TI necessarily shows that "even the best machines" have bugs. :-)

Any complex system has bugs. This is not unique to software, but it most visible in software because we built software systems of greater complexity than most other kinds of systems.


#36

I agree with you Eric. That is why operating system and even browsers (also complex enough) have vulnerabilities which hackers exploit.

The interesting thing though is that the TI-89 Titanium was able to solve my son's problem, while the TI nspire failed. That is why the TI-89 has become inseparable from my son.

Namir

Edited: 7 Nov 2009, 7:16 a.m.

#37

Quote:
Any complex system has bugs.


It shouldn't have to be. This is just an excuse perpetuated by manufacturers to reduce NRE and maximise profit.
"It's not perfect but good enough" is being used as an excuse to release rubbish.

Aircraft have complex safety critical systems with complex software. Would we accept the kind of bugs there that we see in consumer equipment?

Of course it's more expensive, but producers are more concerned with quarterly profits than quality products. What is better - having to buy several calculators to get the correct answers in all circumstances or one more expensive one that does everything right?

#38

You would have to test a complex system by arranging for any possible state it might be in to make sure all bugs are found. Is that possible for a modern calculator?


#39

It is not about testing all possible states. That can take a lifetime, even for hardware - imagine testing a modern high density FPGA for all possible input/ouput combinations!

It is about the development effort, properly executing the process of developing the system from requirements to testing and production. There is the idea that it's time consuming and expensive - but there always seems to be time & money go back and fix things a hundred times (or upsetting end users) rather than do it properly in the first place. I mentioned aircraft safety critical systems, but developing everything to that level would be "over the top". However, there are simpler processes to follow that could produce low bug-count systems for consumer use.

Coming back to testing, a properly thought out and executed test plan can ensure wide coverage without endless test effort. How difficult can it be to test the length/checksum function for programs on the HP35s?


#40

Quote:
It is not about testing all possible states. That can take a lifetime,

A lifetime? For an even moderately complex product it would take an astronomical number of times longer than the current age of the universe.

This is why code coverage analysis was invented. While it helps, it doesn't completely solve the problem.

#41

I have to agree with Bart, if I'm correct that he thinks it's a matter of expense.

Software and hardware QA are not new anymore, and there is no practical reason for the complexity of these systems to grow any faster than the methods available to test them.

The expense of testing has not become greater in proportion to other costs in the product life cycle, so if new products are actually more buggy, it's a result either of decisions not to test them as thoroughly as warranted by their complexity, or simply of being in an earlier stage of "incremental improvement" than other products in the comparison.

The decision not to test adequately would be a decision to cheapen the product.

Richard


#42

Aye there's the rub. I have no doubt they consider their tests adequate. Perfection takes a lot longer and time pressure doesn't permit perfection. I was reminded of the Mt. Palomar telescope, when the shop was asked how far short of perfection the mirror was, said about 100 years. Sam


#43

Quote:
...how far short of perfection the mirror was...


But how short of perfection did it need to be to meet the purpose of the telescope? That should be captured in the requirements. Once the requirements have been properly established, the design should endeavour to meet those requirements 100%. To use "it's not perfect, but good enough" as an excuse not to meet the requirements means wanting to release an inferior product.

Well defined requirements will also minimise requirement creep. Any "nice-to-have's" that pop up during design should be kept aside as new requirements for a possible product upgrade.
#44

Quote:
Software and hardware QA are not new anymore, and there is no practical reason for the complexity of these systems to grow any faster than the methods available to test them.

The practical reason why the complexity of the systems is increasing rapidly (and faster than testing) is that customers always expect newer products or models to have more features.

If you take the previous model of a product which contained 200,000 lines of code, add some features, and the new model contains 250,000 lines of code, you have NOT increased the testing problem by 25%. The increase is exponential, but improvements to testing performance and even to testing methodology are on a much shallower curve.

Until customers stop demanding added features, or a breakthrough is made in software verification, software quality will continue to deteriorate.


#45

Quote:
Until customers stop demanding added features, or a breakthrough is made in software verification, software quality will continue to deteriorate.

IMO customers don't "demand" new features, software companies invent new features to generate sales of "improved" versions.

Most customers I know would just be happy if the "old" versions could be made bug free.


#46

Customers "demand" new features because they buy software that is marketed touting new features. If they stopped doing that, and instead complained to the software companies about how bug-ridden the software is, things would be different.

It's called "voting with your wallet".

The reality is that most people simply put up with the bugs.

#47

In CAD software, there is a hugely important customer-driven evolution. Once you can do "a" with the software, you soon realize that if it could do "a + b" automatically, you'd save another 200 hours per year of modeling effort etc etc. Maybe word processors, spreadsheets, programmable toaster ovens and calculators suffer from "feature bloat," but in commercial CAD software, while there is "feature bloat" with respect to the parts of the package you don't use, there is always improvement (meaning, can it make the work go faster) to be done for your own use.

The packages that don't keep up with the customer demands, fall away and disappear. sometimes this is to the consternation of the customers, who have invested in knowing how to use the package. The CAD software market really depends on collaboration. Examples of what I am talking about would be Solidworks, Rhinoceros, Aerohydro, AutoCAD.

Edited: 8 Nov 2009, 12:52 p.m.

#48

Quote:
The decision not to test adequately would be a decision to cheapen the product.

"Adequately" is a judgement call; there is no a priori objective standard for "adequate".

The amount and type of product testing done is a business decision, based on guesses as to how much the quality of the software will affect sales.

#49

Note that high-end automobiles now contain many times more lines of code than the latest aircraft avionics. This is because for the aircraft avionics they know that they will have to extensively test every feature and many combinations of features under a very large range of system conditions (both plausible and implausible), and they still won't catch all the bugs. For the automobile the testing requirements are much lower, with exceptions for only a few subsystems.

With regard to aircraft software you write:

Quote:
Of course it's more expensive,

I'm not sure from your post that you have any idea just how much more expensive it is. We're not talking about 50% more expensive, or even 200% more expensive. We're talking about at least two orders of magnitude more expensive. You will NEVER see that level of testing put into a consumer product, because there is no possible business justification for it.


#50

There seems to be too much of a focus in this thread on testing. My argument is that planning receives too little attention. It is in the requirements capture and detailed software planning stages that the opportunity for "doing it right first time" exists. By the time testing is being done, it's at the point of "going back to fix it". So often I see programs where coding is done directly from poorly defined high level requirements because "there isn't money for that" or "there isn't time because we have to get something on the table". It's like having a big jellyfish on top of a thin pole, and with each test-and-fix iteration the jellyfish becomes bigger and more difficult to keep on the pole. Adequate high & low level requirements capture and detailed software planning & design can create the right size pole.

As to the effort and expense of aircraft software you are right when it comes to safety critical software. I did say in an earlier post that this was over the top for consumer products. Also, a big chunk of this added time and cost is due to the supporting documentation and independent verification required for safety critical systems. This is not needed for non safety critical software. Because of this, that type of aircraft software is considerably more complex than the safety critical stuff, but when developed well still has good performance - and is not that expensive. Some of those that complain about the expense usually under-estimated the cost at the bid stage and ended up paying several times what need have been, to fix it - i.e. it comes back to bad planning. Or didn't apply the process from the beginning and had to go back and re-do it.

On generic PC's I can understand problems exist because the software house has no control over the hardware platform that the software will be used on. However, calculator manufacturers have complete control over the whole platform.

Edited: 7 Nov 2009, 10:59 p.m.


#51

Bart --

That's a fine "mini-essay" with some very good points.

Several explanations for bug-ridden, flawed, and poorly-designed calculators and other consumer gadgets have been discussed in this space:

  • Relentless pace of advancing technology renders today's novelty tomorrow's obsolete relic. Why spend the money, time, and effort to make a first-class product that would be replaced in only a few years?
  • Race-to-the-bottom price competition, enabled by low-cost microprocessors as well as the cheap labor of globalization, leaving little margin for quality engineering.
  • Re-emphasis from "substance" (sensible, quality products) toward "style", gimmickry, and short-term profit.
  • Today's cheap, expedient way to fix errors by releasing patches to software and firmware via the internet.

It didn't used to be that way:

Even by modern standards, there was nothing simple about the detailed functional specifications of the HP-15C and HP-42S in particular. Yet, these models are known for near-perfection in operation, and excellence of design. Both pioneered new territory in certain ways, but -- with only a few trivial exceptions -- every feature performs exactly as designed. The vast majority of their functional designs themselves are also excellent.

This could not have happened by accident: Not only was the functionality carefully thought through, but systematic, methodical procedures must have been employed to ensure that nothing would be overlooked. Rigorous and comprehensive testing ensured that the original product release did not require significant revision or re-design. I'm always astounded how well the HP-15C and HP-42S were executed in their basically-single incarnations.

I believe that certain methods and practices of the not-too-distant past have been forgotten or discontinued in the development of much of today's consumer technology.

-- KS

(Minor editing to wordsmith)


Edited: 8 Nov 2009, 2:13 p.m. after one or more responses were posted


#52

Your bullet points explain what I was getting at in my first post - manufacturers maximising profit at the expense of quality, and then trying to smooth the waters by saying "ooo, it's a complex system...it's not going to be perfect you know".

Were the sales figures of the Pioneers and 48 series so poor that HP calculator devision felt they had to stoop to inferior competitors low quality to stay profitable?

#53

Good points, Bart.

This kind of says it all.

When I became a programmer in 1974, a critical aspect of that job was "desk checking" the program, which meant going through it line-by-line and ensuring that it was logically correct, even before compiling. I'm afraid that is no longer done. In fact, testing is sometimes left for beta testers these days.

I agree that the requirements definition process is absolutely critical to the development of a succesful project. But that's just one phase. What is really needed is smart people and knowledgeable and supportive management at every phase of the life cycle, and that's always tough to get.


#54

Quote:
What is really needed is smart people and knowledgeable and supportive management at every phase of the life cycle, and that's always tough to get.


Yup, that's what my "mini-essay" was all about.

What also gets my back up though is that the users are then persuaded to accept serious bugs are inevitable. By serious I mean bugs that prevent the equipment from performing to a hard and fast requirement. i.e. a calculator not adequately (or not at all) performing a function that is part and parcel of the machine. Examples are Namir's TI-inspire problem and the program length / checksum function of the HP35s.

#55

Remember when Excel did maths wrong a few years ago?

Nothing new can be trusted....


#56

Hi, Bill --

Quote:
Remember when Excel did maths wrong a few years ago?

Nothing new can be trusted....


If you're referring to the long-standing floating-point math results provided by MS Excel 2000 and later versions, here's a thread from 2007 that links to an earlier lengthy one from 2002:

http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/archv016.cgi?read=107533#107533

A link is provided by Gunnar Degnbol to a 2006 draft paper by W. M. Kahan about Excel mathematics.

There was a well-publicized display bug in Excel 2007, which apparently was patched by KB943075.

-- KS

Edited: 8 Nov 2009, 5:18 p.m.

#57

Quote:
There seems to be too much of a focus in this thread on testing.

No matter how well you design the software before you start writing it, you will still wind up with bugs in it that won't get caught until testing. That might be testing by your QA department, or testing by your customers.

If you start from a good design, you'll have fewer bugs, but not zero.


#58

I did not mean to belittle the importance of testing, but if the aim is to use testing as the first real port-of-call to minimise bugs then the battle is lost before it has been started.

Edited: 8 Nov 2009, 1:01 p.m.


#59

Sounds like we're in agreement.

#60

the enter key is broken on most non-hp calcs.


#61

:)))

#62

I recently got a Casio FX-3900PV that locks up all the time.

Casio's are usually bullet proof in terms of firmware, but this programmable model locks up all the time if you say pick up the calculator and accidentally press random buttons. You just see "busy" on the display and even the OFF button won't work.

No wonder is has a handy reset button on the back!

Dave.


Possibly Related Threads...
Thread Author Replies Views Last Post
  [Download] libhpcalcs: a toolkit for communicating with Prime calcs... debrouxl 13 1,700 11-18-2013, 05:22 AM
Last Post: debrouxl
  HP-35 blind buy but buggy! Max Stone 7 1,375 11-11-2013, 05:56 PM
Last Post: Dieter
  Swiss Micros calcs Mike Powell 6 1,032 10-26-2013, 05:03 PM
Last Post: Mike Powell
  Help me alert regular users to dangers to their calcs Bruce Larrabee 12 1,360 10-06-2013, 08:30 PM
Last Post: Bruce Larrabee
  USB Chargers for HP calcs Matt Agajanian 3 668 08-18-2013, 10:58 PM
Last Post: Craig Ruff
  Look on Fournier pressure sensor for HP calcs Mic 0 394 04-11-2013, 12:57 PM
Last Post: Mic
  Safe charging LED HP calcs Anoop Sahal 6 913 03-29-2013, 09:08 AM
Last Post: Anoop Sahal
  HP Calcs & 'NUMB3RS' Matt Agajanian 1 493 11-11-2012, 10:07 AM
Last Post: Eddie W. Shore
  Official revival of old calcs wildpig 7 910 08-20-2012, 04:11 PM
Last Post: Ingo
  DM calcs arrived, but... David Ramsey 4 721 08-04-2012, 12:42 PM
Last Post: Michael de Estrada

Forum Jump: