As last night’s SASQAG meeting, James Whittaker again presented his Future of Testing talk (GTAC version still not on YouTube).

A statement near the end piqued me. 

            "Users will be genuinely surprised when it fails."

I think James is tainted by his past experiences.  There are past and current companies that already work like that.  My first job out of college at Tandem Computers warped my expectations forever.   As a fault tolerant computer company, not failing (hardware or software) was important.   In my first week there I found a trivial bug in their editor with a recursive macro called that crashed.  I was told to report the bug and get it fixed.  That was the attitude in general, all bugs reported and generally all bugs fixed.  There was of course triage and minor bugs deferred until next release, but I never saw bug tracking systems for tens of thousands of bugs until I came to Microsoft.   Long ago Tandem saw the criticality of avoiding data corruption and a fail fast philosophy.  Code reviews of the kernel were keen on what could go wrong.  Many others from that crucible ended up at Microsoft with an expectation for quality in their soul.  Many worked on Microsoft SQL with an expectation for outstanding quality.

The Agile community with their “Test Driven Development” and automated regression runs every check in help instill a quality expectation in developers.  But many already had it.  I have my Software Quality Engineering big red buttons from the mid 1980’s – “TEST then CODE”.  My favorite was a developer who bet me a dinner I couldn’t find a bug in his code.   Why the challenge?  Mutual respect and improvement.   I didn’t take the challenge.  Why?  The developer was good; they had done the right things.  I was not currently on their project (but had been months previously).    I could probably find a bug, but it would probably cost me my entire weekend and that wasn’t worth a dinner to me.   But the real point is the proper confidence of the developer.  Not arrogance or bravado.  He knew if I worked hard enough I would find a bug and he would be happy to receive it and improve his product.  To him, a dinner was a small price to pay for a better product.

My expectation is always that when I am asked to test a developer's work product it is because they believe it is ready to ship to the best of their ability.  They’ve code reviewed (peer review, pair programming, etc.) it and unit tested and run the standard tests available to them.  The only bugs I should be finding are those requiring resources outside the scope of a single developer, such as large scale stress testing or full system (integration) testing.