Random musings:

Testing is getting paid by the bug?  http://www.utest.com/

“a unique pay-per-performance business model, to provide our customers with a cost effective solution to test any product,”
            “pay you based on your approved bugs submitted.”

Attended Risk Based Testing talk by Paul Gerrard as part of MS internal Engineering Excellence and Trustworthy Computer Forum (EE&TwC – as the invited speaker Scott Berkun  said – this contorted name shows a turf fight between groups that compromised on “&” rather than a useful, catchy phrase or acronym for people to remember.  The EE&TwC theme was design!).

Most interesting thing to me was in a list of Risk Reduction Strategies:          Information Buying
What is “Information Buying”?     It’s testing!
What a great way to really hit home about what testing is really about.
We test to gain information about a system to reduce risk.  You pay for the testing to get that information.

Note that paying only for bugs, means you have no idea what parts of your system are good.  Does no bugs mean it is good, or just that nobody tried it?

The other nugget from Gerrard was we should measure not by deliverables, was the code checked in, but by test evidence.   What was the pass rate, coverage, etc. of the checked in code.    If you have test evidence, you can infer the item being tested has been delivered.

I just gave a basic Model Based Testing (MBT) introductory talk to the VSTS (Visual Studio Team System) Test SIG (Special Interest Group).  Always amazed how few people have heard of MBT before.  I was asked how big a model you need to discover bugs.  The questioner had seen his own simple example of just 2 states.  I related the 2 state model I saw find bugs.  It was used by a tester on Indigo we had just taught MBT.  They were testing the .Net asynchronous calling pattern for several APIs.  Their simple model was:

This has just 2 states (invoked or not) and 2 actions (Begin or End).   Each action can conceptually occur from each state (4 transitions).   The ones in red should result in an error without changing state.    The tester then ran this simple model over about 50 APIs and found about 1/3 of them failed doing NotInvoked – EndInvoke ->

You can also see my article on Testing For Exceptions for why a sequence of more than one transitions might be useful.

I’ll be moderating a panel about Collaborative Quality at PNSQC and presenting Thursday, 9:45 “Patterns and Practices for Model-Based Testing” at StarWest.   Maybe I’ll see you at ISSTA in Seattle?

More conferences past and present about the MBT work of my Protocols Team.

Wolfgang Grieskamp : ETAPS MBT 2008 - Using Model-Based Testing for Quality Assurance of Protocol Documentation
Wolfgang Grieskamp : ICST 2008 – Model-Based Quality Assurance of Windows Protocol Documentation
Nico Kicillof : QSIC 2008 – Model-Based Quality Assurance of the SMB2 Protocol Documentation