I've recently read a lot of posts about improving 'software quality', yet many of these posts look at a considered subset of what quality is, and very naively don't try to define it. A QA tester will often qualify quality by the number of bugs raised. A Product Manager may use the number of features in a product as a measure of quality. And end user, ultimately, will rate the quality of the software on his or her experience alone.
Earlier this year, I authored a project recovery paper, and within, I coined the phrase, the Quality Continuum.
As you may know, a continuum is a continuous succession in which no part is distinct or distinguishable from adjacent part. The Quality Continuum, as I defined it, is the notion of Quality that transcends the development phase. If you are gathering Marketing requirements, or performing your final System test before release, Quality must be defined in the same way. For instance, it is often said in software development, there are four variables: people, time, scope and quality. Back when I was a Development Manager, I remember our CTO telling me that quality is not negotiable, so shouldn't be on the list, Yet months later, as we were preparing to ship, he cut scope he also negotiated that certain Priority 3 (P3) and P4 level defects could ship to meet our deadline, and even demoted some P2's to P3.
To meet our deadline, we cut scope, which angered product management, as they considered the reducing release feature-set as lowering the product quality. By reducing the number of features, we also eliminated the potential for more defects.
It has become obvious to me over the years that scope and quality are intrinsically linked are are not distinct variables in software development. In his blog post title To Fix Or Not To Fix?: Another Good Question, Kent Beck argues that the XP concept of Defect Zero, eliminates the need to negotiate quality; defects are fixed as they are found. But herein are the issues; How do we define quality? Can all issues really be fixed in the given time? What if the issue is not easily reproducible or discoverable (eg- a Threading issue)? What if it is an edge case, and in 90% of the customer use cases, won't be an issue?Just as with software product requirements, you need to understand the boundaries and constraints of the problem, when considering product quality, I find it first very useful to define exactly what we taking about hen we say Quality. We've so far defined Functionality Scope and Reliability (Defects or bugs) as two of the most obvious attributes of quality. Another attribute I have encountered is Maintainability. Even is a given feature works are defined, and without defects, if it's design is so fragile that trivial changes easily cause new defects, isn't this of poor quality?
After reading Bob MacNeal post titled Quality's Okay, But Who's Paying the Freight?, I started searching for other characteristics of software quality, and found this reference to the ISO/IEC 9126 standard for Software Product Evaluation of Quality characteristics from 1991. Brilliant! Besides validating my attributes of Functionality, Reliability, and Maintainability, it also includes Usability, Efficiency, and Portability. In Scrum, I've included Usability and Performance (Efficiency) checks as 'Done' criteria, in the Review meetings, so they make sense as attributes as well. Also in my default 'Done' template, I include Security (is the code free of potential buffer overruns, SQL injection, etc). In the ISO 9126 model, Security is a a sub characteristic of Functionality, But I disagree with this hypothesis. Security should be considered an attribute on its own, as it is can be cross functional and it deserves more visibility. ISO 9126 was initially drafted in 1991, after all ;-)I've thought long and hard about including Portability as an attribute of quality. My initial reaction was it is not, as it isn't an issue with every project. I considered it as an optional attribute. But my final conclusion is that Portability is a requirement and therefore a sub-attribute of Functionality, as not all projects require it and not all programming languages supports portability (For instance C# & .NET, although arguably Mono makes this point somewhat mute), and often I've seen very portable code with lots of conditional includes that make the code very hard to maintain.
Every organisation I've worked for has negotiated what defects would not be fixed at release time, and some organisations use this forum to draft its Known Issues List in the release notes. When Ken states that an organisation shouldn't need a bug tracking tool, I wholeheartedly agree; the product backlog should be the place to keep requirements, features, defects, etc. One of the things I detest most about many project and product management software packages is the idea that defects are different from stories/features. The Product Backlog is the ideal place to keep track of stories/features and defects. But if bugs are found mid iteration, and completed in that iteration or saved until the next, realistically, something must give (Good estimation and planning practices will be covered in a subsequent post). Every organisation, and every project/product management software tools I've used allow a way of prioritising defects
Defining the Defect priorities
To be Continued .....
