Requirements Prioritization

Beyond the Limits of One-Dimensional Lists

Prioritizing requirements for a software release is an activity which frequently crosses the border between science and psychology. The goal is to determine the right set of things to do for a release. For many IT projects this turns out to be a moving target.

In software product development, requirement priorities are set by the product manager. Typically a product manager focuses exclusively on the market need of a requirement and its selling potential. Excellent software companies look at additional factors as well:

  • Cost
  • Risk
  • Fit to product strategy and architecture
  • Ability to deliver

Creating estimates for these factors cannot be done by a single person. Experts from different domains are needed.

There is a couple of techniques available to cope with this issue. Most of them are based on “cost-value” approaches. In my projects, I have applied a variety of them ranging from simple to complex ones. Often we started with something similar to Karl Wieger’s requirements prioritization approach (see www.processimpact.com).

Ultimately, these techniques yield a list of requirements ordered by their relative priority. Such a list is charming because you can always pick the most important requirement next. It is so charming, that this approach has made it into today’s most prominent development approach: Agile methodologies. In agile projects, the term “product backlog” describes a priority-ordered list of work items, which are addressed from top to bottom.

However, there are some shortcomings with these approaches, which need to be overcome in industrial practice. The key problem is the underlying assumption that requirements can be prioritized independently from each other.

Experience shows that this is seldom the case. Most requirements are interdependent from other requirements.

Across many industry domains including software and IT, the fundamental approach to building a system is always the same: A high-level plan is decomposed until the units of work are manageable. Elements of such units share some characteristics. They belong to the same domain and have similar complexity. Rating such elements in relative order works well. However, if the elements are from different domains, you may run into problems: They might have architectural dependencies which cannot be addressed in isolation. Many times, I saw projects getting stuck, because one team was waiting for something that another team had decided to lower in priority.

This problem of deadlock situations can be addressed by explicit dependency management. In requirements engineering, the concept of traceability is used to manage dependent work-items. Directional links express dependency relationships. These techniques exist and are available in most modern requirements management tools. However, in practice, these solutions can be very hard to accomplish, because they require more discipline than most projects are able to bring up.

So I recommend to deploy lightweight traceability using tagging mechanisms at requirements. Sometimes multiple backlogs are used. Each backlog holds items from the same domain. I have seen such approaches working pretty well in many project situations. So they might be the optimal solutions with the right mix of rigor and flexibility.

And don’t forget the importance of communication: Whatever approach one uses, it will be successful only if it is accompanied by good communication structures in the organization.

Two Essential Stages of Testing

Good test coverage is important and sometimes not easy to achieve. A simple principle can lay a solid foundation for test coverage: Distinguishing two essential stages of testing.

The initial low-level stage tests basic development artifacts immediately or soon after they have become available. This is usually called component, module or unit testing.

The later high-level stage tests the entire system or its higher-level aggregates as early as possible and lasting until very close before product delivery. This is usually called acceptance test.

The concepts of low-level and high-level tests are not new. Important is to relate them to different phases of the development cycle (or to activities within agile iterations, likewise; levels become stages) and to systematically plan associated activities. This way, good test coverage  can be achieved very efficiently.

Low-level testing stage High-level testing stage
Tests cover implementation of each basic development artifact from an implementation point of view (white or grey box testing) Tests cover design of the entire system or major system part from a business or usage point of view (black box testing)
Test cases defined by developers; derived from requirements and design Test cases defined by testers, domain experts from the software team, and/or customers; derived from explicit requirements or tacit domain knowledge
Tests conducted by developers Tests conducted by testers and/or customers or users
Defects usually fixed immediately or otherwise entered into defect database Defects usually entered into defect database, fixed, and being re-tested

I have seen many projects in trouble, because they did not properly address these test stages. Sometimes, low-level testing was replaced by pure faith (“My programs always run well”). Sometimes, high-level testing was shallow and ineffective (“We don’t have any time for more tests”). Often, the relation between both stages was not managed well, lowering product quality and limiting the efficiency of testing.

Taking care that both stages of testing are being addressed is a first and important step towards improved testing. Both stages are complementing each other well, so that higher test coverage can be achieved without very little planning and qualification efforts. This is also a good basis for subsequent improvement activities.

Additional details on the concepts of low-level and high-level testing are described in the testing literature, although the relation between the levels and phases of the development lifecycle is often not explored very much. A very instructive book is TMap® Next by Koomen et al. (2006). Another elaboration on the two testing levels are Brian Marick’s agile testing quadrants. Lisa Crispin’s presentation on  agile test planning provides a detailed explanation of those quadrants. Finally, an inspiring reflection on testing is Robert L. Glass’s text on The Many Flavors of Testing.

Quality Center Workflow Debugging

State of the art workflow debugging with HP Quality Center using Microsoft Visual Studio.

Users of HP Quality Center (HP QC) who do workflow programming often suffer from the very limited – not to say: lacking – support that HP QC offers for debugging of workflow code. It can easily happen that programmers spend hours in search of a tiny but nasty defect.

But there is help that greatly improves workflow programming and debugging in HP QC: You may use Microsoft Visual Studio (MVS) as a debugging environment for HP QC workflow programs. You can then add breakpoints, inspect variables, step through programs, and much more. All you need to do is the following:

  1. Launch HP QC
  2. Open the workflow editor in HP QC
  3. Launch MVS
  4. Attach the HP QC process to Visual studio
    (process name “iexplore.exe”, title “Script Editor”)
  5. Within project explorer window double click on “blank”
    (or “leer” in case you run a German version of MVS)

Final step to get HP QC into MVSNow, you should have your workflow code accessible within Microsoft Visual Studio, and debugging will be much more comfortable and efficient than before. See the attached picture to get an impression about debugging with breakpoint and variable inspection.

If you can’t see the line with “blank” make sure that you are in the requirements module and have at least a single custom line of workflow in the requirements module.

I got it to work with QC Version 10 and MVS 2008 Professional.