Many government organizations and heavily regulated industries use a separate group to perform Independent Validation & Verification (IV&V) of software before it is deployed. Sometimes this is tradition, but in many cases, it is mandated by higher authorities. How can we help these organizations increase their agility, while also retaining the benefit of this independent view? In this first part of a two-part series, I will examine Traditional IV&V and the hybrid approach that is typically employed when development teams use Agile methods.
Before we discuss IV&V approaches, let’s segue into the purpose of IV&V. We value an independent view because the application or system being developed is critically important. If it is not implemented correctly, the organization will be at significant risk: the larger the system, the greater the likelihood it contains some critical element. Organizations that require IV&V rely on an outside, independent group—usually a testing body—to validate the system’s functionality and ensure nothing is missed.
Traditional IV&V
Traditional IV&V establishes a number of patterns and constraints. Figure 1 shows a typical phased-gate development effort. Once requirements are finalized, the IV&V group builds its tests independently to validate the behavior of the application. When the development group finishes its development and testing, the system is handed over to IV&V for acceptance. This usually requires installation to a specific environment.
Figure 1
If everything goes well, the software is deployed. However, if things don’t go well, rework will be required. Development may need to make changes, or requirements analysis will need to provide greater specificity. The primary problem is the length of such a feedback loop; it is difficult to make substantial changes so late. The traditional IV&V approach assumes that IV&V will be just a final check in the box.
A secondary problem is that automating the necessary tests can be very difficult since the environment and code aren’t fully known until IV&V takes possession of them. As a result, much testing is manual or leverages tooling that works against the user interface. These tests are typically brittle and difficult to change.
Another problem is cultural: Traditional IV&V contributes to an “Us vs Them” feeling. The fact that there is an “outside team telling us our problems” creates friction between the IV&V group and the rest of the organization. This clash is typically most visible between development and IV&V.
Hybrid IV&V
Some organizations maintain this same approach, with a separate IV&V team, when they transition to Agile. This often leads to specifying all the requirements (PBIs, stories, etc.) upfront, but some organizations get to a hybrid situation as shown in Figure 2.
Figure 2
The IV&V cycle takes place during each iteration (sprint). Once planning is complete, the development team works against the stories while IV&V develops tests for them. Following the review, the software is provided for IV&V testing. This can work but introduces a few problems.
- First, it assumes that stories will not change during the iteration. This idea ignores the adage: Card – Conversation – Confirmation. Stories frequently change during an iteration because they aren’t supposed to be fully articulated. The business (Product Owner) and the development team are both expected to learn more as it is developed. When this happens, IV&V is often out of the loop.
- Secondly, because IV&V testing begins at the end of the iteration, or, more correctly, at the start of the next, it is always at least one iteration behind. While this shortens the lengthy feedback cycle of the traditional approach, it still a long delay, and when issues are found they become disruptive.
- Thirdly, the IV&V group is simultaneously attempting to execute their tests and build the next test set. This either increases the size of the group (and resulting expense) or results in overworking. They may begin to take shortcuts and calculated risks.
And as you can see, the external group is still telling the development team its problems, thereby setting up the potential for cultural clashes just like with the traditional approach.
In the next post, I’ll discuss how IV&V can be more effectively integrated into Agile approaches.