One of Ireland's leading financial institutions used Enovation to improve test practices. From the outset one of the primary objectives of the project was to minimise the drain on the business and allow the UAT phase to focus on its intended role of ‘final verification’.
Enovation Solutions were chosen as the test services provider on this project because of the long standing partnership with this client in the development area and previous consultancy work in test process improvement.
The Challenge
During an audit of the banks testing activities in 2005 Enovation found that one of the main problems lay with unclear roles and responsibilities between system test and UAT.The UAT team believed the system test team to be running a full test pass, and therefore expected high quality deliveries. While the system test team saw the UAT team as a safety net that will cover areas they simply don’t have time or expertise to get to. As a result bugs were reaching UAT that should have been found in system test. This subsequently increased the time required to perform a comprehensive UAT test pass and therefore either pushed the cycle beyond the targeted release date or forced a reduction in coverage and as a result introduced potential quality risks.
Added to this was the fact that over reliance on business resources was increasing the cost of the test cycles and diverting valuable staff from their day jobs. Enovation also found that there was no industry standard approach to test case creation or test case tracking, which meant that quality was not consistent across all projects.
A Professional Approach
From the beginning of the project Enovation provided the institution with a team of professional system testers to increase upstream testing and reduce the drain on the business at the UAT stage. Enovation’s testers have many years experience and are fully trained in the tools and techniques of the testing industry. The testers did over 500 man days of system testing, including the creation and execution of 3000 system test cases. A direct result of this was a greatly reduced requirement for UAT testing, with fewer than 1000 test cases needed for UAT. This highlighted the fact that in this new model UAT was focused on just the task of user acceptance and devoid of the overlap with system test, frequently seen in previous releases.
As part of the overall project Enovation also provided:
- A Test Case Suite - Enovation introduced industry standard test case creation methodologies to allow the institution to create test suites in a more structured manner.
- Test Cycle Analysis - Enovation managed the implementation of both a process and a tool for estimating test activities and the inclusion of these activities into project schedules.
- A Test Management Tool – Enovation provided an in-house application which could be accessed online for greater transparency.
- Issue Reporting System - Enovation implemented a detailed and easily controlled bug tracking system to help increase workflow and record changes.
- Project Tracking Tool – Enovation provided a project tracking and control tool which focused on milestones, red flags, scope changes and assigned actions.
Benefits
By introducing an independent system test team from Enovation Solutions and using proven industry tools and techniques the result was; more effective system test cycles, shorter UAT cycles with users focused on formal business verification not bug finding, and significant cost savings to the business.
It was very clear when the testing cycles were complete and the metrics analysed that the new approach had been successful. We had significantly improved the quality of the releases entering UAT by introducing an effective and best practice driven system test cycle. As a result UAT was very focused on their own objective and not the elimination of bugs. What typically had been a multiple release cycle was reduced to a one pass phase, in effect cutting the time and cost dramatically.
