A programme continues to evolve after the go-live period as bugs/issues or further enhancement opportunities are identified continuously. Testing for changes made in the production environment is more limited in terms of the testing scope, which often endorses less significant enhancements, yet fails to highlight crucial functional issues. This set-back in quality is referred to as regression impact.
Regression testing involves creating an extensive testing framework which runs comprehensive testing after each and every change is made in the software under development. Without performing regression testing on each of the latest versions, it is difficult to rule out the possibility of having any regression impact.
Despite its importance, regression testing after go-live is challenging due to the lack of resources, fewer testing specialists available on site, out-of-date documentation and little or no budget left for comprehensive testing. This results in systems being released to the end-users without adequate testing. Brickendon Consultants highlighted that the majority of the unintended problems that occur in any major functionalities are found by the end-users after the release. This not only causes significant monetary loss, but also seriously damages a customer’s reputation in the market.
Absence of Regression Testing Resulting in Regression Impact:
Brickendon Solution
Brickendon Consultants designed the Continuous Testing Framework (CTF) to prevent any regression impact by enabling regression testing for when changes are made both during the project as well as after go-live.
Given that lack of resources is a key concern in the effective performance of regression testing, CTF aims to optimise the testing workflow through automation, using the available resources in the project, known as a lean workflow.
CTF is designed to ensure Regression Testing is carried out in the most efficient manner, whilst providing comprehensive testing during and after project completion. The CTF is a framework built on the principle of lean workflow which runs automated regression testing continuously (24×7), in a streamlined and simplified manner, without any manual intervention. In a fast-paced project environment, CTF enables rapid testing as and when any change is triggered in the application, thereby supporting the Agile Methodology as well as ensuring the resilience of the program.
The CTF solution uses automated functional test designs at each stage, running them parallel and continuously against the corresponding developed application. Automation scripts are an important part of CTF. If an automation solution is being implemented or is already operational, incorporating CTF can enhance the benefits of this automation.
Once the development team completes the initial coding, this is then deployed in the test environment. As regression test coding runs continually, the developers are informed of any test failure found in the initial coding block immediately, allowing developers to address development code issues in a timely manner. Compared to other testing procedures, CTF provides a significantly more efficient testing lifecycle processes.
Key Components of the Framework
There are four elements to CTF:
CTF obtains the latest automated test script from the version control system and executes on the latest Application Under Test (AUT) deployed in the Test Environment. CTF then runs and executes a complete Test Cycle, storing the results in a database which is then made available for publication, typically through an internal dashboard.
Implementing CTF in the early stages of a project can create more clarity in the project direction and decrease the turnaround time of test/build lifecycles. In a production environment, CTF can successfully complement the lack of human resource, without affecting the maintenance budget or the timeline. When using Patch Releases, running CTF against the Application (with the fix) drills down to the depth of the application in order to conduct comprehensive testing and ensure that bugs/issues are resolved successfully. Significantly, with CTF the quality of the programme does not regress, that is, any part of the resolution process does not cause any problem to key functionalities.
Strengths of CTF
CTF maximises the benefits of automated testing and extends them for relevant project participants, such as developers. As CTF runs continuously, based on the automation scripts, it reduces testing lead-time significantly as any errors made are immediately highlighted and therefore often quite easy to fix. Identifying problems as regressions and then linking the problem to the exact version (revisions) of the software where the problem was introduced, results in faster fixes. Its continuous testing capability significantly reduces the resource required to maintain the programme after go-live without compromising the quality. Testing teams can focus on designing more functional automation by saving manual efforts running regression tests.
Client Benefits
CTF is flexible and can be implemented in development as well as deployed environments. Shortly after the submission of the development code, CTF can produce a report highlighting issues, which means developers are able to obtain information about bugs and address them quickly, reducing turnaround time of build and test iteration. This results in delivering the best quality software, successfully avoiding any monetary loss and also maintaining the reputation of the company in the market.
