A large European investment bank had a legacy software application in the form of a customer-facing trading system. Every time a change was made to the application, a huge testing effort was required to ensure the new release had not affected the system’s existing functionality. Failure to do this correctly could severely compromise the reputation of the bank and in turn lead to huge financial losses. Done manually, this testing process was both time consuming and unreliable and was costing the company a lot in time and money.
The Challenges
- To start the whole testing project from scratch as there were no business requirements documents in existence
- To convince all stakeholders that it was necessary to change some of the code in the legacy system in order to successfully complete the project
- To ensure all requirements were forecast in advance and all issues highlighted upfront
- To ensure enough time was allocated to accommodate automation requirements because the application owners were themselves very busy in production maintenance
- To build all test cases from scratch because the application had no existing manual test pack
- To ensure accountability and clarity at all stages of the project
- To draw up a proper proof of concept (POC) to ensure compatibility of all the technologies used
- To manage client expectations and ensure their requirements were reasonable, particularly given that this was the first automation project for the department
- To adhere to the client’s expectation of a quick return on investment
Brickendon Solution
1. Implement Brickendon’s specialised approach of Time Check Point System (TCPS). This ensures the exact scenarios are clear to both testers and stakeholders, as well as developers, project managers and sponsors.
TCPS is a very accurate tracking system designed to estimate and allocate required resources accurately as well as prioritising and categorising relevant tasks, providing for a standard testing framework.
- Approaching the problem in this way brought clarity to everyone involved in the project, and made it much easier to track the progress and ensure all parties were aware of their accountability. In short, it added clarity to the overall schedule and ensured the project and stakeholder teams were aligned right from the very beginning of the project
- The client was then given a timeframe to review the check point list and make any desired amendments before the scope was frozen
- By giving the client a defined window in which to refine the brief, the parameters did not change once the project was in motion, removing the need for additional reviews or consultations, and facilitating the testing team’s ability to meet the required deadlines
- However, the use of a very defined project scope meant that if there was a requirement to deviate from the original plan, Brickendon’s testing team could quickly assess the impact of the proposed changes and measure the increased effort in both time, money and resource required to implement the change. A timescale was estimated for each TCP using a complexity analysis system, giving the project credibility and transparency
- The checkpoints were divided into seven different deliverables, with deadlines every two weeks, meaning it was easier to stay on track and there was no surprise at the end of the project
2. Implement Brickendon’s unique Functional Test Automation Solution (FTAS) to ensure best coverage for test data and test case scenarios.
FTAS prioritises test cases by associated risks and uses a hybrid model based on a keyword and data-driven framework. The library contains all possible parameters for intelligent testing, enabling best coverage for test data and test case scenarios. It also ensures reusability and very high test coverage with minimal effort.
3. Initiate use of Brickendon’s innovative Integrated Test Tools Approach ITTA methodology, ensuring the automation of the whole testing process.
- This meant that when the automation test suite was triggered, the failures were automatically logged in a bug tracking system and an email distributed to the developers automatically.
- This methodology also ensured that the requirements, test cases, test results, defects and releases were all fully integrated.
ITTA is designed to reduce the need for manual intervention in the management of the test lifecycle and to consolidate the multiple test tools used.
4. Brickendon instigated a three-dimensional tracking process in the name of nailing down accountability:
1) Internal delivery date (for peer review)
2) Client delivery date
3) Client sign-off date
- If the first deliverable is missed then everyone involved is immediately aware of the delay and this can be addressed. Every process is documented and its progress is strictly followed.
- This not only helps to ensure the project is running on time, but also helps in the management of client expectations.
- A traffic light system was devised for the project, whereby its status had to be reported in terms of red, amber or green depending on progress.
- This is tracked and reported on a weekly basis. This helped the project stakeholders to have a transparent view of the project’s progress and to help them make informed decisions.
By increasing clarity everyone involved in the project knew where they were at every stage of the process and every individual felt accountable for their part. This helps testers retain responsibility for their part of the process and allows problems to be flagged up early. It also helps increase efficiency of management and this allowed the managers to exponentially improve the quality rather than simply firefighting fundamental issues.
Client Benefits
- This is the only test automation project completed on time within the bank during the last three years.
- A transparent and accountable automated testing project with detailed project outline, precise delivery dates, and accountable managers
- 360-degree stakeholder and management involvement – regular updates outlining project progress on a daily, weekly and monthly basis, flagged by a traffic light warning system to highlight any problems or delays as soon as they arise. Highlights to project sponsors directly what value is being added and where their money is going
- No fake completion dates – all dates are real and you can only invoice once you have completed the deliverable
- Increased productivity due to shorter deadlines and no big rush at the end
- Green status was maintained throughout the project, meaning that every single deliverable was completed on time
- Framework and methodology used to run the project could be reused around the business and in other departments
- Guarantee of huge test coverage with minimal effort
- Ability to release the software into production with full confidence that there will be no adverse impact on either your reputation or financial situation
- Considerable time savings thanks to the use of automation, which enables thorough testing to be run overnight and results to be published automatically through Brickendon’s ITTA technology. There was previously a three-week time lag
- Use of Brickendon’s TCPS methodology assured a very efficient test estimation and tracking system, which laid the foundation for the project’s success
