Automated software testing: 2x increase in functionality release speed

Innowise executed automated software testing to ensure that integrating new system functionalities did not compromise the integrity of the existing features.

Customer

Industry

Information technology

Region

Switzerland

Client since

2023

Our client is a provider of cloud ERP solutions designed to bolster operational efficiency and resilience against operational disruptions.

Detailed information about the client cannot be disclosed under the terms of NDA.

Challenge

Verifying the stability of the ERP system after new functionality integration

Our client introduced new functionality into their ERP system and integrated it with the existing build. They relied on Innowise to assess the stability of the deployed software build. We provided automated software testing services, aiming at identifying potential showstoppers in the system and fixing them.

Solution

Employing automated software testing services to identify and resolve potential showstoppers

Innowise elevated the client’s ERP system with a multi-faceted approach to automated software testing. It incorporated smoke, functional, and regression testing, all executed within a continuous integration/continuous delivery (CI/CD) pipeline. Smoke testing was employed to quickly verify the stability of key functionalities, while functional testing was vital in examining each feature for correctness. Regression testing was crucial in ensuring that new updates did not adversely affect existing functionalities. The integration of these testing methods into the CI/CD pipeline streamlined the development process. This approach enabled rapid identification and resolution of issues, and ensured a consistent rollout of robust, high-quality features in the ERP system.

Evaluation of test cases for automation

At the outset, we were tasked with combing through the test cases crafted by manual testers. Drawing on our extensive background in software testing, we delved into each case, manually performing the steps of each test case to assess its complexity and automation viability. Our objective was to determine the most efficient way to automate repetitive and time-consuming tasks without impairing the accuracy of test results.

Implementing smoke testing

Our testers set up a collection of pre-recorded smoke test cases that sprang into action against each new build. These tests were essential for confirming that the software’s major functionalities were running, assuring consistency and thoroughness. If an automated test flagged an issue, it immediately alerted our team, allowing for swift identification and action on the problem.

Our process’s flexibility empowered our team to swiftly address any issues uncovered in the build. We promptly implemented the required adjustments and redeployed the build without delay, ensuring minimal disruption.

Integrating functional testing in each sprint

We seamlessly transitioned from the broad-scope verification of smoke tests to the targeted and meticulous functional testing approach. This ensured that every new update was effective in isolation and blended seamlessly with the established software components. We performed functional tests every sprint so that each incremental development was rigorously evaluated, upholding the software’s overall quality and cohesiveness.

Software regression testing

We followed a streamlined yet thorough process for implementing automated regression testing. Our team set up a testing environment that mirrored the actual production conditions, ensuring our results were as accurate as possible. With Selenium, we automated regression test scripts, significantly boosting efficiency, a crucial factor for complex systems.

Once automated, we ran these scripts against the latest build to check for inconsistencies or failures. After running the tests, we thoroughly analyzed the results, pinpointing any emerging bugs or issues.

If we detected any problems, we promptly addressed them by tweaking the code and re-running the tests to confirm the effectiveness of our fixes. This cycle of testing, analyzing, and refining was a constant with each new software update, ensuring the software’s stability and intended functionality. This efficient approach to regression testing was key in maintaining the software’s reliability and performance, thereby reducing the risk of bugs and providing a smooth user experience.

Continuous integration/continuous delivery (CI/CD) pipeline

In addition to our rigorous testing process, we’ve also integrated our tests into the continuous integration/continuous delivery (CI/CD) pipeline, enhancing our development cycle efficiency and responsiveness. We utilized the Allure tool for detailed reporting, which provided clear insights into the test results. This integration allowed us to set up an automated system where tests were launched each night. This nightly testing routine ensured that any new changes were immediately evaluated, allowing for swift identification and resolution of issues. Incorporating Allure reports into our CI/CD pipeline meant that our team had access to comprehensive test data every morning, streamlining decision-making and maintaining a constant focus on quality and stability. This integration accelerated our testing process and reinforced the reliability of each software build we delivered.

Detailed steps in our CI/CD pipeline:

Code: Developers wrote and refined code, ensuring it met our standards for functionality and design.
Commit: We then committed the updated code to our version control system, marking the first checkpoint in the pipeline.
Build: This committed code underwent a build process, where it was converted into a runnable form, ready for testing.
Unit Tests: We subjected the build to unit tests, where individual components were tested for correctness. It was carried out to guarantee that each component functioned flawlessly.
Smoke Tests: Following the unit tests, we conducted smoke tests on the build to quickly ascertain if critical functionalities were operating as expected.
Commit: Post smoke tests, we committed the code again to the version control system, incorporating any changes made following the tests.
Review: We thoroughly reviewed each commit, ensuring that the changes adhered to our quality standards and aligned with project objectives.
Staging: The build was moved to a staging environment, which closely mirrored the production setup, for further testing.
System Tests: In staging, we conducted comprehensive system tests to validate the build in an environment that simulated real-world usage.
Production: Once the build successfully passed all stages of testing, it was deployed to the production environment, making it available for end-users.

Technologies

AQA

Python, Pytest, Selenium, Allure

Back-end

Java, Spring, Hibernate

VSC

Git, Gitlab

Cloud

AWS Grafana IAM, SNS, SQS, KMS, RDS(PostgreSQL), S3, SES

DevOps

Gitlab, Docker, Docker Compose

Process

Agile methodology and sprint structure

In our project, we strictly adhered to Agile methodology, structuring our development process into two-week sprints. This approach allowed us to break down the project into manageable chunks, ensuring consistency and efficiency in each phase.

Task management and team collaboration

At the start of each sprint, we laid out clear objectives and tasks, which were tracked using Jira. This tool kept our team aligned and on track. We employed it not just as a task tracker but as a central hub for collaboration and communication, so that every team member was always up-to-date with the latest developments and priorities.

Sprint review and detailed reporting

As we neared the end of each sprint, our focus shifted to consolidating our work and preparing for the sprint review. We carefully compiled detailed reports that highlighted our testing coverage and results. These reports were more than just a summary of tasks completed, they provided an in-depth analysis of what we had achieved, what challenges we faced, and how we overcame them. They also included feedback from our testing processes and any learnings that could be carried forward into the next sprint.

Team

AQA Engineers

DevOps Engineer

Results

Accelerated feature deployment and 50% cut in production bugs

Implementing automated testing for our client’s ERP system resulted in notable enhancements and efficiencies:

Faster release of new functionalities: with automated testing, new functionalities were released twice as fast compared to the previous manual process.
Reduction in bugs: The integration of CI/CD led to a 50% reduction in bugs reaching production, significantly enhancing the software’s reliability.

Given this positive outcome, our client has expressed their enthusiasm for our collaboration. They recognize the value that our expertise and the implementation of cutting-edge technologies bring to their business. Moving forward, we will work closely with our client to build upon the successes achieved and explore further opportunities to enhance their systems and processes.

Project duration

September 2023 - Ongoing

increase in functionality release speed

50%

fewer bugs reaching production

Related cases

Contact us!

Book a call or fill out the form below and we’ll get back to you once we’ve processed your request.

What happens next?

Having received and processed your request, we will get back to you shortly to detail your project needs and sign an NDA to ensure the confidentiality of information.

After examining requirements, our analysts and developers devise a project proposal with the scope of works, team size, time, and cost estimates.

We arrange a meeting with you to discuss the offer and come to an agreement.

We sign a contract and start working on your project as quickly as possible.