Web app AQA testing: 6x reduction in regression time

Innowise has integrated a comprehensive automation testing solution as a service into every stage of the development process.

Customer

Industry

Education

Region

New Zealand

Client since

2022

Our client is at the forefront of crafting transformative educational journeys that elevate and empower individuals and enterprises. By integrating cutting-edge learning technologies and strategies, they champion change by focusing on people-centric solutions, bringing unparalleled value to organizations and their members.Detailed information about the client cannot be disclosed under the terms of the NDA.

Challenge

Elevating web application efficiency

Our client came to us with a specific challenge. They needed thorough UI testing for end-to-end test scenarios across various browsers for their web application auditing service. On top of that, they aimed to enhance their application’s performance, both on the client and server sides. Along with addressing these primary concerns, they also wanted a detailed reporting system and a well-set-up test infrastructure to support a wide range of testing environments.

Solution

Conducting a comprehensive web application QA testing

Innowise provided a comprehensive and continuous testing cycle across all stages of the software development process. The AQA team shaped the testing requirements and formulated the test plan and strategy. Our active engagement and deep-seated expertise assured that the application QA testing process was rigorous and aligned with our client’s unique needs and goals.

Efficiency through parallel testing

In our quest to optimize web application QA testing, we employed a highly efficient approach – parallel testing. In this strategy, we executed twelve distinct test chunks concurrently, eliminating any potential time wastage that sequential testing might incur.

In parallel execution, test suites ran independently. This streamlined and agile testing method significantly expedited the overall testing process, enhancing productivity and efficiency while maintaining the highest quality assurance standards.

Playwright for comprehensive UI testing

With the increasing complexity of modern web applications, the need for a robust UI testing framework becomes paramount. Playwright came into our view not as a mere tool but as a game-changer. Offering support for multiple browsers, Playwright allows native interactions, ensuring tests that emulate real-world user behaviors.

Its ability to capture screenshots, record videos, and monitor network activities made it a comprehensive solution. Our team crafted an intricate set of test scripts, harnessing Playwright’s capabilities to their fullest, ensuring no stone was left unturned.

Application performance testing: lighthouse and k6

While the user interface plays a pivotal role in user retention, performance acts as the silent guardian that ensures a seamless user experience. LightHouse and k6 became our tools of choice. LightHouse, an open-source automated tool, significantly enhanced accessibility from 69 to an impressive 95. The output from LightHouse provided a rich array of recommendations, serving as a valuable guide for enhancement. Front-end developers harnessed these insights, using LightHouse’s targeted suggestions as a blueprint for refinement. This approach not only streamlined the development process but also significantly raised the bar regarding the application’s quality and user accessibility.

In its turn, k6, with its modern architecture, allowed us to generate a high load on the server side, simulating real-world conditions to provide a holistic assessment. By identifying bottlenecks and other potential pitfalls, we ensured that the application remained agile, even under duress.

Bridging gaps through API E2E tests

While our UI tests primarily focused on user interactions, we recognized the crucial role of data flow and server responses. To ensure a seamless user experience, we introduced API end-to-end tests that delved deep into the application’s core.

These tests acted as a vital link, harmonizing front-end user interactions with back-end data processes. They scrutinized data integrity and response times, mimicking real-world scenarios. We meticulously assessed data reliability, promptly identified discrepancies, and validated response times, ensuring a highly responsive user experience.

Our comprehensive approach provided a holistic evaluation of the web application, enabling us to identify and resolve potential bottlenecks, ensuring the application’s uninterrupted operation while upholding data integrity and responsiveness.

Comprehensive reporting & training

A robust testing strategy is incomplete without a clear reporting mechanism. Understanding the nuances of each test run, identifying recurrent issues, and formulating actionable strategies becomes vital. Playwright’s HTML reports, known for their granularity, provided micro-insights into individual test scenarios. When combined with the broad overview of the Report Portal, these transformed raw data into actionable intelligence.

But our endeavor continued beyond just providing insights. We organized extensive training sessions, ensuring that the client’s testing team was familiar with and proficient at AQA responsibilities. This included executing tests that detected bugs and navigating the bug life cycle particularly in the context of automation. Through workshops, hands-on sessions, and Q&A rounds, we empowered the client to take the reins and drive forward with confidence.

AI-based reporting mechanism

We improved the reporting process by integrating an AI-driven solution to identify and categorize test failures and inconsistencies. As part of the first phase, the AI system was painstakingly trained using data from previous test failures, where statuses were manually assigned. This foundational stage allowed AI to learn from a diverse range of outputs and accurately understand the nuances of different test outcomes.

In the second phase, post-training, the system suggested potential statuses for newly failed tests. These suggestions, provided to the AQA in the field, could be reviewed and approved as necessary. The AI was trained to identify a variety of statuses, including bug, automation issue, and infrastructure issue, thus speeding up the process of test analysis and enhancing the efficiency and accuracy of reporting mechanisms.

Technologies

Front-end

TypeScript, React Native, Redux, react-navigation

Back-end

Python, Django, Celery, Celery beat

AQA

TypeScript/JavaScript, Playwright, Lighthouse, K6, Grafana, Report Portal

VCS

Git, Bitbucket

Cloud

Elastic Beanstalk, DynamoDB, AWS Grafana IAM, SNS, SQS, KMS, RDS(PostgreSQL), S3, Lambda, SES, KVS

DevOps

Bitbucket Pipelines, DataDog, Nginx, Docker, Docker Compose

Process

Our AQA team used the Agile Scrum approach during application QA testing, which was essential for adapting to the client’s changing needs. This approach allowed us to quickly deliver vital elements like the Vision and Scope document after discovery and present a customer journey map with an interactive prototype during the UI/UX design phase.

We used Bitbucket CI to manage tasks effectively and focused on open communication to ensure everyone was on the same page. This straightforward strategy improved our teamwork, reduced obstacles, and ensured we always met or exceeded the client’s expectations.

Team

Project Manager

Back-end Developers

Front-end Developers

AQA

DevOps

Manual QA

Results

6x reduction in regression time and 3x increase in defect identification per sprint.

We created a comprehensive testing framework for UI, API, and web application performance testing, achieving 100% coverage of already available features. To maintain quality, we implemented nightly regression end-to-end and deployment-based regression tests, completing 2000 tests in just 25 minutes.

Moreover, we simplified reporting by using AI to identify failed and inconsistent tests. We established a simple email-based report distribution system to keep everyone informed. Additionally, we introduced visual regression testing for critical scenarios, boosting confidence in the testing process.

Project duration

May 2022 - December 2022

reduction in regression time

boost in defect identification per sprint

Related cases

Contact us!

Book a call or fill out the form below and we’ll get back to you once we’ve processed your request.

What happens next?

Having received and processed your request, we will get back to you shortly to detail your project needs and sign an NDA to ensure the confidentiality of information.

After examining requirements, our analysts and developers devise a project proposal with the scope of works, team size, time, and cost estimates.

We arrange a meeting with you to discuss the offer and come to an agreement.

We sign a contract and start working on your project as quickly as possible.