Digital workspace QA automation: 105 times faster testing

Innowise developed a QA automation system for an international digital workplace solutions provider, streamlining their testing process across multiple components including email, calendar, chats, and file storage.

Customer

Industry

Information Technology

Region

Client since

2022

Our client, Zextras, is an international provider specializing in open-source digital workplace solutions. Their flagship product, Carbonio, is designed to support telecommunications and internet service providers, with a focus on digital sovereignty and data ownership for organizations.

Challenge

Managing regression issues and unstable content in a complex, multifaceted digital workplace solution

Carbonio is an all-in-one collaboration and communication solution, offering an array of integrated features. Presented as both a web client and mobile applications, it includes its own email server, calendar, contact management, an office suite for collaborative document work, a virtual drive, instant messaging, video conferencing capabilities, and support for third-party email clients such as Outlook or Thunderbird. The platform ensures high availability and efficient data storage management.

As the platform continued to develop, the need for an advanced testing framework became increasingly apparent. Carbonio, encompassing a variety of components, each with distinct development environments and versions, presented considerable testing challenges.

Minor updates frequently led to regression issues, impacting not only individual modules but also their integrated operations. The client’s team also encountered challenges with unstable dynamic content which complicated problem-solving.

Solution

Implemented automated software testing services to streamline UI testing, manage regression and content loading issues

Within automated software testing services, we developed an automated QA solution to address the complexities of Zextras’ Carbonio platform. This system focused on streamlining the UI testing process across various components, including email, calendar, chat, and file storage. We integrated advanced automation tools to efficiently manage regression testing and dynamic content loading issues. The system was tailored to work with different development environments. By leveraging our expertise in QA automation, we enabled more effective coordination among diverse teams and significantly improved the testing efficiency for the multifunctional Carbonio platform.

Automation architecture

To meet the specific requirements of Zextras Carbonio platform, we designed an advanced automation architecture that closely mimics real user interactions and workflows. This architecture includes a number of key components:

Playwright framework: We chose Playwright for its ability to emulate user actions across different browsers and platforms. It’s capable of handling complex scenarios like file uploads, downloads, and interactions with various web elements.
CI/CD integration: Our team seamlessly integrated the automation suite into the existing Continuous Integration and Continuous Deployment (CI/CD) pipelines. This integration allows for automated test executions as part of the build and deployment processes, ensuring that any potential issues are identified early in the development cycle.
Dynamic environment support: We designed the architecture to be environment-agnostic. Tests can be executed on different environments by simply specifying the server URL. This flexibility is crucial for testing Carbonio’s modules in various configurations and setups.
Scenario-based testing: We developed a comprehensive set of test scripts that cover common user scenarios. These scenarios range from basic functionalities like email sending/receiving in the email module to more complex interactions in the file storage and video conferencing modules.
Data-driven approach: We employed a data-driven testing approach to run the same set of tests with different data sets, providing thorough coverage of various use cases and edge cases.
Reporting and monitoring: The automation framework includes detailed reporting and monitoring tools. After each test run, it generates reports that include information about test execution, screenshots of failures, and logs. This detailed reporting aids in quick identification and resolution of issues.

Critical path scenario coverage

As a part of automated software testing services for Carbonio platform, we focused on thoroughly covering critical path scenarios, which are crucial for the platform’s daily operations and functionality. Over the course of the first year, our dedicated team, consisting of two seasoned automation experts, successfully developed and implemented automated tests for over 650 of these vital scenarios. They encompassed key functionalities across the platform’s modules, including email transactions, calendar scheduling, file storage operations, and chat interactions.

Browser support and resource optimization

Within our automated software testing services, a key focus was on ensuring comprehensive browser support while optimizing resource usage. We targeted the most popular browsers – Chrome, Safari, Firefox, and Edge. To preemptive problem detection, our automation was executed on the browser drivers for Chromium, Webkit, and Firefox.

This approach streamlined our resources and expedited the testing process. By leveraging the shared Chromium engine used by both Chrome and Edge, we were able to conduct tests primarily on Chromium, minimizing the need to duplicate efforts across two browsers.

Moreover, our ability to detect issues in the browser drivers provided an advantage: by identifying and addressing problems in the drivers, we could resolve them before they manifested in the actual browser versions, which typically update to the new driver versions after a 1-2 week delay. This proactive stance in problem-solving ensured that the platform remained stable across all supported browsers.

Proactive problem detection

Our problem detection system greatly contributed to the integrity of the Carbonio platform’s testing process. Utilizing our automation tools, we were able to detect issues with browser drivers at an early stage. This foresight was instrumental in preempting potential problems that could arise in later browser versions, allowing us to apply fixes before they could impact the user experience.

Comprehensive reporting system

In parallel, we implemented a comprehensive reporting system that significantly enhanced our ability to diagnose and address issues swiftly. Each automated test generated detailed reports, which included not only textual information but also visual aids like screenshots and videos. These reports were especially useful in scenarios where visual confirmation of a test’s outcome was necessary. Moreover, the reports contained logs of all application requests, providing our developers with granular data for pinpointing the root causes of content loading issues.

Technologies

AQA

Playwright, Fiddler, Jenkins, Allure, GitHub

Back end

Java 11, Java 17, LDAP, Soap, Maven, Linux, Bash, Perl, AWS, Java Lambda

System engineering

Prometheus, Grafana

Version control

GitHub

Process

Our process for automated software testing services unfolded through a series of structured phases, each augmented by our technical choices and strategic methodologies:

Requirement analysis: Our initial step involved deep-diving into the client’s needs and the specific challenges posed by the Carbonio platform.
Tool and framework selection: Based on the requirements, we selected Playwright for its compatibility with various browsers and robustness in handling complex web interactions. Jenkins was chosen for CI/CD to ensure smooth integration and continuous testing throughout the development lifecycle and to interact with development pipelines.
Agile development approach: Adhering to Agile principles, our development process was iterative and flexible. We regularly collaborated with the client through tools like Jira for task management and Carbonio itself for meetings and discussions, ensuring adaptability to changing requirements.

Test case development and execution: Our testers developed a comprehensive set of test cases covering key path scenarios. These tests were executed across multiple environments, providing thorough coverage.
Continuous integration and deployment: The automated tests were integrated into the CI/CD pipeline, enabling consistent and efficient testing with any build.
Reporting and feedback loop: With an advanced reporting system in place, we provided detailed feedback on test executions, including screenshots, videos, and logs for a rapid issue identification and resolution.
Ongoing support and optimization: Post-implementation, we continued to provide support and optimization services, refining the automation scripts as the Carbonio platform evolved and expanding test coverage.

Our focus remained on delivering a solution that was not only technically robust but also aligned with the client’s long-term goals for their digital workplace platform.

Team

AQA Engineers

Results

630 automated tests covering critical functionalities and 1890 autotests per hour across browsers

The automated software testing services for Zextras Carbonio platform have not only met but exceeded expectations, yielding improvements in product quality, operational efficiency, and client satisfaction. Continuously evolving, our automation solutions are playing a crucial role in the platform’s ongoing success. Enhanced results include:

Quality improvement: Automation led to a significant increase in testing efficiency, with prompt identification and resolution of regression issues, resulting in more stable and shorter release cycles.
Enhanced performance: Automated software testing services streamlined various platform aspects, reducing time in detecting and resolving issues, and improving the handling of dynamic content loading.
Operational efficiency: Our efforts greatly reduced the time and resources required for manual testing, contributing to a more efficient workflow within the client’s teams.
Speed and coverage: Our system is able to run 1890 tests across multiple browsers like Chrome, Firefox, and Safari in just 1 hour. The automation framework covers crucial platform functionality and is continuously expanding to include more platform parts and user scenarios.
Time efficiency: Our automation framework runs 630 user scenario checks in only 20 minutes, compared to 35 hours required for manual testing in one browser environment.
Insights and reporting: The comprehensive reporting system with clear graphic integration aids all team members, including non-technical personnel, in quickly assessing test outcomes.
Bug detection and prevention: More than 70 bugs have been detected and reported through our automated software testing services, not including duplicates found in manual testing in earlier development stages.
Agility in release cycles: The automation has been crucial in instances where rapid quality assessment of builds was needed, particularly in the final days of QA sessions before releases. It ensures that any last-minute issues are identified and resolved promptly.

Our ongoing work continues to focus on optimizing and expanding the capabilities of our automation solutions, further contributing to the platform’s success and reliability.

Project duration

July 2022 - Ongoing

630

automated tests covering critical functionalities

105

times faster testing

1890

autotests in 1 hour across multiple browsers

Related cases

Contact us!

Book a call or fill out the form below and we’ll get back to you once we’ve processed your request.

What happens next?

Having received and processed your request, we will get back to you shortly to detail your project needs and sign an NDA to ensure the confidentiality of information.

After examining requirements, our analysts and developers devise a project proposal with the scope of works, team size, time, and cost estimates.

We arrange a meeting with you to discuss the offer and come to an agreement.

We sign a contract and start working on your project as quickly as possible.