Precision medicine data management platform for streamlined healthcare data analysis thanks to infrastructure upgrade

Innowise enhanced an advanced data management platform for precision medicine diagnostics, streamlining the analysis of diverse healthcare datasets to accelerate patient-treatment matching and provide critical insights for drug development.

Customer

Industry

Healthcare

Region

Client since

2023

Our client is a pioneering company in precision medicine diagnostics. Their product serves as a critical intermediary between medical facilities, patients with conditions like cancer or heart disease, and pharmaceutical companies developing treatments for these diseases. The advanced data management solution aggregates and analyzes diverse datasets, including lab test results, patient outcomes, medicine efficacy to precisely match patients with appropriate treatments and clinical trials, while also providing valuable insights to pharmaceutical companies for drug development and targeted patient population identification.

Challenge

Inefficient data processing pipelines and environments setup

The company faced significant inefficiencies in their data processing pipelines and environment setup, hampering their ability to effectively aggregate, process, and analyze critical diagnostic test data from multiple sources. These inefficiencies led to delays in data availability for both data engineers and end-users, potential data quality issues, and suboptimal resource utilization in their AWS infrastructure.

The client also experienced challenges with adding new users and managing permissions for existing users within the AWS environment. The Innowise team consisting of DevOps engineers and data scientists was entrusted with these tasks.

Solution

Upgraded data management solution with optimized infrastructure, and enhanced security

Our experts led a comprehensive overhaul of the client’s software to implement a multi-faceted solution.

CI/CD pipelines optimization

Our DevOps engineers redesigned the infrastructure workflows to improve its efficiency and scalability. We performed profiling of the existing data pipelines to identify gaps and then optimized data structures and formats to reduce redundancy and improve processing efficiency. To further speed up data transformation and analysis, the experts implemented parallel processing techniques. We also improved and refactored code to enhance its maintainability. These efforts resulted in a streamlined, high-performance data pipeline system.

Environment optimization and deployment

We optimize the utilization of AWS cloud infrastructure by right-sizing instances, and implementing auto-scaling. We also applied Infrastructure-as-Code principles using Terraform to automate provisioning and management of cloud resources. Docker helped to containerize the data processing environment for consistency across development, testing, and production. A CI/CD pipeline was established to automate code integration, testing, and deployments. We also set up automated testing for the environment to timely catch configuration issues.

User and permission management optimization

We implemented AWS IAM best practices to enhance user and permission management. This included creating policies based on the principle of least privilege and setting up multi-factor authentication (MFA) for all IAM users. We optimized EC2 instance types based on workload analysis and set up CloudWatch alarms for proactive monitoring. Furthermore, to mitigate security risks, we developed automated scripts for user management and permissions.

Technologies

Back end

Python

Cloud platform

AWS

Infrastructure as a Code

Terraform

Containerization

Docker, Amazon EKS

Database

AWS RDS

Security and access management

AWS IAM, Secret Manager

Monitoring and logging

AWS Cloudwatch, Grafana, Prometheus

CI/CD

GitHub Actions

Compute service

AWS EC2

Process

Our project to enhance the precision medicine data management platform followed a structured approach, ensuring each aspect of the solution was aligned with the client’s needs.

Understanding requirements

We examined the client's data processing pipelines and AWS infrastructure, pinpointing inefficiencies and areas for improvement.

Architecture redesign

We restructured the system to enhance data handling, scalability, and security within AWS.

Agile development

Using Python and related tools, we improved back-end processes, data structures, and implemented parallel processing techniques.

Infrastructure automation

We created Terraform scripts to streamline AWS resource management.

Containerization and CI/CD

We containerized the data processing environment with Docker and set up automated integration, testing, and deployment pipelines.

Testing

We evaluated data processing speed, accuracy, system reliability, and IAM security measures.

Team

Project Manager

DevOps Engineers

Data Scientists

QA Engineer

Results

Increase in data processing speed, reduction of cloud computing costs, and seamless data exchange

The implementation of our solution led to significant improvements in our client’s’ data management capabilities.

Data processing speed: the optimized pipelines reduced data loading times by 35%, allowing faster access to processed data.
Resource efficiency: the reconfigured AWS environment led to a 29% reduction in cloud computing costs.
Data exchange: the seamless data exchange process now allows pharmaceutical companies to access relevant patient data faster.

Project duration

October 2023 - January 2024

35%

reduction in data loading times

29%

decrease in AWS cloud computing costs

Related cases

Contact us!

Book a call or fill out the form below and we’ll get back to you once we’ve processed your request.

What happens next?

Having received and processed your request, we will get back to you shortly to detail your project needs and sign an NDA to ensure the confidentiality of information.

After examining requirements, our analysts and developers devise a project proposal with the scope of works, team size, time, and cost estimates.

We arrange a meeting with you to discuss the offer and come to an agreement.

We sign a contract and start working on your project as quickly as possible.

Precision medicine data management platform for streamlined healthcare data analysis thanks to infrastructure upgrade

Customer

Challenge

Inefficient data processing pipelines and environments setup

Solution