Medical research software: 60% reduction in manual data handling

Innowise executed a multifaceted medical research software upgrade for an ontology provider, incorporating AI-driven search, custom data dashboards, and ontology integration into a chemical research company’s infrastructure.

Customer

Industry

Healthcare

Region

Client since

2022

Our client, a leading entity in the ontology domain, operates in Germany. The company specializes in the development of technologies that extract information from structured and unstructured data, transforming it into knowledge for research, discoveries, and decision-making. Their expertise spans across chemistry, biology, and related scientific fields. They own a vast ontology system, a structured framework of interconnected scientific terms and concepts.

Detailed information about the client cannot be disclosed under the terms of the NDA.

Challenge

Limited search and annotation features, complexity in dashboard creation, and legacy system's manual document handling

The primary challenges our client faced centered around three main areas: developing a front end for their AI-powered search system, automating data visualization within medical research software, and integrating their ontologies into an existing chemical research company’s system:

Developing AI-powered search system’s front-end: The client’s primary challenge was enhancing the front end of their ontology-based search system, tailored for web and mobile platforms. This system was integral for managing an extensive collection of scientific articles. The upgrade required facilitating search capabilities, viewing sources, and annotating scientific concepts and terms within various document formats. The previous system’s limitations, notably the lack of search filters and annotation functionalities, impeded the full utilization of their scientific database.
Automating data visualization for scientific research:< The client faced a challenge in automating data visualization for their scientific data analysis. The required system needed to support data scientists in identifying, preparing, and validating data, as well as in creating informative dashboards. This was crucial for classifying and linking medical entities, identifying molecular targets for new pharmaceuticals, and facilitating research on diseases.
Ontology system integration in chemical research: Integrating the client’s ontologies into a chemical research company’s existing system presented a unique challenge. The company’s legacy system heavily depended on manual processes for document handling and data entry. Our task was to modernize this system by automating document analysis and database uploads, developing a new interface, and establishing a back-end system. This system had to support two distinct user roles: individuals responsible for uploading and editing documents and administrators for reviewing and confirming these entries.

Solution

AI-powered search system, automated dashboards, and seamless ontology integration in chemical research

Innowise’s team focused on three key aspects of the project:

Enhancing a specialized AI-powered search system for web and mobile interfaces.
Automating data visualization for pharmaceutical research through dynamic dashboards.
Integrating an ontology into a chemical research company’s system, thereby modernizing their document analysis and database management.

Enhancing the search system in medical research software

Our team focused on developing and enhancing a specialized AI-powered search system – a key subsystem within a larger framework, designed for web and mobile interfaces. This task involved multiple technical and functional improvements:

Advanced document search: We enabled the system to conduct in-depth searches across various document formats from a vast document repository. The system allowed users to locate documents, view internal and external sources, and identify key scientific concepts and annotations highlighted within these documents.
Annotation and categorization: A critical feature was the ability for users to select specific words or tickets within documents and assign them to relevant domains for annotation. We integrated GPT-based AI features to guide users in correctly annotating and categorizing each term or entity.
Submission and review process: Upon making edits or adding new information to a document, the system facilitated a review process. Users could submit these changes, which would then be sent to an administrator for status assignment and approval of the new annotations, comments, or categorizations.
Query and analysis features: Users now are able to select documents from a large database and add them to a collective basket. They can then query these documents using the search bar in the Analyzer, asking specific questions or requesting summaries and analyses based on GPT technology.
Custom filter development: Our developer created sophisticated filters for document search, tailored to various source types.
Document viewer challenges: One of the complex tasks was developing a document viewer capable of displaying marked annotations on PDF documents. This required intricate back-end coordination to overlay annotations correctly.
Legacy code and architecture overhaul: We addressed the challenges presented by legacy code and lack of architectural structure, ensuring the system was built on solid, modern technological foundations.
Integration of multiple GPT versions: Our team enhanced the system with multiple versions of GPT (3.5, 4, Davinci), enabling more versatile document analysis.
LLM integration: Innowise focused on custom LLM development that allows users to enter queries in natural language. Once the queries have been converted into back-end requests, they can be sent to the server.

Data science dashboard automation

Our data science team focused on automating data visualization through dashboards, a crucial component for the client’s research in identifying molecular targets for new pharmaceutical treatments. The primary diseases under study included obesity and muscle diseases.

Dashboard creation: The team’s objective was to create dashboards for visualizing pharmaceutical data. This involved processing large datasets, which are a vast number of annotated medical articles with unique ID and metadata, to form sizable GBQ tables.
Data visualization: Using Looker Studio, we transformed these large data tables into smaller, more manageable formats for dashboard creation. This visualization step was necessary for experts to better review and filter data.
Dashboard automation: Post-approval by medical experts, we automated the dashboard creation using data engineering techniques. This involved using repositories containing SQL scripts to fetch required information. These scripts were scheduled to run at specific intervals, ensuring the dashboards remained up-to-date with the latest research findings.
Continuous updates and integration: Our solution allowed for the continuous integration of new relevant publications into the dashboards. This dynamic updating process was facilitated by Google Cloud Functions. It kept dashboards updated with the most recent data.
Query management: We handled queries through large tables, pulling out specific information based on search queries. The team then visualized these statistics in the dashboards and identified any issues in the search queries.

Ontology integration in chemical research

Our project focused on integrating our client’s ontologies into an established lab management software at a chemical research company. This task involved several key steps to modernize and automate their outdated system:

System analysis and research: We began with a thorough analysis and research of the client’s legacy system. This lab management software, primarily used for storing reports and research findings, was based on older Java versions and JSP technologies.
Developing new interface and back end: Our approach included developing a new interface and back-end system to automate the process of document analysis and database updating, which was previously done manually.
User types and functionalities: We designed the system to cater to two distinct user types:
Document uploader: Researchers who add documents to the system. After a document is annotated, it appears on a special page where the uploader can review the results, make edits, and confirm submission to the database.
Administrator: Responsible for reviewing and confirming document additions. This role involves a comprehensive review of documents, with the ability to edit, approve, or make changes before final database entry.
Back-end development and legacy code overhaul: Our developer undertook the task of overhauling the existing legacy code. This involved writing JSP pages according to client specifications and developing back-end functionalities (requests, responses, data processing, and database entry).
Admin interface development: We also developed an admin part of the system where the admin (typically the head of the research department) receives a notification with a link to the interface displaying information from the database.
Integration of ontology API: The core of our solution was integrating the Ontology API into the client’s lab management software. This API served as a point for sending document-related queries and receiving responses, which were then processed and displayed through the frontend before being sent to the client’s database.
Handling documents and data: In this system, documents uploaded to the ontological system were processed, and the resulting data was saved in the company’s chemical research database. This allowed for automatic analysis of documents and retrieval of important chemical compound information.
Full-stack development: Our developer worked as a full-stack engineer, dealing with both front-end and back-end aspects and ensuring seamless integration of all system components.

Technologies

Programming languages

JavaScript, TypeScript, Java

Front-end

React, react-pdf, Redux, Redux-thunk, React-redux, Primereact, SASS, Lodash, Axios, FileSaver, GPT-Tokenizer

Back-end

Spring Boot, Java with Lucene Libraries, Stardog

Data science and analytics

Python (Pandas, Numpy, Plotly, Matplotlib), GCP (Google Big Query, Google Cloud Storage, Cloud Run), Looker, Data Studio, Apache Solr, custom tools for data processing and visualization

Process

Our approach to the development process was methodical and adhered to Agile principles, which ensured flexibility and continuous improvement.

At the beginning, we conducted thorough research to understand the client’s needs and existing systems to deliver a detailed ‘Vision and Scope’ document. Based on the initial findings, we proceeded to design and develop the necessary features for each stream. Our team held regular sprint meetings to confirm that our work aligned with client expectations. All features were implemented and subjected to rigorous testing for performance and accuracy, with the client providing continuous feedback.

For effective communication and project tracking, we utilized Microsoft tools and Monday.com, ensuring a transparent process and real-time updates.

Team

Project Manager

React Developers

Java Developers

ML/Python Developer

Data Engineers

Results

50% faster annotation process, 60% reduction in manual data handling, and 3x speed increase in data search for scientists

In our collaborative effort with the client, spanning three key streams, we’ve made significant strides in advancing their scientific research capabilities. Here’s a snapshot of the actual results:

Streamlined search operations: Our team’s efforts in refining the search system led to a doubling of search speed, benefiting researchers in their quest for swift access to scientific data.
Precision in annotations: The introduction of the automated annotation system resulted in increased annotation accuracy, a critical factor for in-depth scientific studies.
Increased efficiency in data handling: By automating data visualization processes, we’ve effectively halved the time researchers spent on manual data handling, translating to more time for core research activities.
Enhanced data processing velocity: Data processing and visualization speeds were amplified threefold, marking a leap in handling complex datasets.
Optimized user experience: The modernized user interface of our systems has led to a notable uptick in user satisfaction, fostering better engagement within the scientific community.
Liberated research time: Automating routine tasks has led to a 60% reduction in manual data handling, liberating the researchers’ time, previously consumed by manual work.

Currently, our dedicated team continues to work diligently on the system, focusing on the development of LLMs to further refine and enhance the client’s system.

Project duration

July 2022 - Ongoing

60%

reduction in manual data handling

speed increase in data search

50%

faster annotation process

Related cases

Contact us!

Book a call or fill out the form below and we’ll get back to you once we’ve processed your request.

What happens next?

Having received and processed your request, we will get back to you shortly to detail your project needs and sign an NDA to ensure the confidentiality of information.

After examining requirements, our analysts and developers devise a project proposal with the scope of works, team size, time, and cost estimates.

We arrange a meeting with you to discuss the offer and come to an agreement.

We sign a contract and start working on your project as quickly as possible.