Comprehensive technological enhancements across multiple streams for an ontology provider

Innowise executed a multifaceted medical research software upgrade for an ontology provider, incorporating AI-driven search, custom data dashboards, and ontology integration.

Challenge

The primary challenges our client faced centered around three main areas: developing a front end for their AI-powered search system, automating data visualization within medical research software, and integrating their ontologies into an existing system:

Developing AI-powered search system’s front-end: The client’s primary challenge was enhancing the front end of their ontology-based search system, tailored for web and mobile platforms. This system was integral for managing an extensive collection of scientific articles. The upgrade required facilitating search capabilities, viewing sources, and annotating scientific concepts and terms within various document formats. The previous system’s limitations, notably the lack of search filters and annotation functionalities, impeded the full utilization of their scientific database.
Automating data visualization for scientific research:< The client faced a challenge in automating data visualization for their scientific data analysis. The required system needed to support data scientists in identifying, preparing, and validating data, as well as in creating informative dashboards. This was crucial for classifying and linking medical entities, identifying molecular targets for new pharmaceuticals, and facilitating research on diseases.
Ontology system integration: Integrating the client’s ontologies into an existing system presented a unique challenge. The company’s legacy system heavily depended on manual processes for document handling and data entry. Our task was to modernize this system by automating document analysis and database uploads, developing a new interface, and establishing a back-end system. This system had to support two distinct user roles: individuals responsible for uploading and editing documents and administrators for reviewing and confirming these entries.

Solution

Innowise’s team focused on three key aspects of the project:

Enhancing a specialized AI-powered search system for web and mobile interfaces.
Automating data visualization for pharmaceutical research through dynamic dashboards.
Integrating an ontology into a company’s system, thereby modernizing their document analysis and database management.

Enhancing the search system in medical research software

Our team focused on developing and enhancing a specialized AI-powered search system – a key subsystem within a larger framework, designed for web and mobile interfaces. This task involved multiple technical and functional improvements:

Advanced document search: We enabled the system to conduct in-depth searches across various document formats from a vast document repository. The system allowed users to locate documents, view internal and external sources, and identify key scientific concepts and annotations highlighted within these documents.
Annotation and categorization: A critical feature was the ability for users to select specific words or tickets within documents and assign them to relevant domains for annotation. We integrated GPT-based AI features to guide users in correctly annotating and categorizing each term or entity.
Submission and review process: Upon making edits or adding new information to a document, the system facilitated a review process. Users could submit these changes, which would then be sent to an administrator for status assignment and approval of the new annotations, comments, or categorizations.
Query and analysis features: Users now are able to select documents from a large database and add them to a collective basket. They can then query these documents using the search bar in the Analyzer, asking specific questions or requesting summaries and analyses based on GPT technology.
Custom filter development: Our developer created sophisticated filters for document search, tailored to various source types.
Document viewer challenges: One of the complex tasks was developing a document viewer capable of displaying marked annotations on PDF documents. This required intricate back-end coordination to overlay annotations correctly.
Legacy code and architecture overhaul: We addressed the challenges presented by legacy code and lack of architectural structure, ensuring the system was built on solid, modern technological foundations.
Integration of multiple GPT versions: Our team enhanced the system with multiple versions of GPT (3.5, 4, Davinci), enabling more versatile document analysis.
LLM integration: Innowise focused on custom LLM development that allows users to enter queries in natural language. Once the queries have been converted into back-end requests, they can be sent to the server.

Data science dashboard automation

Our data science team focused on automating data visualization through dashboards, a crucial component for the client’s research in identifying molecular targets for new pharmaceutical treatments. The primary diseases under study included obesity and muscle diseases.

Dashboard creation: The team’s objective was to create dashboards for visualizing pharmaceutical data. This involved processing large datasets, which are a vast number of annotated medical articles with unique ID and metadata, to form sizable GBQ tables.
Data visualization: Using Looker Studio, we transformed these large data tables into smaller, more manageable formats for dashboard creation. This visualization step was necessary for experts to better review and filter data.
Dashboard automation: Post-approval by medical experts, we automated the dashboard creation using data engineering techniques. This involved using repositories containing SQL scripts to fetch required information. These scripts were scheduled to run at specific intervals, ensuring the dashboards remained up-to-date with the latest research findings.
Continuous updates and integration: Our solution allowed for the continuous integration of new relevant publications into the dashboards. This dynamic updating process was facilitated by Google Cloud Functions. It kept dashboards updated with the most recent data.
Query management: We handled queries through large tables, pulling out specific information based on search queries. The team then visualized these statistics in the dashboards and identified any issues in the search queries.

Ontology integration in research

Our project focused on integrating our client’s ontologies into an established lab management software. This task involved several key steps to modernize and automate their outdated system:

System analysis and research: We began with a thorough analysis and research of the client’s legacy system. This lab management software, primarily used for storing reports and research findings, was based on older Java versions and JSP technologies.
Developing new interface and back end: Our approach included developing a new interface and back-end system to automate the process of document analysis and database updating, which was previously done manually.
User types and functionalities: We designed the system to cater to two distinct user types:
Document uploader: Researchers who add documents to the system. After a document is annotated, it appears on a special page where the uploader can review the results, make edits, and confirm submission to the database.
Administrator: Responsible for reviewing and confirming document additions. This role involves a comprehensive review of documents, with the ability to edit, approve, or make changes before final database entry.
Back-end development and legacy code overhaul: Our developer undertook the task of overhauling the existing legacy code. This involved writing JSP pages according to client specifications and developing back-end functionalities (requests, responses, data processing, and database entry).
Admin interface development: We also developed an admin part of the system where the admin (typically the head of the research department) receives a notification with a link to the interface displaying information from the database.
Integration of ontology API: The core of our solution was integrating the Ontology API into the client’s lab management software. This API served as a point for sending document-related queries and receiving responses, which were then processed and displayed through the frontend before being sent to the client’s database.
Handling documents and data: In this system, documents uploaded to the ontological system were processed, and the resulting data was saved in the company’s research database. This allowed for automatic analysis of documents and retrieval of important information.
Full-stack development: Our developer worked as a full-stack engineer, dealing with both front-end and back-end aspects and ensuring seamless integration of all system components.

Technologies

Programming languages

JavaScript, TypeScript, Java

Front-end

React, react-pdf, Redux, Redux-thunk, React-redux, Primereact, SASS, Lodash, Axios, FileSaver, GPT-Tokenizer

Back-end

Spring boot, Java with Lucene Libraries, Stardog

Data Science and analytics

Python (Pandas, Numpy, Plotly, Matplotlib), GCP (Google Big Query, Google Cloud Storage, Cloud Run), Looker, Data Studio, Apache Solr, custom tools for data processing and visualization

Process

Our approach to the development process was methodical and adhered to Agile principles, which ensured flexibility and continuous improvement.

At the beginning, we conducted thorough research to understand the client’s needs and existing systems to deliver a detailed ‘Vision and Scope’ document. Based on the initial findings, we proceeded to design and develop the necessary features for each stream. Our team held regular sprint meetings to confirm that our work aligned with client expectations. All features were implemented and subjected to rigorous testing for performance and accuracy, with the client providing continuous feedback.

For effective communication and project tracking, we utilized Microsoft tools and Monday.com, ensuring a transparent process and real-time updates.

Team required

Project Manager

React Developers

Java Developers

ML/Python Developer

Data Engineers

Results

In our collaborative effort with the client, spanning three key streams, we’ve made significant strides in advancing their scientific research capabilities. Here’s a snapshot of the actual results:

Streamlined search operations: Our team’s efforts in refining the search system led to a doubling of search speed, benefiting researchers in their quest for swift access to scientific data.
Precision in annotations: The introduction of the automated annotation system resulted in increased annotation accuracy, a critical factor for in-depth scientific studies.
Increased efficiency in data handling: By automating data visualization processes, we’ve effectively halved the time researchers spent on manual data handling, translating to more time for core research activities.
Enhanced data processing velocity: Data processing and visualization speeds were amplified threefold, marking a leap in handling complex datasets.
Optimized user experience: The modernized user interface of our systems has led to a notable uptick in user satisfaction, fostering better engagement within the scientific community.
Liberated research time: Automating routine tasks has led to a 60% reduction in manual data handling, liberating the researchers’ time, previously consumed by manual work.

Related cases

Contact us

Book a call or fill out the form below and we’ll get back to you once we’ve processed your request.

Name

Company

Phone

Message

Send us a voice message

Attach documents

Upload file

You can attach 1 file up to 2MB. Valid file formats: pdf, jpg, jpeg, png.

By clicking Send, you consent to Innowise processing your personal data per our Privacy Policy to provide you with relevant information. By submitting your phone number, you agree that we may contact you via voice calls, SMS, and messaging apps. Calling, message, and data rates may apply.

You can also send us your request
to contact@innowise.com

What happens next?

Once we’ve received and processed your request, we’ll get back to you to detail your project needs and sign an NDA to ensure confidentiality.

After examining your wants, needs, and expectations, our team will devise a project proposal with the scope of work, team size, time, and cost estimates.

We’ll arrange a meeting with you to discuss the offer and nail down the details.

Finally, we’ll sign a contract and start working on your project right away.