Chatbot for data analytic development: 67% improvement in latency

Using the existing large language models (LLM), we have developed an analytical platform similar to ChatGPT that can analyze the company’s internal data and generate responses to questions based on that information.

Customer

Industry

eCommerce

Region

Client since

2023

Our client, an emerging startup, had a vision for a product designed for sale to their major clients in the retail sector.

Detailed information about the client cannot be disclosed under the terms of the NDA.

Challenge

The need for an analytical platform that offers instant access to internal documents and delivers data-driven insights

Primary pain point: Internal documents, including employee records, marketing data, and sales information lack accessibility. With thousands of files in formats like PDF, CSV, Parquet, TXT, and DOCX, locating and analyzing specific information is time-consuming and error-prone.

Secondary challenges: As a company grows, the volume of documents and information increases, further intensifying the challenges of data accessibility and analysis. Without a proper document analytics system, these issues become increasingly evident over time.

Recognizing these challenges, our client contacted Innowise to get a chatbot for data analytics, with the goal of offering it to their major clients.

Solution

The chatbot data analytics software tailored to handle internal data

Innowise has developed the chatbot data analytics software using the existing large language models. The chat system functions similarly to available bots but is tailored to handle internal data. The development involved building a complete system for integrating LLM with the relational and document databases, including internal client data storage solutions and providing smooth interaction between the platform and users.

Information extraction

The document analysis and processing capabilities enable extracting relevant information from internal company documents such as policies, instructions, guides, operational data, and technical specifications. This allows the user to quickly obtain accurate and up-to-date answers to their questions without having to manually search and analyze data.

RAG AI-enhanced performance

We boosted the chatbot’s performance by conducting daily manual tests and refining the chatbot using retrieval-augmented generation (RAG) AI. This approach combines information retrieval with natural language generation, making responses more informative and relevant. We also introduced a feedback system to analyze user preferences, which further improved RAG and increased user trust in the chatbot.

Fast answer time

By implementing caching, query optimization, and parallel processing, we significantly improved the speed and efficiency of user interactions with the chatbot. Users can receive responses more swiftly, thanks to the frequently requested information stored in the cache. Additionally, we use parallel processing to distribute the workload, enabling the system to handle multiple requests at once. This makes the chatbot more responsive, even during peak times.

Data extraction from Data Mart

We have created a data repository for processing structured relational data. This chatbot feature includes requests to retrieve information from the Data Mart. By providing direct access to the Data Mart through the chatbot, users can effortlessly obtain the information they need without consulting other sources. This simplified access means that decision-makers have up-to-date insights at their fingertips, facilitating agile responses to market changes and strategic opportunities.

AI-powered document retrieval system

We refined document management and retrieval by integrating Azure Data Lake Gen 2 for document ingestion, segmenting documents into chunks, and utilizing Azure OpenAI to generate embeddings. These embeddings are stored in Azure AI Search for efficient analysis and retrieval. User queries are processed through Azure OpenAI Search, comparing query embeddings with stored document embeddings to deliver relevant responses instantly.

Diverse information presentation options

The information is presented in the form of charts created with Plotly, tables styled with Material UI, and straightforward text content. This mix makes the content more engaging and helps communicate the details in a way that’s easy to understand and act on.

Voice query logic with text-to-speech translation

Our team integrated voice query functionality alongside text-based interactions in the chatbot for data analytics. Users can now effortlessly interact with the bot via voice commands, with the added capability of translating spoken text into written form.

Technologies

Frontend

Axios, Material UI, Plotly, React, React Context, react-markdown, TypeScript

Backend

Azure AI Search, Azure App Service, Azure Data Factory, Azure Data Lake Gen2, Azure Databricks, Azure Functions, Azure OpenAI, Bicep, Cosmos DB, Spark

Libraries

Axios, Material UI, Matplotlib, NumPy, Pandas, Plotly, PySpark, React Context, react-markdown, Streamlit, TypeScript

Process

Firstly, we conducted a detailed analysis of the business requirements and mapped out a comprehensive plan for the software based on that.

Next, we created a visual representation of the chatbot, which included wireframes, prototypes, and mockups, based on the information we gathered. The design phase focused on creating a user-friendly interface that would deliver customers easy navigation and access to the chat bot’s features.

The development covered creating a full-scale system to integrate LLM with both relational and document databases, including internal client data storage solutions. We provided smooth interaction between the platform and users by employing natural language processing (NLP) to immediately extract key information and integrating retrieval-augmented generation (RAG) AI for contextually relevant responses.

We optimized performance through caching, improved query efficiency, and parallel processing, while providing direct access to structured data from the Data Mart.

Finally, we incorporated voice query and text-to-speech features to elevate accessibility and meet diverse user needs.

Team

Front-End Developer

Back-End Developer

Data Scientist

Data Engineer

Data Engineer / DevOps

Results

A 67% improvement in latency for queries and data processing

Our team has developed a tailored analytics platform, which our clients then personally evaluated through hands-on testing. This has resulted in several noticeable outcomes:

Operational agility and faster, informed decision-making: Deploying a distributed storage and computing system with Azure Databricks, ADLS Gen2, and Spark capabilities has boosted the solution with faster data processing and scalability for handling substantial datasets.
A 67% improvement in latency for queries and data processing: Maintaining low latency means faster response times for queries and data processing, leading to improved reliability and platform performance.
Team increased productivity: Teams have got a major efficiency boost with rapid file access and management. With better collaboration and less admin work, team members can focus more on their core tasks and push projects ahead with greater speed.

This advanced chatbot platform delivers exceptional performance and elevates user experience by swiftly extracting key information from internal documents using NLP. Integrated with RAG AI for contextually relevant answers, it optimizes response time through caching, query efficiency, and parallel processing while providing direct access to structured data from the Data Mart. Voice query and text-to-speech capabilities elevate accessibility, catering to diverse user needs.

Our client started offering the solution to their customers, and it quickly gained traction with impressive sales figures. The solution’s effectiveness and ease of use have led to high satisfaction rates among their clients, further solidifying its success in the market.

Project duration

October 2023 - February 2024

67%

faster query and data processing

34%

increase in teams’ performance

Related cases

Contact us!

Book a call or fill out the form below and we’ll get back to you once we’ve processed your request.

What happens next?

Having received and processed your request, we will get back to you shortly to detail your project needs and sign an NDA to ensure the confidentiality of information.

After examining requirements, our analysts and developers devise a project proposal with the scope of works, team size, time, and cost estimates.

We arrange a meeting with you to discuss the offer and come to an agreement.

We sign a contract and start working on your project as quickly as possible.