Building Agentic AI chatbots for workforce analytics

The rapid evolution of Generative AI (GenAI), particularly Large Language Models (LLMs), is transforming how organizations engage with data, automate processes, and make strategic decisions. In workforce analytics, understanding the interplay between job roles, evolving skill requirements, and organizational impact has become critical.

Traditional AI solutions often fall short in handling nuanced, multi-layered business questions due to limited flexibility and contextual understanding. To bridge this gap, we developed a custom agentic AI chatbot that combines autonomous AI agents with targeted workflows to deliver intelligent, human-like query responses.

This chatbot decomposes complex questions, identifies the most relevant data sources, and uses specialized agents to generate visualizations, extract insights, or summarize enterprise documents. Unlike rigid, rule-based bots, the agentic AI architecture supports real-time adaptability and scalability for multiple business use-case.

This whitepaper outlines the system’s architecture, methodology, and deep dive into the specific use case–enhancing workforce analytics and strategic decision-making. It also provides insights into how to build agentic AI systems that can transform business processes.

Agentic AI systems are designed to make decisions, take actions, and use tools autonomously, similar to a skilled assistant who understands objectives and determines the most efficient way to achieve them. These systems are becoming essential for organizations aiming to build AI agents that can handle complex, real-world tasks.

The concept of an "agent" varies in scope. It may refer to a fully autonomous AI agent capable of complex tasks or modular components following structured workflows. In both cases, the core distinction lies in how tasks are executed.

2.1. Agents

An agent is an autonomous AI system capable of achieving specific goals without constant human intervention. Typically, this involves an LLM augmented with retrieval mechanisms, tool integrations, and memory management.

Figure 2.1. Components of an Agent
(Source: Anthropic AI)

2.2 Agentic Workflows

Agentic AI workflows are structured patterns that guide how LLMs break down tasks, delegate sub-tasks, and execute actions. These workflows support:

• Sequential task processing
• Task classification and redirection
• Parallel execution
• Dynamic delegation
• Iterative refinement

Depending on the task’s complexity, workflows can operate autonomously or follow a structured pattern with human-in-the-loop options.

Figure 2.2. Components of an Agentic Workflow
(Source: Anthropic AI)

2.3. Comparing Agents and Agentic Workflows

Several frameworks exist to simplify the development of agentic systems, including AI agent builder tools like LangGraph (LangChain), Amazon Bedrock Agents, and CrewAI. While these platforms offer structured environments, they often introduce complexity and dependencies that may not align with enterprise-specific needs. For organizations looking to build AI agents with domain control, custom AI models offer greater flexibility.

3.1 Objective

This case study describes the development of an intelligent Q&A chatbot that leverages Agentic AI frameworks to deliver accurate context-aware responses. The system interacts seamlessly with structured and unstructured data, providing users with actionable insights for workforce analytics and business intelligence. The chatbot evaluates the impact of Generative AI on job roles, benchmarks evolving skills, and drives informed workforce transformation. This approach is part of a broader shift towards AI chatbot for business applications that move beyond simple automation to handle strategic and decision-making driven tasks.

Figure 3.1. High level solution workflow

3.2 Datasets used for training the models

3.3. Workflow of the chatbot

For this use case, we designed custom agentic workflows to maintain full control over execution, optimize performance, and simplify integration with business logic. This hands-on approach to building agentic AI systems ensures better adaptability, maintenance and dugging compared to general-purpose frameworks. Our modular design includes:

• Question decomposition
• Data routing
• Parallel execution
• Visualization
• Summarization

This modular structure enables organizations to build AI agents aligned with their specific analytics or reporting needs.

The AI agent chatbot accepts user queries and decomposes them into smaller sub-questions using a Query Breakdown Agent. Users can review, edit, and reorder these sub-queries for clarity and precision.

Once finalized, the system triggers a series of agentic AI workflows:

1. Query Refinement Agent enhances each sub-query by adding relevant context from prior conversations.
2. Database Routing Agent selects the appropriate data source between structured databases or unstructured documents.
3. Question Routing Agent categorizes the query into visualization, analytical, or business insight tasks.
4. Visualization Agent generates relevant charts or graphs as required.
5. SQL Agent converts data queries into SQL commands to retrieve accurate results.
6. Summarization Agent translates complex tabular outputs into concise, natural language summaries.
7. Business Agent identifies patterns and provides business insights based on historical chats
8. Retrieval Augmented Generation (RAG) Agents retrieve recise answers from internal documents using semantic search.
9. Data Structuring Agent organizes results into downloadable structured tables upon user request.

Figure 3.3. Architecture of the chatbot

3.4. Multimodal RAG pipeline

To process unstructured data from PDFs and PPTs, the AI agent chatbot uses a Multimodal RAG pipeline that:

• Converts documents into images for layout preservation
• Embeds both text and images into a vector database
• Retrieves relevant content via semantic search
• Uses GPT-4o for accurate interpretation of complex layouts like flowcharts

Figure 3.4. Multimodal RAG architecture

3.5 Additional features in the chatbot

• Manual question editing allows users to adjust and refine sub-queries for better alignment with intent.
• Title matching uses fuzzy matching and sentence embeddings to detect and correct job titles, improving accuracy when titles are missing from the dataset.
• Cache memory stores previous queries and answers in a vector database, speeding up responses for repeat or similar queries. This feature aligns with best practices for scalable AI agent builder implementations.

3.6 Illustrative user workflow over the chatbot

The chatbot is deployed via Streamlit to offer offering an interactive interface. Users can explore complex queries from start to finish, with visualizations and structured summaries provided on-demand. Such a setup allows enterprises to experience how to build agentic AI solutions that deliver real-time analytics.

Figure 3.6. Detailed journey of a user query through the chatbot workflow

4.1 Evaluation process for high performance

A benchmark of 100 questions was created across all datasets, each compared to a ground truth response. An LLM-based evaluator assigned similarity scores between 0 and 1, where:

• 1 = Exact match
• 0 = No similarity

Initial accuracy ranged between 70–75%. After optimization, accuracy improved to approximately 90%.

4.2. Key optimizations executed to improve accuracy

The following table outlines the core technologies used across the application’s language processing, data storage, and user interface layers.

This custom agentic AI-powered chatbot demonstrates a scalable, adaptable solution for workforce analytics and business intelligence for enterprises. By combining modular agents with tailored workflows and optimized data pipelines, the system delivers context-aware insights across a wide range of queries while maintaining control.

Its agentic AI architecture enables enterprises to evaluate GenAI’s impact on job roles, benchmark evolving skill sets, and streamline decision-making. The solution highlights a scalable approach to building agentic AI systems to support domain-specific, high-performance enterprise applications.