Enhancing Log Analytics with Generative AI
This thesis explores the integration of Generative AI (GenAI) models, specifically Large Language Models (LLMs), into the field of log analytics to address the growing challenges posed by the increasing volume and complexity of log data in modern IT systems. Traditional log analysis methods, which often rely on manual inspection or rule-based approaches, struggle to keep up with the scale and variability of logs in large-scale environments. To address these limitations, this work introduces the ModuGPT framework, a modular system designed to simplify the development of LLM-powered applications for log analytics. The framework introduces Prompt Elements, a structured catalog of reusable components that streamline prompt engineering and improve the consistency and effectiveness of LLM interactions. Additionally, three specialized tools are developed within the framework: Insight-to-Text, which generates natural language explanations and recommendations for log insights; Text-to-ES, which provides a natural language interface for querying Elasticsearch; and SmartSearch, which enhances traditional search functionality by combining semantic and keyword-based methods.