LLM Monitoring

What is LLM Monitoring?

LLM monitoring observes a deployed application's inputs, outputs, quality signals, latency, cost, failures, and user behavior over time; offline model evaluation complements this operational view.

Teams typically draw on several offline evaluation methods and true production monitoring methods to understand LLM behavior:

1. Perplexity — An offline evaluation measure that quantifies how well a language model predicts a sample of text. Lower perplexity values indicate better performance.

2. Human Evaluation — An offline evaluation method that assesses LLM outputs but can be subjective and prone to bias. Different human evaluators may have varying opinions, and the evaluation criteria may lack consistency.

3. Benchmarking — Models are scored on standardized benchmark datasets during offline evaluation, before or alongside deployment. This assesses a model's general capabilities under controlled conditions rather than how a deployed system behaves in production.

4. Usage and Engagement Metrics — A production monitoring metric that measures how often users engage with the LLM features, the quality of those interactions, and how likely they are to use it in the future.

5. Retention Metrics — A production monitoring metric that measures how sticky the feature is and whether the user gets retained into the LLM feature.

6. LLM-as-a-Judge — This method uses another LLM to score the outputs of the model being assessed, and can be applied both in offline evaluation and in production monitoring. This approach has been found to largely reflect human preferences for certain use cases.

7. System Monitoring — A production monitoring method that tracks the complete components of the system that you have control of, such as the prompt or prompt template and context. It assesses how well your inputs determine your outputs over time.

It's important to note that existing evaluation and monitoring methods often don't capture the diversity and creativity of LLM outputs. Metrics that only focus on accuracy and relevance overlook the importance of generating diverse and novel responses. Benchmark-based evaluation in particular tends to focus on specific datasets or tasks, which don't fully reflect the challenges of real-world applications.

To address these issues, researchers and practitioners are exploring various approaches and strategies, such as combining multiple evaluation and monitoring metrics for a more comprehensive assessment of LLM performance, creating diverse and representative reference data for evaluation, and grounding monitoring methods in real-world scenarios and tasks.

How does LLM Monitoring work?

LLM Monitoring, such as facilitated by Klu.ai, works by providing a comprehensive environment for tracking Large Language Models. It includes features for prompt engineering, semantic search, version control, testing, and performance monitoring. The platform also provides resources for handling the ethical and transparency issues associated with deploying LLMs.

• Comprehensive tracking: The platform provides an environment to monitor models on various tasks, analyze their ability to generalize, and assess their robustness against adversarial attacks.
• Bias and fairness monitoring: The platform provides features for monitoring the model's bias, fairness, and ethical considerations.
• Performance monitoring: The platform provides usage and system performance insights across features and teams, helping understand user preference, model performance, and label your data.
• Fine-tuning custom models: The platform allows you to curate your best data for fine-tuning custom models.
• Secure and portable data: Your data is secure and portable with Klu.ai.

What are the applications of LLM Monitoring?

LLM Monitoring can be used to track a wide range of Large Language Models. These include models for natural language processing, text generation, knowledge representation, multimodal learning, and personalization.

• Natural language processing: The monitoring process can track LLMs used to understand text, answer questions, summarize, translate and more.
• Text generation: The monitoring process can track LLMs used to generate coherent, human-like text for a variety of applications like creative writing, conversational AI, and content creation.
• Knowledge representation: The monitoring process can track LLMs used to store world knowledge learned from data and reason about facts and common sense concepts.
• Multimodal learning: The monitoring process can track LLMs used to understand and generate images, code, music, and more when trained on diverse data.
• Personalization: The monitoring process can track LLMs that are fine-tuned on niche data to provide customized services.

How is LLM Monitoring impacting AI?

LLM Monitoring is significantly impacting AI by simplifying the process of tracking, fine-tuning, and deploying Large Language Models. It is enabling rapid progress in the field by providing a comprehensive set of tools and methodologies that streamline the process of monitoring LLMs. However, as LLMs become more capable, it is important to balance innovation with ethics. The monitoring process provides resources for addressing issues around bias, misuse, and transparency. It represents a shift to more generalized AI learning versus task-specific engineering, which scales better but requires care and constraints.

• Rapid progress: The monitoring process is enabling rapid progress in AI by simplifying the process of tracking, fine-tuning, and deploying Large Language Models.
• Broad applications: The monitoring process is enabling the tracking of a wide range of applications that leverage the capabilities of LLMs.
• Responsible deployment: The monitoring process provides resources for addressing issues around bias, misuse, and transparency as LLMs become more capable.
• New paradigms: The monitoring process represents a shift to more generalized AI learning versus task-specific engineering, which scales better but requires care and constraints.

How do you monitor prompts?

Tracking the performance of prompts against Large Language Models (LLMs) is crucial because the quality of a prompt can significantly influence the output. This work generally falls into two categories: offline evaluation of the underlying model, and ongoing monitoring of the system built on top of it.

Offline evaluation quantifies a foundational model's general capabilities across tasks using standardized benchmarks. Common examples include HellaSwag (testing commonsense sentence completion), TruthfulQA (measuring the truthfulness of model responses), and MMLU (Massive Multitask Language Understanding, which tests knowledge and reasoning across dozens of academic subjects). These benchmarks inform model selection but say little about how a specific application will behave once deployed.

LLM System Monitoring, by contrast, tracks the components you control in production, such as the prompt or prompt template and context, to assess how well your inputs determine your outputs. For instance, monitoring can track your chatbot responses for usefulness or politeness, and reveal how that performance changes over time in production.

One approach to monitoring an LLM-based system is to use another LLM as a judge. First, you establish a scoring rubric for your monitoring metric by putting together a dedicated LLM-based evaluator whose only task is to label data as effectively as a human labeled your "golden dataset." You then validate that evaluator against the human-labeled examples. Once validated, you run this metric against the live outputs of your LLM application to track quality over time.

There are tools available to assist with this work. For instance, promptfoo.dev is an open-source library for testing LLM prompts, letting teams evaluate outputs against a representative sample of user inputs to reduce subjectivity when tuning prompts. Braintrust, meanwhile, provides a web UI experiment view for digging into what test cases improved or got worse, and can also be used to track production traces over time.

However, it's important to note that the quality of prompts generated by LLMs can be highly unpredictable, which in turn leads to a significant increase in the performance variance of LLMs. Therefore, it is critical to find ways to control the quality of prompts generated by LLMs to ensure the reliability of their outputs.