mirror of
https://github.com/amithkoujalgi/ollama4j.git
synced 2025-10-14 01:18:58 +02:00
Add metrics documentation for Ollama4j library
- Introduced a new `metrics.md` file detailing the metrics available in the Ollama4j library, including total requests, response time, error rate, and active sessions. - Provided examples of accessing metrics in Java and integrating with monitoring tools like Prometheus and Grafana. - Included a warning note indicating that the documentation is a work in progress.
This commit is contained in:
parent
fd18a247d7
commit
453112d09f
71
docs/docs/metrics.md
Normal file
71
docs/docs/metrics.md
Normal file
@ -0,0 +1,71 @@
|
|||||||
|
---
|
||||||
|
sidebar_position: 5
|
||||||
|
|
||||||
|
title: Metrics
|
||||||
|
---
|
||||||
|
|
||||||
|
import CodeEmbed from '@site/src/components/CodeEmbed';
|
||||||
|
|
||||||
|
# Metrics
|
||||||
|
|
||||||
|
:::warning[Note]
|
||||||
|
This is work in progress
|
||||||
|
:::
|
||||||
|
|
||||||
|
Monitoring and understanding the performance of your models and requests is crucial for optimizing and maintaining your
|
||||||
|
applications. The Ollama4j library provides built-in support for collecting and exposing various metrics, such as
|
||||||
|
request counts, response times, and error rates. These metrics can help you:
|
||||||
|
|
||||||
|
- Track usage patterns and identify bottlenecks
|
||||||
|
- Monitor the health and reliability of your services
|
||||||
|
- Set up alerts for abnormal behavior
|
||||||
|
- Gain insights for scaling and optimization
|
||||||
|
|
||||||
|
## Available Metrics
|
||||||
|
|
||||||
|
Ollama4j exposes several key metrics, including:
|
||||||
|
|
||||||
|
- **Total Requests**: The number of requests processed by the model.
|
||||||
|
- **Response Time**: The time taken to generate a response for each request.
|
||||||
|
- **Error Rate**: The percentage of requests that resulted in errors.
|
||||||
|
- **Active Sessions**: The number of concurrent sessions or users.
|
||||||
|
|
||||||
|
These metrics can be accessed programmatically or integrated with monitoring tools such as Prometheus or Grafana for
|
||||||
|
visualization and alerting.
|
||||||
|
|
||||||
|
## Example Metrics Dashboard
|
||||||
|
|
||||||
|
Below is an example of a metrics dashboard visualizing some of these key statistics:
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
## Example: Accessing Metrics in Java
|
||||||
|
|
||||||
|
You can easily access and display metrics in your Java application using Ollama4j.
|
||||||
|
|
||||||
|
Make sure you have added the `simpleclient_httpserver` dependency in your app for the app to be able to expose the
|
||||||
|
metrics via `/metrics` endpoint:
|
||||||
|
|
||||||
|
```xml
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>io.prometheus</groupId>
|
||||||
|
<artifactId>simpleclient_httpserver</artifactId>
|
||||||
|
<version>0.16.0</version>
|
||||||
|
</dependency>
|
||||||
|
```
|
||||||
|
|
||||||
|
Here is a sample code snippet demonstrating how to retrieve and print metrics:
|
||||||
|
|
||||||
|
<CodeEmbed src="https://raw.githubusercontent.com/ollama4j/ollama4j-examples/refs/heads/main/src/main/java/io/github/ollama4j/examples/MetricsExample.java" />
|
||||||
|
|
||||||
|
This will start a simple HTTP server with `/metrics` endpoint enabled. Metrics will now available
|
||||||
|
at: http://localhost:8080/metrics
|
||||||
|
|
||||||
|
## Integrating with Monitoring Tools
|
||||||
|
|
||||||
|
To integrate Ollama4j metrics with external monitoring systems, you can export the metrics endpoint and configure your
|
||||||
|
monitoring tool to scrape or collect the data. Refer to the [integration guide](../integration/monitoring.md) for
|
||||||
|
detailed instructions.
|
||||||
|
|
||||||
|
For more information on customizing and extending metrics, see the [API documentation](../api/metrics.md).
|
Loading…
x
Reference in New Issue
Block a user