Compare commits

..

28 Commits

Author SHA1 Message Date
Amith Koujalgi
3fc7e9423c Updated docs 2024-10-31 17:57:02 +05:30
Amith Koujalgi
405a08b330 Updated docs 2024-10-31 16:25:05 +05:30
Amith Koujalgi
921f745435 Custom roles support
Adds support for custom roles using `OllamaChatMessageRole`
2024-10-31 16:15:21 +05:30
Amith Koujalgi
bedfec6bf9 Update generate-embeddings.md 2024-10-30 11:07:40 +05:30
Amith Koujalgi
afa09e87a5 Update OllamaAPI.java 2024-10-30 11:02:37 +05:30
Amith Koujalgi
baf2320ea6 Updated javadoc 2024-10-30 11:01:23 +05:30
Amith Koujalgi
948a7444fb Update README.md 2024-10-30 00:54:33 +05:30
Amith Koujalgi
ec0eb8b469 Update README.md 2024-10-30 00:44:08 +05:30
Amith Koujalgi
8f33de7e59 Update README.md 2024-10-30 00:43:09 +05:30
Amith Koujalgi
8c59e6511b Update README.md 2024-10-30 00:41:55 +05:30
Amith Koujalgi
b93fc7623a Updated javadoc 2024-10-30 00:28:53 +05:30
Amith Koujalgi
bd1a57c7e0 Added support for new embed API /api/embed 2024-10-30 00:03:49 +05:30
Amith Koujalgi
7fabead249 Merge pull request #73 from daguava/tool-role
Add 'tool' role
2024-10-29 22:05:43 +05:30
Mitchell Lutzke
268a973d5e Add tool role 2024-10-27 17:06:25 -07:00
Amith Koujalgi
d949a3cb69 Merge pull request #72 from daguava/minp-custom-options
Add minp option and ability to set custom options
2024-10-27 20:05:24 +05:30
Mitchell Lutzke
e2443ed68a Add throws to the docs 2024-10-26 22:08:03 -07:00
Mitchell Lutzke
37193b1f5b slight cleanup 2024-10-26 21:36:43 -07:00
Mitchell Lutzke
e33071ae38 Add minp option and ability to set custom options 2024-10-26 21:22:46 -07:00
Amith Koujalgi
fffc8dc526 Update README.md 2024-10-16 00:55:01 +05:30
Amith Koujalgi
def950cc9c Update README.md 2024-10-15 14:27:07 +05:30
Amith Koujalgi
f4db7ca326 Update README.md 2024-10-15 11:51:20 +05:30
Amith Koujalgi
18760250ea Update README.md 2024-10-11 21:21:20 +05:30
Amith Koujalgi
233597efd1 Update README.md 2024-10-01 23:38:49 +05:30
Amith Koujalgi
cec9f29eb7 Update README.md 2024-09-15 09:43:37 +05:30
Amith Koujalgi
20cb92a418 Update README.md 2024-09-15 08:48:31 +05:30
Amith Koujalgi
b0dc38954b Update README.md 2024-09-15 08:47:22 +05:30
Amith Koujalgi
1479d0a494 Update README.md 2024-09-08 16:28:46 +05:30
Amith Koujalgi
b328daee43 Update README.md 2024-09-08 16:28:23 +05:30
14 changed files with 625 additions and 156 deletions

View File

@@ -4,12 +4,11 @@
<img src='https://raw.githubusercontent.com/ollama4j/ollama4j/65a9d526150da8fcd98e2af6a164f055572bf722/ollama4j.jpeg' width='100' alt="ollama4j-icon"> <img src='https://raw.githubusercontent.com/ollama4j/ollama4j/65a9d526150da8fcd98e2af6a164f055572bf722/ollama4j.jpeg' width='100' alt="ollama4j-icon">
</p> </p>
<div align="center">
A Java library (wrapper/binding) for [Ollama](https://ollama.ai/) server. A Java library (wrapper/binding) for Ollama server.
Find more details on the [website](https://ollama4j.github.io/ollama4j/). Find more details on the [website](https://ollama4j.github.io/ollama4j/).
<div align="center">
![GitHub stars](https://img.shields.io/github/stars/ollama4j/ollama4j) ![GitHub stars](https://img.shields.io/github/stars/ollama4j/ollama4j)
![GitHub forks](https://img.shields.io/github/forks/ollama4j/ollama4j) ![GitHub forks](https://img.shields.io/github/forks/ollama4j/ollama4j)
@@ -268,7 +267,10 @@ make integration-tests
Newer artifacts are published via GitHub Actions CI workflow when a new release is created from `main` branch. Newer artifacts are published via GitHub Actions CI workflow when a new release is created from `main` branch.
#### Who's using Ollama4j? ## ⭐ Give us a Star!
If you like or are using this project to build your own, please give us a star. It's a free way to show your support.
## Who's using Ollama4j?
- `Datafaker`: a library to generate fake data - `Datafaker`: a library to generate fake data
- https://github.com/datafaker-net/datafaker-experimental/tree/main/ollama-api - https://github.com/datafaker-net/datafaker-experimental/tree/main/ollama-api
@@ -277,18 +279,26 @@ Newer artifacts are published via GitHub Actions CI workflow when a new release
- `ollama-translator`: Minecraft 1.20.6 spigot plugin allows to easily break language barriers by using ollama on the - `ollama-translator`: Minecraft 1.20.6 spigot plugin allows to easily break language barriers by using ollama on the
server to translate all messages into a specfic target language. server to translate all messages into a specfic target language.
- https://github.com/liebki/ollama-translator - https://github.com/liebki/ollama-translator
- `AI Player`: A minecraft mod which aims to add a "second player" into the game which will actually be intelligent.
- https://github.com/shasankp000/AI-Player
- https://www.reddit.com/r/fabricmc/comments/1e65x5s/comment/ldr2vcf/ - https://www.reddit.com/r/fabricmc/comments/1e65x5s/comment/ldr2vcf/
- `Ollama4j Web UI`: A web UI for Ollama written in Java using Spring Boot and Vaadin framework and - `Ollama4j Web UI`: A web UI for Ollama written in Java using Spring Boot and Vaadin framework and
Ollama4j. Ollama4j.
- https://github.com/ollama4j/ollama4j-web-ui - https://github.com/ollama4j/ollama4j-web-ui
- `JnsCLI`: A command-line tool for Jenkins that manages jobs, builds, and configurations directly from the terminal while offering AI-powered error analysis for quick troubleshooting. - `JnsCLI`: A command-line tool for Jenkins that manages jobs, builds, and configurations directly from the terminal while offering AI-powered error analysis for quick troubleshooting.
- https://github.com/mirum8/jnscli - https://github.com/mirum8/jnscli
- `Katie Backend`: An Open Source AI-based question-answering platform that helps companies and organizations make their private domain knowledge accessible and useful to their employees and customers.
- https://github.com/wyona/katie-backend
- `TeleLlama3 Bot`: A Question-Answering Telegram Bot.
- https://git.hiast.edu.sy/mohamadbashar.disoki/telellama3-bot
- `moqui-wechat`: A wechat plugin
- https://github.com/heguangyong/moqui-wechat
#### Traction ## Traction
[![Star History Chart](https://api.star-history.com/svg?repos=ollama4j/ollama4j&type=Date)](https://star-history.com/#ollama4j/ollama4j&Date) [![Star History Chart](https://api.star-history.com/svg?repos=ollama4j/ollama4j&type=Date)](https://star-history.com/#ollama4j/ollama4j&Date)
### Get Involved ## Get Involved
<div align="center"> <div align="center">
@@ -316,6 +326,22 @@ Contributions are most welcome! Whether it's reporting a bug, proposing an enhan
with code - any sort with code - any sort
of contribution is much appreciated. of contribution is much appreciated.
## 🏷️ License and Citation
The code is available under [MIT License](./LICENSE).
If you find this project helpful in your research, please cite this work at
```
@misc{ollama4j2024,
author = {Amith Koujalgi},
title = {Ollama4j: A Java Library (Wrapper/Binding) for Ollama Server},
year = {2024},
month = {January},
url = {https://github.com/ollama4j/ollama4j}
}
```
### References ### References
- [Ollama REST APIs](https://github.com/jmorganca/ollama/blob/main/docs/api.md) - [Ollama REST APIs](https://github.com/jmorganca/ollama/blob/main/docs/api.md)

View File

@@ -0,0 +1,65 @@
---
sidebar_position: 8
---
# Custom Roles
Allows to manage custom roles (apart from the base roles) for chat interactions with the models.
_Particularly helpful when you would need to use different roles that the newer models support other than the base
roles._
_Base roles are `SYSTEM`, `USER`, `ASSISTANT`, `TOOL`._
### Usage
#### Add new role
```java
import io.github.ollama4j.OllamaAPI;
import io.github.ollama4j.models.chat.OllamaChatMessageRole;
public class Main {
public static void main(String[] args) {
String host = "http://localhost:11434/";
OllamaAPI ollamaAPI = new OllamaAPI(host);
OllamaChatMessageRole customRole = ollamaAPI.addCustomRole("custom-role");
}
}
```
#### List roles
```java
import io.github.ollama4j.OllamaAPI;
import io.github.ollama4j.models.chat.OllamaChatMessageRole;
public class Main {
public static void main(String[] args) {
String host = "http://localhost:11434/";
OllamaAPI ollamaAPI = new OllamaAPI(host);
List<OllamaChatMessageRole> roles = ollamaAPI.listRoles();
}
}
```
#### Get role
```java
import io.github.ollama4j.OllamaAPI;
import io.github.ollama4j.models.chat.OllamaChatMessageRole;
public class Main {
public static void main(String[] args) {
String host = "http://localhost:11434/";
OllamaAPI ollamaAPI = new OllamaAPI(host);
List<OllamaChatMessageRole> roles = ollamaAPI.getRole("custom-role");
}
}
```

View File

@@ -8,12 +8,85 @@ Generate embeddings from a model.
Parameters: Parameters:
- `model`: name of model to generate embeddings from
- `input`: text/s to generate embeddings for
```java
import io.github.ollama4j.OllamaAPI;
import io.github.ollama4j.types.OllamaModelType;
import io.github.ollama4j.models.embeddings.OllamaEmbedRequestModel;
import io.github.ollama4j.models.embeddings.OllamaEmbedResponseModel;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
public class Main {
public static void main(String[] args) {
String host = "http://localhost:11434/";
OllamaAPI ollamaAPI = new OllamaAPI(host);
OllamaEmbedResponseModel embeddings = ollamaAPI.embed("all-minilm", Arrays.asList("Why is the sky blue?", "Why is the grass green?"));
System.out.println(embeddings);
}
}
```
Or, using the `OllamaEmbedRequestModel`:
```java
import io.github.ollama4j.OllamaAPI;
import io.github.ollama4j.types.OllamaModelType;
import io.github.ollama4j.models.embeddings.OllamaEmbedRequestModel;
import io.github.ollama4j.models.embeddings.OllamaEmbedResponseModel;import java.util.Arrays;
import java.util.Collections;
import java.util.List;
public class Main {
public static void main(String[] args) {
String host = "http://localhost:11434/";
OllamaAPI ollamaAPI = new OllamaAPI(host);
OllamaEmbedResponseModel embeddings = ollamaAPI.embed(new OllamaEmbedRequestModel("all-minilm", Arrays.asList("Why is the sky blue?", "Why is the grass green?")));
System.out.println(embeddings);
}
}
```
You will get a response similar to:
```json
{
"model": "all-minilm",
"embeddings": [[-0.034674067, 0.030984823, 0.0067988685]],
"total_duration": 14173700,
"load_duration": 1198800,
"prompt_eval_count": 2
}
````
:::note
This is a deprecated API
:::
Parameters:
- `model`: name of model to generate embeddings from - `model`: name of model to generate embeddings from
- `prompt`: text to generate embeddings for - `prompt`: text to generate embeddings for
```java ```java
import io.github.ollama4j.OllamaAPI; import io.github.ollama4j.OllamaAPI;
import io.github.ollama4j.types.OllamaModelType; import io.github.ollama4j.types.OllamaModelType;
import java.util.List; import java.util.List;
public class Main { public class Main {
@@ -40,11 +113,6 @@ You will get a response similar to:
0.009260174818336964, 0.009260174818336964,
0.23178744316101074, 0.23178744316101074,
-0.2916173040866852, -0.2916173040866852,
-0.8924556970596313, -0.8924556970596313
0.8785552978515625,
-0.34576427936553955,
0.5742510557174683,
-0.04222835972905159,
-0.137906014919281
] ]
``` ```

View File

@@ -63,6 +63,10 @@
<groupId>org.apache.maven.plugins</groupId> <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId> <artifactId>maven-javadoc-plugin</artifactId>
<version>3.5.0</version> <version>3.5.0</version>
<configuration>
<!-- to disable the "missing" warnings. Remove the doclint to enable warnings-->
<doclint>all,-missing</doclint>
</configuration>
<executions> <executions>
<execution> <execution>
<id>attach-javadocs</id> <id>attach-javadocs</id>

View File

@@ -1,14 +1,14 @@
package io.github.ollama4j; package io.github.ollama4j;
import io.github.ollama4j.exceptions.OllamaBaseException; import io.github.ollama4j.exceptions.OllamaBaseException;
import io.github.ollama4j.exceptions.RoleNotFoundException;
import io.github.ollama4j.exceptions.ToolInvocationException; import io.github.ollama4j.exceptions.ToolInvocationException;
import io.github.ollama4j.exceptions.ToolNotFoundException; import io.github.ollama4j.exceptions.ToolNotFoundException;
import io.github.ollama4j.models.chat.OllamaChatMessage; import io.github.ollama4j.models.chat.*;
import io.github.ollama4j.models.chat.OllamaChatRequest; import io.github.ollama4j.models.embeddings.OllamaEmbedRequestModel;
import io.github.ollama4j.models.chat.OllamaChatRequestBuilder;
import io.github.ollama4j.models.chat.OllamaChatResult;
import io.github.ollama4j.models.embeddings.OllamaEmbeddingResponseModel; import io.github.ollama4j.models.embeddings.OllamaEmbeddingResponseModel;
import io.github.ollama4j.models.embeddings.OllamaEmbeddingsRequestModel; import io.github.ollama4j.models.embeddings.OllamaEmbeddingsRequestModel;
import io.github.ollama4j.models.embeddings.OllamaEmbedResponseModel;
import io.github.ollama4j.models.generate.OllamaGenerateRequest; import io.github.ollama4j.models.generate.OllamaGenerateRequest;
import io.github.ollama4j.models.generate.OllamaStreamHandler; import io.github.ollama4j.models.generate.OllamaStreamHandler;
import io.github.ollama4j.models.ps.ModelsProcessResponse; import io.github.ollama4j.models.ps.ModelsProcessResponse;
@@ -36,7 +36,7 @@ import java.util.*;
/** /**
* The base Ollama API class. * The base Ollama API class.
*/ */
@SuppressWarnings("DuplicatedCode") @SuppressWarnings({"DuplicatedCode", "resource"})
public class OllamaAPI { public class OllamaAPI {
private static final Logger logger = LoggerFactory.getLogger(OllamaAPI.class); private static final Logger logger = LoggerFactory.getLogger(OllamaAPI.class);
@@ -97,12 +97,7 @@ public class OllamaAPI {
HttpClient httpClient = HttpClient.newHttpClient(); HttpClient httpClient = HttpClient.newHttpClient();
HttpRequest httpRequest = null; HttpRequest httpRequest = null;
try { try {
httpRequest = httpRequest = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-type", "application/json").GET().build();
getRequestBuilderDefault(new URI(url))
.header("Accept", "application/json")
.header("Content-type", "application/json")
.GET()
.build();
} catch (URISyntaxException e) { } catch (URISyntaxException e) {
throw new RuntimeException(e); throw new RuntimeException(e);
} }
@@ -121,19 +116,17 @@ public class OllamaAPI {
/** /**
* Provides a list of running models and details about each model currently loaded into memory. * Provides a list of running models and details about each model currently loaded into memory.
* *
* @return ModelsProcessResponse * @return ModelsProcessResponse containing details about the running models
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws OllamaBaseException if the response indicates an error status
*/ */
public ModelsProcessResponse ps() throws IOException, InterruptedException, OllamaBaseException { public ModelsProcessResponse ps() throws IOException, InterruptedException, OllamaBaseException {
String url = this.host + "/api/ps"; String url = this.host + "/api/ps";
HttpClient httpClient = HttpClient.newHttpClient(); HttpClient httpClient = HttpClient.newHttpClient();
HttpRequest httpRequest = null; HttpRequest httpRequest = null;
try { try {
httpRequest = httpRequest = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-type", "application/json").GET().build();
getRequestBuilderDefault(new URI(url))
.header("Accept", "application/json")
.header("Content-type", "application/json")
.GET()
.build();
} catch (URISyntaxException e) { } catch (URISyntaxException e) {
throw new RuntimeException(e); throw new RuntimeException(e);
} }
@@ -142,36 +135,30 @@ public class OllamaAPI {
int statusCode = response.statusCode(); int statusCode = response.statusCode();
String responseString = response.body(); String responseString = response.body();
if (statusCode == 200) { if (statusCode == 200) {
return Utils.getObjectMapper() return Utils.getObjectMapper().readValue(responseString, ModelsProcessResponse.class);
.readValue(responseString, ModelsProcessResponse.class);
} else { } else {
throw new OllamaBaseException(statusCode + " - " + responseString); throw new OllamaBaseException(statusCode + " - " + responseString);
} }
} }
/** /**
* List available models from Ollama server. * Lists available models from the Ollama server.
* *
* @return the list * @return a list of models available on the server
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws URISyntaxException if the URI for the request is malformed
*/ */
public List<Model> listModels() public List<Model> listModels() throws OllamaBaseException, IOException, InterruptedException, URISyntaxException {
throws OllamaBaseException, IOException, InterruptedException, URISyntaxException {
String url = this.host + "/api/tags"; String url = this.host + "/api/tags";
HttpClient httpClient = HttpClient.newHttpClient(); HttpClient httpClient = HttpClient.newHttpClient();
HttpRequest httpRequest = HttpRequest httpRequest = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-type", "application/json").GET().build();
getRequestBuilderDefault(new URI(url)) HttpResponse<String> response = httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString());
.header("Accept", "application/json")
.header("Content-type", "application/json")
.GET()
.build();
HttpResponse<String> response =
httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString());
int statusCode = response.statusCode(); int statusCode = response.statusCode();
String responseString = response.body(); String responseString = response.body();
if (statusCode == 200) { if (statusCode == 200) {
return Utils.getObjectMapper() return Utils.getObjectMapper().readValue(responseString, ListModelsResponse.class).getModels();
.readValue(responseString, ListModelsResponse.class)
.getModels();
} else { } else {
throw new OllamaBaseException(statusCode + " - " + responseString); throw new OllamaBaseException(statusCode + " - " + responseString);
} }
@@ -182,29 +169,24 @@ public class OllamaAPI {
* href="https://ollama.ai/library">available models</a>. * href="https://ollama.ai/library">available models</a>.
* *
* @param modelName the name of the model * @param modelName the name of the model
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws URISyntaxException if the URI for the request is malformed
*/ */
public void pullModel(String modelName) public void pullModel(String modelName) throws OllamaBaseException, IOException, URISyntaxException, InterruptedException {
throws OllamaBaseException, IOException, URISyntaxException, InterruptedException {
String url = this.host + "/api/pull"; String url = this.host + "/api/pull";
String jsonData = new ModelRequest(modelName).toString(); String jsonData = new ModelRequest(modelName).toString();
HttpRequest request = HttpRequest request = getRequestBuilderDefault(new URI(url)).POST(HttpRequest.BodyPublishers.ofString(jsonData)).header("Accept", "application/json").header("Content-type", "application/json").build();
getRequestBuilderDefault(new URI(url))
.POST(HttpRequest.BodyPublishers.ofString(jsonData))
.header("Accept", "application/json")
.header("Content-type", "application/json")
.build();
HttpClient client = HttpClient.newHttpClient(); HttpClient client = HttpClient.newHttpClient();
HttpResponse<InputStream> response = HttpResponse<InputStream> response = client.send(request, HttpResponse.BodyHandlers.ofInputStream());
client.send(request, HttpResponse.BodyHandlers.ofInputStream());
int statusCode = response.statusCode(); int statusCode = response.statusCode();
InputStream responseBodyStream = response.body(); InputStream responseBodyStream = response.body();
String responseString = ""; String responseString = "";
try (BufferedReader reader = try (BufferedReader reader = new BufferedReader(new InputStreamReader(responseBodyStream, StandardCharsets.UTF_8))) {
new BufferedReader(new InputStreamReader(responseBodyStream, StandardCharsets.UTF_8))) {
String line; String line;
while ((line = reader.readLine()) != null) { while ((line = reader.readLine()) != null) {
ModelPullResponse modelPullResponse = ModelPullResponse modelPullResponse = Utils.getObjectMapper().readValue(line, ModelPullResponse.class);
Utils.getObjectMapper().readValue(line, ModelPullResponse.class);
if (verbose) { if (verbose) {
logger.info(modelPullResponse.getStatus()); logger.info(modelPullResponse.getStatus());
} }
@@ -220,17 +202,15 @@ public class OllamaAPI {
* *
* @param modelName the model * @param modelName the model
* @return the model details * @return the model details
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws URISyntaxException if the URI for the request is malformed
*/ */
public ModelDetail getModelDetails(String modelName) public ModelDetail getModelDetails(String modelName) throws IOException, OllamaBaseException, InterruptedException, URISyntaxException {
throws IOException, OllamaBaseException, InterruptedException, URISyntaxException {
String url = this.host + "/api/show"; String url = this.host + "/api/show";
String jsonData = new ModelRequest(modelName).toString(); String jsonData = new ModelRequest(modelName).toString();
HttpRequest request = HttpRequest request = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-type", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData)).build();
getRequestBuilderDefault(new URI(url))
.header("Accept", "application/json")
.header("Content-type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(jsonData))
.build();
HttpClient client = HttpClient.newHttpClient(); HttpClient client = HttpClient.newHttpClient();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString()); HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
int statusCode = response.statusCode(); int statusCode = response.statusCode();
@@ -248,17 +228,15 @@ public class OllamaAPI {
* *
* @param modelName the name of the custom model to be created. * @param modelName the name of the custom model to be created.
* @param modelFilePath the path to model file that exists on the Ollama server. * @param modelFilePath the path to model file that exists on the Ollama server.
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws URISyntaxException if the URI for the request is malformed
*/ */
public void createModelWithFilePath(String modelName, String modelFilePath) public void createModelWithFilePath(String modelName, String modelFilePath) throws IOException, InterruptedException, OllamaBaseException, URISyntaxException {
throws IOException, InterruptedException, OllamaBaseException, URISyntaxException {
String url = this.host + "/api/create"; String url = this.host + "/api/create";
String jsonData = new CustomModelFilePathRequest(modelName, modelFilePath).toString(); String jsonData = new CustomModelFilePathRequest(modelName, modelFilePath).toString();
HttpRequest request = HttpRequest request = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)).build();
getRequestBuilderDefault(new URI(url))
.header("Accept", "application/json")
.header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8))
.build();
HttpClient client = HttpClient.newHttpClient(); HttpClient client = HttpClient.newHttpClient();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString()); HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
int statusCode = response.statusCode(); int statusCode = response.statusCode();
@@ -282,17 +260,15 @@ public class OllamaAPI {
* *
* @param modelName the name of the custom model to be created. * @param modelName the name of the custom model to be created.
* @param modelFileContents the path to model file that exists on the Ollama server. * @param modelFileContents the path to model file that exists on the Ollama server.
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws URISyntaxException if the URI for the request is malformed
*/ */
public void createModelWithModelFileContents(String modelName, String modelFileContents) public void createModelWithModelFileContents(String modelName, String modelFileContents) throws IOException, InterruptedException, OllamaBaseException, URISyntaxException {
throws IOException, InterruptedException, OllamaBaseException, URISyntaxException {
String url = this.host + "/api/create"; String url = this.host + "/api/create";
String jsonData = new CustomModelFileContentsRequest(modelName, modelFileContents).toString(); String jsonData = new CustomModelFileContentsRequest(modelName, modelFileContents).toString();
HttpRequest request = HttpRequest request = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)).build();
getRequestBuilderDefault(new URI(url))
.header("Accept", "application/json")
.header("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8))
.build();
HttpClient client = HttpClient.newHttpClient(); HttpClient client = HttpClient.newHttpClient();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString()); HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
int statusCode = response.statusCode(); int statusCode = response.statusCode();
@@ -313,17 +289,15 @@ public class OllamaAPI {
* *
* @param modelName the name of the model to be deleted. * @param modelName the name of the model to be deleted.
* @param ignoreIfNotPresent ignore errors if the specified model is not present on Ollama server. * @param ignoreIfNotPresent ignore errors if the specified model is not present on Ollama server.
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws URISyntaxException if the URI for the request is malformed
*/ */
public void deleteModel(String modelName, boolean ignoreIfNotPresent) public void deleteModel(String modelName, boolean ignoreIfNotPresent) throws IOException, InterruptedException, OllamaBaseException, URISyntaxException {
throws IOException, InterruptedException, OllamaBaseException, URISyntaxException {
String url = this.host + "/api/delete"; String url = this.host + "/api/delete";
String jsonData = new ModelRequest(modelName).toString(); String jsonData = new ModelRequest(modelName).toString();
HttpRequest request = HttpRequest request = getRequestBuilderDefault(new URI(url)).method("DELETE", HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)).header("Accept", "application/json").header("Content-type", "application/json").build();
getRequestBuilderDefault(new URI(url))
.method("DELETE", HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8))
.header("Accept", "application/json")
.header("Content-type", "application/json")
.build();
HttpClient client = HttpClient.newHttpClient(); HttpClient client = HttpClient.newHttpClient();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString()); HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
int statusCode = response.statusCode(); int statusCode = response.statusCode();
@@ -342,9 +316,13 @@ public class OllamaAPI {
* @param model name of model to generate embeddings from * @param model name of model to generate embeddings from
* @param prompt text to generate embeddings for * @param prompt text to generate embeddings for
* @return embeddings * @return embeddings
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @deprecated Use {@link #embed(String, List)} instead.
*/ */
public List<Double> generateEmbeddings(String model, String prompt) @Deprecated
throws IOException, InterruptedException, OllamaBaseException { public List<Double> generateEmbeddings(String model, String prompt) throws IOException, InterruptedException, OllamaBaseException {
return generateEmbeddings(new OllamaEmbeddingsRequestModel(model, prompt)); return generateEmbeddings(new OllamaEmbeddingsRequestModel(model, prompt));
} }
@@ -353,28 +331,69 @@ public class OllamaAPI {
* *
* @param modelRequest request for '/api/embeddings' endpoint * @param modelRequest request for '/api/embeddings' endpoint
* @return embeddings * @return embeddings
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @deprecated Use {@link #embed(OllamaEmbedRequestModel)} instead.
*/ */
@Deprecated
public List<Double> generateEmbeddings(OllamaEmbeddingsRequestModel modelRequest) throws IOException, InterruptedException, OllamaBaseException { public List<Double> generateEmbeddings(OllamaEmbeddingsRequestModel modelRequest) throws IOException, InterruptedException, OllamaBaseException {
URI uri = URI.create(this.host + "/api/embeddings"); URI uri = URI.create(this.host + "/api/embeddings");
String jsonData = modelRequest.toString(); String jsonData = modelRequest.toString();
HttpClient httpClient = HttpClient.newHttpClient(); HttpClient httpClient = HttpClient.newHttpClient();
HttpRequest.Builder requestBuilder = HttpRequest.Builder requestBuilder = getRequestBuilderDefault(uri).header("Accept", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData));
getRequestBuilderDefault(uri)
.header("Accept", "application/json")
.POST(HttpRequest.BodyPublishers.ofString(jsonData));
HttpRequest request = requestBuilder.build(); HttpRequest request = requestBuilder.build();
HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString()); HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
int statusCode = response.statusCode(); int statusCode = response.statusCode();
String responseBody = response.body(); String responseBody = response.body();
if (statusCode == 200) { if (statusCode == 200) {
OllamaEmbeddingResponseModel embeddingResponse = OllamaEmbeddingResponseModel embeddingResponse = Utils.getObjectMapper().readValue(responseBody, OllamaEmbeddingResponseModel.class);
Utils.getObjectMapper().readValue(responseBody, OllamaEmbeddingResponseModel.class);
return embeddingResponse.getEmbedding(); return embeddingResponse.getEmbedding();
} else { } else {
throw new OllamaBaseException(statusCode + " - " + responseBody); throw new OllamaBaseException(statusCode + " - " + responseBody);
} }
} }
/**
* Generate embeddings for a given text from a model
*
* @param model name of model to generate embeddings from
* @param inputs text/s to generate embeddings for
* @return embeddings
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/
public OllamaEmbedResponseModel embed(String model, List<String> inputs) throws IOException, InterruptedException, OllamaBaseException {
return embed(new OllamaEmbedRequestModel(model, inputs));
}
/**
* Generate embeddings using a {@link OllamaEmbedRequestModel}.
*
* @param modelRequest request for '/api/embed' endpoint
* @return embeddings
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/
public OllamaEmbedResponseModel embed(OllamaEmbedRequestModel modelRequest) throws IOException, InterruptedException, OllamaBaseException {
URI uri = URI.create(this.host + "/api/embed");
String jsonData = Utils.getObjectMapper().writeValueAsString(modelRequest);
HttpClient httpClient = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder(uri).header("Accept", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData)).build();
HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
int statusCode = response.statusCode();
String responseBody = response.body();
if (statusCode == 200) {
return Utils.getObjectMapper().readValue(responseBody, OllamaEmbedResponseModel.class);
} else {
throw new OllamaBaseException(statusCode + " - " + responseBody);
}
}
/** /**
* Generate response for a question to a model running on Ollama server. This is a sync/blocking * Generate response for a question to a model running on Ollama server. This is a sync/blocking
@@ -387,9 +406,11 @@ public class OllamaAPI {
* details on the options</a> * details on the options</a>
* @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false. * @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false.
* @return OllamaResult that includes response text and time taken for response * @return OllamaResult that includes response text and time taken for response
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/ */
public OllamaResult generate(String model, String prompt, boolean raw, Options options, OllamaStreamHandler streamHandler) public OllamaResult generate(String model, String prompt, boolean raw, Options options, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException {
throws OllamaBaseException, IOException, InterruptedException {
OllamaGenerateRequest ollamaRequestModel = new OllamaGenerateRequest(model, prompt); OllamaGenerateRequest ollamaRequestModel = new OllamaGenerateRequest(model, prompt);
ollamaRequestModel.setRaw(raw); ollamaRequestModel.setRaw(raw);
ollamaRequestModel.setOptions(options.getOptionsMap()); ollamaRequestModel.setOptions(options.getOptionsMap());
@@ -406,13 +427,14 @@ public class OllamaAPI {
* @param raw In some cases, you may wish to bypass the templating system and provide a full prompt. In this case, you can use the raw parameter to disable templating. Also note that raw mode will not return a context. * @param raw In some cases, you may wish to bypass the templating system and provide a full prompt. In this case, you can use the raw parameter to disable templating. Also note that raw mode will not return a context.
* @param options Additional options or configurations to use when generating the response. * @param options Additional options or configurations to use when generating the response.
* @return {@link OllamaResult} * @return {@link OllamaResult}
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/ */
public OllamaResult generate(String model, String prompt, boolean raw, Options options) public OllamaResult generate(String model, String prompt, boolean raw, Options options) throws OllamaBaseException, IOException, InterruptedException {
throws OllamaBaseException, IOException, InterruptedException {
return generate(model, prompt, raw, options, null); return generate(model, prompt, raw, options, null);
} }
/** /**
* Generates response using the specified AI model and prompt (in blocking mode), and then invokes a set of tools * Generates response using the specified AI model and prompt (in blocking mode), and then invokes a set of tools
* on the generated response. * on the generated response.
@@ -421,13 +443,11 @@ public class OllamaAPI {
* @param prompt The input text or prompt to provide to the AI model. * @param prompt The input text or prompt to provide to the AI model.
* @param options Additional options or configurations to use when generating the response. * @param options Additional options or configurations to use when generating the response.
* @return {@link OllamaToolsResult} An OllamaToolsResult object containing the response from the AI model and the results of invoking the tools on that output. * @return {@link OllamaToolsResult} An OllamaToolsResult object containing the response from the AI model and the results of invoking the tools on that output.
* @throws OllamaBaseException If there is an error related to the Ollama API or service. * @throws OllamaBaseException if the response indicates an error status
* @throws IOException If there is an error related to input/output operations. * @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException If the method is interrupted while waiting for the AI model * @throws InterruptedException if the operation is interrupted
* to generate the response or for the tools to be invoked.
*/ */
public OllamaToolsResult generateWithTools(String model, String prompt, Options options) public OllamaToolsResult generateWithTools(String model, String prompt, Options options) throws OllamaBaseException, IOException, InterruptedException, ToolInvocationException {
throws OllamaBaseException, IOException, InterruptedException, ToolInvocationException {
boolean raw = true; boolean raw = true;
OllamaToolsResult toolResult = new OllamaToolsResult(); OllamaToolsResult toolResult = new OllamaToolsResult();
Map<ToolFunctionCallSpec, Object> toolResults = new HashMap<>(); Map<ToolFunctionCallSpec, Object> toolResults = new HashMap<>();
@@ -448,7 +468,6 @@ public class OllamaAPI {
return toolResult; return toolResult;
} }
/** /**
* Generate response for a question to a model running on Ollama server and get a callback handle * Generate response for a question to a model running on Ollama server and get a callback handle
* that can be used to check for status and get the response from the model later. This would be * that can be used to check for status and get the response from the model later. This would be
@@ -462,9 +481,7 @@ public class OllamaAPI {
OllamaGenerateRequest ollamaRequestModel = new OllamaGenerateRequest(model, prompt); OllamaGenerateRequest ollamaRequestModel = new OllamaGenerateRequest(model, prompt);
ollamaRequestModel.setRaw(raw); ollamaRequestModel.setRaw(raw);
URI uri = URI.create(this.host + "/api/generate"); URI uri = URI.create(this.host + "/api/generate");
OllamaAsyncResultStreamer ollamaAsyncResultStreamer = OllamaAsyncResultStreamer ollamaAsyncResultStreamer = new OllamaAsyncResultStreamer(getRequestBuilderDefault(uri), ollamaRequestModel, requestTimeoutSeconds);
new OllamaAsyncResultStreamer(
getRequestBuilderDefault(uri), ollamaRequestModel, requestTimeoutSeconds);
ollamaAsyncResultStreamer.start(); ollamaAsyncResultStreamer.start();
return ollamaAsyncResultStreamer; return ollamaAsyncResultStreamer;
} }
@@ -481,10 +498,11 @@ public class OllamaAPI {
* details on the options</a> * details on the options</a>
* @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false. * @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false.
* @return OllamaResult that includes response text and time taken for response * @return OllamaResult that includes response text and time taken for response
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/ */
public OllamaResult generateWithImageFiles( public OllamaResult generateWithImageFiles(String model, String prompt, List<File> imageFiles, Options options, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException {
String model, String prompt, List<File> imageFiles, Options options, OllamaStreamHandler streamHandler)
throws OllamaBaseException, IOException, InterruptedException {
List<String> images = new ArrayList<>(); List<String> images = new ArrayList<>();
for (File imageFile : imageFiles) { for (File imageFile : imageFiles) {
images.add(encodeFileToBase64(imageFile)); images.add(encodeFileToBase64(imageFile));
@@ -498,10 +516,12 @@ public class OllamaAPI {
* Convenience method to call Ollama API without streaming responses. * Convenience method to call Ollama API without streaming responses.
* <p> * <p>
* Uses {@link #generateWithImageFiles(String, String, List, Options, OllamaStreamHandler)} * Uses {@link #generateWithImageFiles(String, String, List, Options, OllamaStreamHandler)}
*
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/ */
public OllamaResult generateWithImageFiles( public OllamaResult generateWithImageFiles(String model, String prompt, List<File> imageFiles, Options options) throws OllamaBaseException, IOException, InterruptedException {
String model, String prompt, List<File> imageFiles, Options options)
throws OllamaBaseException, IOException, InterruptedException {
return generateWithImageFiles(model, prompt, imageFiles, options, null); return generateWithImageFiles(model, prompt, imageFiles, options, null);
} }
@@ -517,10 +537,12 @@ public class OllamaAPI {
* details on the options</a> * details on the options</a>
* @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false. * @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false.
* @return OllamaResult that includes response text and time taken for response * @return OllamaResult that includes response text and time taken for response
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws URISyntaxException if the URI for the request is malformed
*/ */
public OllamaResult generateWithImageURLs( public OllamaResult generateWithImageURLs(String model, String prompt, List<String> imageURLs, Options options, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException, URISyntaxException {
String model, String prompt, List<String> imageURLs, Options options, OllamaStreamHandler streamHandler)
throws OllamaBaseException, IOException, InterruptedException, URISyntaxException {
List<String> images = new ArrayList<>(); List<String> images = new ArrayList<>();
for (String imageURL : imageURLs) { for (String imageURL : imageURLs) {
images.add(encodeByteArrayToBase64(Utils.loadImageBytesFromUrl(imageURL))); images.add(encodeByteArrayToBase64(Utils.loadImageBytesFromUrl(imageURL)));
@@ -534,14 +556,16 @@ public class OllamaAPI {
* Convenience method to call Ollama API without streaming responses. * Convenience method to call Ollama API without streaming responses.
* <p> * <p>
* Uses {@link #generateWithImageURLs(String, String, List, Options, OllamaStreamHandler)} * Uses {@link #generateWithImageURLs(String, String, List, Options, OllamaStreamHandler)}
*
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
* @throws URISyntaxException if the URI for the request is malformed
*/ */
public OllamaResult generateWithImageURLs(String model, String prompt, List<String> imageURLs, public OllamaResult generateWithImageURLs(String model, String prompt, List<String> imageURLs, Options options) throws OllamaBaseException, IOException, InterruptedException, URISyntaxException {
Options options)
throws OllamaBaseException, IOException, InterruptedException, URISyntaxException {
return generateWithImageURLs(model, prompt, imageURLs, options, null); return generateWithImageURLs(model, prompt, imageURLs, options, null);
} }
/** /**
* Ask a question to a model based on a given message stack (i.e. a chat history). Creates a synchronous call to the api * Ask a question to a model based on a given message stack (i.e. a chat history). Creates a synchronous call to the api
* 'api/chat'. * 'api/chat'.
@@ -552,6 +576,9 @@ public class OllamaAPI {
* @throws OllamaBaseException any response code than 200 has been returned * @throws OllamaBaseException any response code than 200 has been returned
* @throws IOException in case the responseStream can not be read * @throws IOException in case the responseStream can not be read
* @throws InterruptedException in case the server is not reachable or network issues happen * @throws InterruptedException in case the server is not reachable or network issues happen
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/ */
public OllamaChatResult chat(String model, List<OllamaChatMessage> messages) throws OllamaBaseException, IOException, InterruptedException { public OllamaChatResult chat(String model, List<OllamaChatMessage> messages) throws OllamaBaseException, IOException, InterruptedException {
OllamaChatRequestBuilder builder = OllamaChatRequestBuilder.getInstance(model); OllamaChatRequestBuilder builder = OllamaChatRequestBuilder.getInstance(model);
@@ -568,6 +595,9 @@ public class OllamaAPI {
* @throws OllamaBaseException any response code than 200 has been returned * @throws OllamaBaseException any response code than 200 has been returned
* @throws IOException in case the responseStream can not be read * @throws IOException in case the responseStream can not be read
* @throws InterruptedException in case the server is not reachable or network issues happen * @throws InterruptedException in case the server is not reachable or network issues happen
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/ */
public OllamaChatResult chat(OllamaChatRequest request) throws OllamaBaseException, IOException, InterruptedException { public OllamaChatResult chat(OllamaChatRequest request) throws OllamaBaseException, IOException, InterruptedException {
return chat(request, null); return chat(request, null);
@@ -584,6 +614,9 @@ public class OllamaAPI {
* @throws OllamaBaseException any response code than 200 has been returned * @throws OllamaBaseException any response code than 200 has been returned
* @throws IOException in case the responseStream can not be read * @throws IOException in case the responseStream can not be read
* @throws InterruptedException in case the server is not reachable or network issues happen * @throws InterruptedException in case the server is not reachable or network issues happen
* @throws OllamaBaseException if the response indicates an error status
* @throws IOException if an I/O error occurs during the HTTP request
* @throws InterruptedException if the operation is interrupted
*/ */
public OllamaChatResult chat(OllamaChatRequest request, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException { public OllamaChatResult chat(OllamaChatRequest request, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException {
OllamaChatEndpointCaller requestCaller = new OllamaChatEndpointCaller(host, basicAuth, requestTimeoutSeconds, verbose); OllamaChatEndpointCaller requestCaller = new OllamaChatEndpointCaller(host, basicAuth, requestTimeoutSeconds, verbose);
@@ -601,6 +634,37 @@ public class OllamaAPI {
toolRegistry.addFunction(toolSpecification.getFunctionName(), toolSpecification.getToolDefinition()); toolRegistry.addFunction(toolSpecification.getFunctionName(), toolSpecification.getToolDefinition());
} }
/**
* Adds a custom role.
*
* @param roleName the name of the custom role to be added
* @return the newly created OllamaChatMessageRole
*/
public OllamaChatMessageRole addCustomRole(String roleName) {
return OllamaChatMessageRole.newCustomRole(roleName);
}
/**
* Lists all available roles.
*
* @return a list of available OllamaChatMessageRole objects
*/
public List<OllamaChatMessageRole> listRoles() {
return OllamaChatMessageRole.getRoles();
}
/**
* Retrieves a specific role by name.
*
* @param roleName the name of the role to retrieve
* @return the OllamaChatMessageRole associated with the given name
* @throws RoleNotFoundException if the role with the specified name does not exist
*/
public OllamaChatMessageRole getRole(String roleName) throws RoleNotFoundException {
return OllamaChatMessageRole.getRole(roleName);
}
// technical private methods // // technical private methods //
private static String encodeFileToBase64(File file) throws IOException { private static String encodeFileToBase64(File file) throws IOException {
@@ -611,11 +675,8 @@ public class OllamaAPI {
return Base64.getEncoder().encodeToString(bytes); return Base64.getEncoder().encodeToString(bytes);
} }
private OllamaResult generateSyncForOllamaRequestModel( private OllamaResult generateSyncForOllamaRequestModel(OllamaGenerateRequest ollamaRequestModel, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException {
OllamaGenerateRequest ollamaRequestModel, OllamaStreamHandler streamHandler) OllamaGenerateEndpointCaller requestCaller = new OllamaGenerateEndpointCaller(host, basicAuth, requestTimeoutSeconds, verbose);
throws OllamaBaseException, IOException, InterruptedException {
OllamaGenerateEndpointCaller requestCaller =
new OllamaGenerateEndpointCaller(host, basicAuth, requestTimeoutSeconds, verbose);
OllamaResult result; OllamaResult result;
if (streamHandler != null) { if (streamHandler != null) {
ollamaRequestModel.setStream(true); ollamaRequestModel.setStream(true);
@@ -633,10 +694,7 @@ public class OllamaAPI {
* @return HttpRequest.Builder * @return HttpRequest.Builder
*/ */
private HttpRequest.Builder getRequestBuilderDefault(URI uri) { private HttpRequest.Builder getRequestBuilderDefault(URI uri) {
HttpRequest.Builder requestBuilder = HttpRequest.Builder requestBuilder = HttpRequest.newBuilder(uri).header("Content-Type", "application/json").timeout(Duration.ofSeconds(requestTimeoutSeconds));
HttpRequest.newBuilder(uri)
.header("Content-Type", "application/json")
.timeout(Duration.ofSeconds(requestTimeoutSeconds));
if (isBasicAuthCredentialsSet()) { if (isBasicAuthCredentialsSet()) {
requestBuilder.header("Authorization", getBasicAuthHeaderValue()); requestBuilder.header("Authorization", getBasicAuthHeaderValue());
} }
@@ -662,7 +720,6 @@ public class OllamaAPI {
return basicAuth != null; return basicAuth != null;
} }
private Object invokeTool(ToolFunctionCallSpec toolFunctionCallSpec) throws ToolInvocationException { private Object invokeTool(ToolFunctionCallSpec toolFunctionCallSpec) throws ToolInvocationException {
try { try {
String methodName = toolFunctionCallSpec.getName(); String methodName = toolFunctionCallSpec.getName();

View File

@@ -0,0 +1,8 @@
package io.github.ollama4j.exceptions;
public class RoleNotFoundException extends Exception {
public RoleNotFoundException(String s) {
super(s);
}
}

View File

@@ -1,19 +1,53 @@
package io.github.ollama4j.models.chat; package io.github.ollama4j.models.chat;
import com.fasterxml.jackson.annotation.JsonValue; import com.fasterxml.jackson.annotation.JsonValue;
import io.github.ollama4j.exceptions.RoleNotFoundException;
import lombok.Getter;
import java.util.ArrayList;
import java.util.List;
/** /**
* Defines the possible Chat Message roles. * Defines the possible Chat Message roles.
*/ */
public enum OllamaChatMessageRole { @Getter
SYSTEM("system"), public class OllamaChatMessageRole {
USER("user"), private static final List<OllamaChatMessageRole> ROLES = new ArrayList<>();
ASSISTANT("assistant");
public static final OllamaChatMessageRole SYSTEM = new OllamaChatMessageRole("system");
public static final OllamaChatMessageRole USER = new OllamaChatMessageRole("user");
public static final OllamaChatMessageRole ASSISTANT = new OllamaChatMessageRole("assistant");
public static final OllamaChatMessageRole TOOL = new OllamaChatMessageRole("tool");
@JsonValue @JsonValue
private String roleName; private final String roleName;
private OllamaChatMessageRole(String roleName){ private OllamaChatMessageRole(String roleName) {
this.roleName = roleName; this.roleName = roleName;
ROLES.add(this);
}
public static OllamaChatMessageRole newCustomRole(String roleName) {
OllamaChatMessageRole customRole = new OllamaChatMessageRole(roleName);
ROLES.add(customRole);
return customRole;
}
public static List<OllamaChatMessageRole> getRoles() {
return new ArrayList<>(ROLES);
}
public static OllamaChatMessageRole getRole(String roleName) throws RoleNotFoundException {
for (OllamaChatMessageRole role : ROLES) {
if (role.roleName.equals(roleName)) {
return role;
}
}
throw new RoleNotFoundException("Invalid role name: " + roleName);
}
@Override
public String toString() {
return roleName;
} }
} }

View File

@@ -8,12 +8,11 @@ import io.github.ollama4j.models.response.OllamaResult;
* Specific chat-API result that contains the chat history sent to the model and appends the answer as {@link OllamaChatResult} given by the * Specific chat-API result that contains the chat history sent to the model and appends the answer as {@link OllamaChatResult} given by the
* {@link OllamaChatMessageRole#ASSISTANT} role. * {@link OllamaChatMessageRole#ASSISTANT} role.
*/ */
public class OllamaChatResult extends OllamaResult{ public class OllamaChatResult extends OllamaResult {
private List<OllamaChatMessage> chatHistory; private List<OllamaChatMessage> chatHistory;
public OllamaChatResult(String response, long responseTime, int httpStatusCode, public OllamaChatResult(String response, long responseTime, int httpStatusCode, List<OllamaChatMessage> chatHistory) {
List<OllamaChatMessage> chatHistory) {
super(response, responseTime, httpStatusCode); super(response, responseTime, httpStatusCode);
this.chatHistory = chatHistory; this.chatHistory = chatHistory;
appendAnswerToChatHistory(response); appendAnswerToChatHistory(response);
@@ -21,12 +20,10 @@ public class OllamaChatResult extends OllamaResult{
public List<OllamaChatMessage> getChatHistory() { public List<OllamaChatMessage> getChatHistory() {
return chatHistory; return chatHistory;
} }
private void appendAnswerToChatHistory(String answer){ private void appendAnswerToChatHistory(String answer) {
OllamaChatMessage assistantMessage = new OllamaChatMessage(OllamaChatMessageRole.ASSISTANT, answer); OllamaChatMessage assistantMessage = new OllamaChatMessage(OllamaChatMessageRole.ASSISTANT, answer);
this.chatHistory.add(assistantMessage); this.chatHistory.add(assistantMessage);
} }
} }

View File

@@ -0,0 +1,41 @@
package io.github.ollama4j.models.embeddings;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.JsonProcessingException;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.NonNull;
import lombok.RequiredArgsConstructor;
import java.util.List;
import java.util.Map;
import static io.github.ollama4j.utils.Utils.getObjectMapper;
@Data
@RequiredArgsConstructor
@NoArgsConstructor
public class OllamaEmbedRequestModel {
@NonNull
private String model;
@NonNull
private List<String> input;
private Map<String, Object> options;
@JsonProperty(value = "keep_alive")
private String keepAlive;
@JsonProperty(value = "truncate")
private Boolean truncate = true;
@Override
public String toString() {
try {
return getObjectMapper().writerWithDefaultPrettyPrinter().writeValueAsString(this);
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
}
}

View File

@@ -0,0 +1,25 @@
package io.github.ollama4j.models.embeddings;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Data;
import java.util.List;
@SuppressWarnings("unused")
@Data
public class OllamaEmbedResponseModel {
@JsonProperty("model")
private String model;
@JsonProperty("embeddings")
private List<List<Double>> embeddings;
@JsonProperty("total_duration")
private long totalDuration;
@JsonProperty("load_duration")
private long loadDuration;
@JsonProperty("prompt_eval_count")
private int promptEvalCount;
}

View File

@@ -10,12 +10,9 @@ package io.github.ollama4j.types;
public class OllamaModelType { public class OllamaModelType {
public static final String GEMMA = "gemma"; public static final String GEMMA = "gemma";
public static final String GEMMA2 = "gemma2"; public static final String GEMMA2 = "gemma2";
public static final String LLAMA2 = "llama2"; public static final String LLAMA2 = "llama2";
public static final String LLAMA3 = "llama3"; public static final String LLAMA3 = "llama3";
public static final String LLAMA3_1 = "llama3.1"; public static final String LLAMA3_1 = "llama3.1";
public static final String MISTRAL = "mistral"; public static final String MISTRAL = "mistral";
public static final String MIXTRAL = "mixtral"; public static final String MIXTRAL = "mixtral";
public static final String LLAVA = "llava"; public static final String LLAVA = "llava";
@@ -35,7 +32,6 @@ public class OllamaModelType {
public static final String ZEPHYR = "zephyr"; public static final String ZEPHYR = "zephyr";
public static final String OPENHERMES = "openhermes"; public static final String OPENHERMES = "openhermes";
public static final String QWEN = "qwen"; public static final String QWEN = "qwen";
public static final String QWEN2 = "qwen2"; public static final String QWEN2 = "qwen2";
public static final String WIZARDCODER = "wizardcoder"; public static final String WIZARDCODER = "wizardcoder";
public static final String LLAMA2_CHINESE = "llama2-chinese"; public static final String LLAMA2_CHINESE = "llama2-chinese";

View File

@@ -1,5 +1,6 @@
package io.github.ollama4j.utils; package io.github.ollama4j.utils;
import java.io.IOException;
import java.util.HashMap; import java.util.HashMap;
/** Builder class for creating options for Ollama model. */ /** Builder class for creating options for Ollama model. */
@@ -207,6 +208,34 @@ public class OptionsBuilder {
return this; return this;
} }
/**
* Alternative to the top_p, and aims to ensure a balance of qualityand variety. The parameter p
* represents the minimum probability for a token to be considered, relative to the probability
* of the most likely token. For example, with p=0.05 and the most likely token having a
* probability of 0.9, logits with a value less than 0.045 are filtered out. (Default: 0.0)
*/
public OptionsBuilder setMinP(float value) {
options.getOptionsMap().put("min_p", value);
return this;
}
/**
* Allows passing an option not formally supported by the library
* @param name The option name for the parameter.
* @param value The value for the "{name}" parameter.
* @return The updated OptionsBuilder.
* @throws IllegalArgumentException if parameter has an unsupported type
*/
public OptionsBuilder setCustomOption(String name, Object value) throws IllegalArgumentException {
if (!(value instanceof Integer || value instanceof Float || value instanceof String)) {
throw new IllegalArgumentException("Invalid type for parameter. Allowed types are: Integer, Float, or String.");
}
options.getOptionsMap().put(name, value);
return this;
}
/** /**
* Builds the options map. * Builds the options map.
* *
@@ -215,4 +244,6 @@ public class OptionsBuilder {
public Options build() { public Options build() {
return options; return options;
} }
} }

View File

@@ -2,6 +2,10 @@ package io.github.ollama4j.unittests;
import io.github.ollama4j.OllamaAPI; import io.github.ollama4j.OllamaAPI;
import io.github.ollama4j.exceptions.OllamaBaseException; import io.github.ollama4j.exceptions.OllamaBaseException;
import io.github.ollama4j.exceptions.RoleNotFoundException;
import io.github.ollama4j.models.chat.OllamaChatMessageRole;
import io.github.ollama4j.models.embeddings.OllamaEmbedRequestModel;
import io.github.ollama4j.models.embeddings.OllamaEmbedResponseModel;
import io.github.ollama4j.models.response.ModelDetail; import io.github.ollama4j.models.response.ModelDetail;
import io.github.ollama4j.models.response.OllamaAsyncResultStreamer; import io.github.ollama4j.models.response.OllamaAsyncResultStreamer;
import io.github.ollama4j.models.response.OllamaResult; import io.github.ollama4j.models.response.OllamaResult;
@@ -14,7 +18,9 @@ import java.io.IOException;
import java.net.URISyntaxException; import java.net.URISyntaxException;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Collections; import java.util.Collections;
import java.util.List;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*; import static org.mockito.Mockito.*;
class TestMockedAPIs { class TestMockedAPIs {
@@ -97,6 +103,34 @@ class TestMockedAPIs {
} }
} }
@Test
void testEmbed() {
OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class);
String model = OllamaModelType.LLAMA2;
List<String> inputs = List.of("some prompt text");
try {
when(ollamaAPI.embed(model, inputs)).thenReturn(new OllamaEmbedResponseModel());
ollamaAPI.embed(model, inputs);
verify(ollamaAPI, times(1)).embed(model, inputs);
} catch (IOException | OllamaBaseException | InterruptedException e) {
throw new RuntimeException(e);
}
}
@Test
void testEmbedWithEmbedRequestModel() {
OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class);
String model = OllamaModelType.LLAMA2;
List<String> inputs = List.of("some prompt text");
try {
when(ollamaAPI.embed(new OllamaEmbedRequestModel(model, inputs))).thenReturn(new OllamaEmbedResponseModel());
ollamaAPI.embed(new OllamaEmbedRequestModel(model, inputs));
verify(ollamaAPI, times(1)).embed(new OllamaEmbedRequestModel(model, inputs));
} catch (IOException | OllamaBaseException | InterruptedException e) {
throw new RuntimeException(e);
}
}
@Test @Test
void testAsk() { void testAsk() {
OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class); OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class);
@@ -161,4 +195,68 @@ class TestMockedAPIs {
ollamaAPI.generateAsync(model, prompt, false); ollamaAPI.generateAsync(model, prompt, false);
verify(ollamaAPI, times(1)).generateAsync(model, prompt, false); verify(ollamaAPI, times(1)).generateAsync(model, prompt, false);
} }
@Test
void testAddCustomRole() {
OllamaAPI ollamaAPI = mock(OllamaAPI.class);
String roleName = "custom-role";
OllamaChatMessageRole expectedRole = OllamaChatMessageRole.newCustomRole(roleName);
when(ollamaAPI.addCustomRole(roleName)).thenReturn(expectedRole);
OllamaChatMessageRole customRole = ollamaAPI.addCustomRole(roleName);
assertEquals(expectedRole, customRole);
verify(ollamaAPI, times(1)).addCustomRole(roleName);
}
@Test
void testListRoles() {
OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class);
OllamaChatMessageRole role1 = OllamaChatMessageRole.newCustomRole("role1");
OllamaChatMessageRole role2 = OllamaChatMessageRole.newCustomRole("role2");
List<OllamaChatMessageRole> expectedRoles = List.of(role1, role2);
when(ollamaAPI.listRoles()).thenReturn(expectedRoles);
List<OllamaChatMessageRole> actualRoles = ollamaAPI.listRoles();
assertEquals(expectedRoles, actualRoles);
verify(ollamaAPI, times(1)).listRoles();
}
@Test
void testGetRoleNotFound() {
OllamaAPI ollamaAPI = mock(OllamaAPI.class);
String roleName = "non-existing-role";
try {
when(ollamaAPI.getRole(roleName)).thenThrow(new RoleNotFoundException("Role not found"));
} catch (RoleNotFoundException exception) {
throw new RuntimeException("Failed to run test: testGetRoleNotFound");
}
try {
ollamaAPI.getRole(roleName);
fail("Expected RoleNotFoundException not thrown");
} catch (RoleNotFoundException exception) {
assertEquals("Role not found", exception.getMessage());
}
try {
verify(ollamaAPI, times(1)).getRole(roleName);
} catch (RoleNotFoundException exception) {
throw new RuntimeException("Failed to run test: testGetRoleNotFound");
}
}
@Test
void testGetRoleFound() {
OllamaAPI ollamaAPI = mock(OllamaAPI.class);
String roleName = "existing-role";
OllamaChatMessageRole expectedRole = OllamaChatMessageRole.newCustomRole(roleName);
try {
when(ollamaAPI.getRole(roleName)).thenReturn(expectedRole);
} catch (RoleNotFoundException exception) {
throw new RuntimeException("Failed to run test: testGetRoleFound");
}
try {
OllamaChatMessageRole actualRole = ollamaAPI.getRole(roleName);
assertEquals(expectedRole, actualRole);
verify(ollamaAPI, times(1)).getRole(roleName);
} catch (RoleNotFoundException exception) {
throw new RuntimeException("Failed to run test: testGetRoleFound");
}
}
} }

View File

@@ -1,6 +1,7 @@
package io.github.ollama4j.unittests.jackson; package io.github.ollama4j.unittests.jackson;
import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrowsExactly;
import java.io.File; import java.io.File;
import java.util.List; import java.util.List;
@@ -59,6 +60,10 @@ public class TestChatRequestSerialization extends AbstractSerializationTest<Olla
.withOptions(b.setSeed(1).build()) .withOptions(b.setSeed(1).build())
.withOptions(b.setTopK(1).build()) .withOptions(b.setTopK(1).build())
.withOptions(b.setTopP(1).build()) .withOptions(b.setTopP(1).build())
.withOptions(b.setMinP(1).build())
.withOptions(b.setCustomOption("cust_float", 1.0f).build())
.withOptions(b.setCustomOption("cust_int", 1).build())
.withOptions(b.setCustomOption("cust_str", "custom").build())
.build(); .build();
String jsonRequest = serialize(req); String jsonRequest = serialize(req);
@@ -72,6 +77,20 @@ public class TestChatRequestSerialization extends AbstractSerializationTest<Olla
assertEquals(1, deserializeRequest.getOptions().get("seed")); assertEquals(1, deserializeRequest.getOptions().get("seed"));
assertEquals(1, deserializeRequest.getOptions().get("top_k")); assertEquals(1, deserializeRequest.getOptions().get("top_k"));
assertEquals(1.0, deserializeRequest.getOptions().get("top_p")); assertEquals(1.0, deserializeRequest.getOptions().get("top_p"));
assertEquals(1.0, deserializeRequest.getOptions().get("min_p"));
assertEquals(1.0, deserializeRequest.getOptions().get("cust_float"));
assertEquals(1, deserializeRequest.getOptions().get("cust_int"));
assertEquals("custom", deserializeRequest.getOptions().get("cust_str"));
}
@Test
public void testRequestWithInvalidCustomOption() {
OptionsBuilder b = new OptionsBuilder();
assertThrowsExactly(IllegalArgumentException.class, () -> {
OllamaChatRequest req = builder.withMessage(OllamaChatMessageRole.USER, "Some prompt")
.withOptions(b.setCustomOption("cust_obj", new Object()).build())
.build();
});
} }
@Test @Test