mirror of
				https://github.com/amithkoujalgi/ollama4j.git
				synced 2025-10-31 16:40:41 +01:00 
			
		
		
		
	Compare commits
	
		
			28 Commits
		
	
	
		
	
	| Author | SHA1 | Date | |
|---|---|---|---|
|   | 3fc7e9423c | ||
|   | 405a08b330 | ||
|   | 921f745435 | ||
|   | bedfec6bf9 | ||
|   | afa09e87a5 | ||
|   | baf2320ea6 | ||
|   | 948a7444fb | ||
|   | ec0eb8b469 | ||
|   | 8f33de7e59 | ||
|   | 8c59e6511b | ||
|   | b93fc7623a | ||
|   | bd1a57c7e0 | ||
|   | 7fabead249 | ||
|   | 268a973d5e | ||
|   | d949a3cb69 | ||
|   | e2443ed68a | ||
|   | 37193b1f5b | ||
|   | e33071ae38 | ||
|   | fffc8dc526 | ||
|   | def950cc9c | ||
|   | f4db7ca326 | ||
|   | 18760250ea | ||
|   | 233597efd1 | ||
|   | cec9f29eb7 | ||
|   | 20cb92a418 | ||
|   | b0dc38954b | ||
|   | 1479d0a494 | ||
|   | b328daee43 | 
							
								
								
									
										38
									
								
								README.md
									
									
									
									
									
								
							
							
						
						
									
										38
									
								
								README.md
									
									
									
									
									
								
							| @@ -4,12 +4,11 @@ | ||||
|   <img src='https://raw.githubusercontent.com/ollama4j/ollama4j/65a9d526150da8fcd98e2af6a164f055572bf722/ollama4j.jpeg' width='100' alt="ollama4j-icon"> | ||||
| </p> | ||||
|  | ||||
|  | ||||
| A Java library (wrapper/binding) for [Ollama](https://ollama.ai/) server. | ||||
| <div align="center"> | ||||
| A Java library (wrapper/binding) for Ollama server. | ||||
|  | ||||
| Find more details on the [website](https://ollama4j.github.io/ollama4j/). | ||||
|  | ||||
| <div align="center"> | ||||
|  | ||||
|  | ||||
|  | ||||
| @@ -268,7 +267,10 @@ make integration-tests | ||||
|  | ||||
| Newer artifacts are published via GitHub Actions CI workflow when a new release is created from `main` branch. | ||||
|  | ||||
| #### Who's using Ollama4j? | ||||
| ## ⭐ Give us a Star! | ||||
| If you like or are using this project to build your own, please give us a star. It's a free way to show your support. | ||||
|  | ||||
| ## Who's using Ollama4j? | ||||
|  | ||||
| - `Datafaker`: a library to generate fake data | ||||
|     - https://github.com/datafaker-net/datafaker-experimental/tree/main/ollama-api | ||||
| @@ -277,18 +279,26 @@ Newer artifacts are published via GitHub Actions CI workflow when a new release | ||||
| - `ollama-translator`: Minecraft 1.20.6 spigot plugin allows to easily break language barriers by using ollama on the | ||||
|   server to translate all messages into a specfic target language. | ||||
|     - https://github.com/liebki/ollama-translator | ||||
| - `AI Player`: A minecraft mod which aims to add a "second player" into the game which will actually be intelligent. | ||||
|     - https://github.com/shasankp000/AI-Player | ||||
|     - https://www.reddit.com/r/fabricmc/comments/1e65x5s/comment/ldr2vcf/ | ||||
| - `Ollama4j Web UI`: A web UI for Ollama written in Java using Spring Boot and Vaadin framework and | ||||
|   Ollama4j. | ||||
|     - https://github.com/ollama4j/ollama4j-web-ui | ||||
| - `JnsCLI`: A command-line tool for Jenkins that manages jobs, builds, and configurations directly from the terminal while offering AI-powered error analysis for quick troubleshooting. | ||||
|     -  https://github.com/mirum8/jnscli | ||||
| - `Katie Backend`: An Open Source AI-based question-answering platform that helps companies and organizations make their private domain knowledge accessible and useful to their employees and customers. | ||||
|     - https://github.com/wyona/katie-backend | ||||
| - `TeleLlama3 Bot`: A Question-Answering Telegram Bot. | ||||
|     - https://git.hiast.edu.sy/mohamadbashar.disoki/telellama3-bot | ||||
| - `moqui-wechat`: A wechat plugin | ||||
|     - https://github.com/heguangyong/moqui-wechat | ||||
|  | ||||
| #### Traction | ||||
| ## Traction | ||||
|  | ||||
| [](https://star-history.com/#ollama4j/ollama4j&Date) | ||||
|  | ||||
| ### Get Involved | ||||
| ## Get Involved | ||||
|  | ||||
| <div align="center"> | ||||
|  | ||||
| @@ -316,6 +326,22 @@ Contributions are most welcome! Whether it's reporting a bug, proposing an enhan | ||||
| with code - any sort | ||||
| of contribution is much appreciated. | ||||
|  | ||||
| ## 🏷️ License and Citation | ||||
|  | ||||
| The code is available under [MIT License](./LICENSE). | ||||
|  | ||||
| If you find this project helpful in your research, please cite this work at | ||||
|  | ||||
| ``` | ||||
| @misc{ollama4j2024, | ||||
|     author       = {Amith Koujalgi}, | ||||
|     title        = {Ollama4j: A Java Library (Wrapper/Binding) for Ollama Server}, | ||||
|     year         = {2024}, | ||||
|     month        = {January}, | ||||
|     url          = {https://github.com/ollama4j/ollama4j} | ||||
| } | ||||
| ``` | ||||
|  | ||||
| ### References | ||||
|  | ||||
| - [Ollama REST APIs](https://github.com/jmorganca/ollama/blob/main/docs/api.md) | ||||
|   | ||||
							
								
								
									
										65
									
								
								docs/docs/apis-generate/custom-roles.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										65
									
								
								docs/docs/apis-generate/custom-roles.md
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,65 @@ | ||||
| --- | ||||
| sidebar_position: 8 | ||||
| --- | ||||
|  | ||||
| # Custom Roles | ||||
|  | ||||
| Allows to manage custom roles (apart from the base roles) for chat interactions with the models. | ||||
|  | ||||
| _Particularly helpful when you would need to use different roles that the newer models support other than the base | ||||
| roles._ | ||||
|  | ||||
| _Base roles are `SYSTEM`, `USER`, `ASSISTANT`, `TOOL`._ | ||||
|  | ||||
| ### Usage | ||||
|  | ||||
| #### Add new role | ||||
|  | ||||
| ```java | ||||
| import io.github.ollama4j.OllamaAPI; | ||||
| import io.github.ollama4j.models.chat.OllamaChatMessageRole; | ||||
|  | ||||
| public class Main { | ||||
|  | ||||
|     public static void main(String[] args) { | ||||
|         String host = "http://localhost:11434/"; | ||||
|         OllamaAPI ollamaAPI = new OllamaAPI(host); | ||||
|  | ||||
|         OllamaChatMessageRole customRole = ollamaAPI.addCustomRole("custom-role"); | ||||
|     } | ||||
| } | ||||
| ``` | ||||
|  | ||||
| #### List roles | ||||
|  | ||||
| ```java | ||||
| import io.github.ollama4j.OllamaAPI; | ||||
| import io.github.ollama4j.models.chat.OllamaChatMessageRole; | ||||
|  | ||||
| public class Main { | ||||
|  | ||||
|     public static void main(String[] args) { | ||||
|         String host = "http://localhost:11434/"; | ||||
|         OllamaAPI ollamaAPI = new OllamaAPI(host); | ||||
|  | ||||
|         List<OllamaChatMessageRole> roles = ollamaAPI.listRoles(); | ||||
|     } | ||||
| } | ||||
| ``` | ||||
|  | ||||
| #### Get role | ||||
|  | ||||
| ```java | ||||
| import io.github.ollama4j.OllamaAPI; | ||||
| import io.github.ollama4j.models.chat.OllamaChatMessageRole; | ||||
|  | ||||
| public class Main { | ||||
|  | ||||
|     public static void main(String[] args) { | ||||
|         String host = "http://localhost:11434/"; | ||||
|         OllamaAPI ollamaAPI = new OllamaAPI(host); | ||||
|  | ||||
|         List<OllamaChatMessageRole> roles = ollamaAPI.getRole("custom-role"); | ||||
|     } | ||||
| } | ||||
| ``` | ||||
| @@ -8,12 +8,85 @@ Generate embeddings from a model. | ||||
|  | ||||
| Parameters: | ||||
|  | ||||
| - `model`: name of model to generate embeddings from | ||||
| - `input`: text/s to generate embeddings for | ||||
|  | ||||
| ```java | ||||
| import io.github.ollama4j.OllamaAPI; | ||||
| import io.github.ollama4j.types.OllamaModelType; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbedRequestModel; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbedResponseModel; | ||||
| import java.util.Arrays; | ||||
| import java.util.Collections; | ||||
| import java.util.List; | ||||
|  | ||||
| public class Main { | ||||
|  | ||||
|     public static void main(String[] args) { | ||||
|  | ||||
|         String host = "http://localhost:11434/"; | ||||
|  | ||||
|         OllamaAPI ollamaAPI = new OllamaAPI(host); | ||||
|  | ||||
|         OllamaEmbedResponseModel embeddings = ollamaAPI.embed("all-minilm", Arrays.asList("Why is the sky blue?", "Why is the grass green?")); | ||||
|  | ||||
|         System.out.println(embeddings); | ||||
|     } | ||||
| } | ||||
| ``` | ||||
|  | ||||
| Or, using the `OllamaEmbedRequestModel`: | ||||
|  | ||||
| ```java | ||||
| import io.github.ollama4j.OllamaAPI; | ||||
| import io.github.ollama4j.types.OllamaModelType; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbedRequestModel; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbedResponseModel;import java.util.Arrays; | ||||
| import java.util.Collections; | ||||
| import java.util.List; | ||||
|  | ||||
| public class Main { | ||||
|  | ||||
|     public static void main(String[] args) { | ||||
|  | ||||
|         String host = "http://localhost:11434/"; | ||||
|  | ||||
|         OllamaAPI ollamaAPI = new OllamaAPI(host); | ||||
|  | ||||
|         OllamaEmbedResponseModel embeddings = ollamaAPI.embed(new OllamaEmbedRequestModel("all-minilm", Arrays.asList("Why is the sky blue?", "Why is the grass green?"))); | ||||
|  | ||||
|         System.out.println(embeddings); | ||||
|     } | ||||
| } | ||||
| ``` | ||||
|  | ||||
| You will get a response similar to: | ||||
|  | ||||
| ```json | ||||
| { | ||||
|     "model": "all-minilm", | ||||
|     "embeddings": [[-0.034674067, 0.030984823, 0.0067988685]], | ||||
|     "total_duration": 14173700, | ||||
|     "load_duration": 1198800, | ||||
|     "prompt_eval_count": 2 | ||||
| } | ||||
| ```` | ||||
|  | ||||
| :::note | ||||
|  | ||||
| This is a deprecated API | ||||
|  | ||||
| ::: | ||||
|  | ||||
| Parameters: | ||||
|  | ||||
| - `model`: name of model to generate embeddings from | ||||
| - `prompt`: text to generate embeddings for | ||||
|  | ||||
| ```java | ||||
| import io.github.ollama4j.OllamaAPI; | ||||
| import io.github.ollama4j.types.OllamaModelType; | ||||
|  | ||||
| import java.util.List; | ||||
|  | ||||
| public class Main { | ||||
| @@ -40,11 +113,6 @@ You will get a response similar to: | ||||
|     0.009260174818336964, | ||||
|     0.23178744316101074, | ||||
|     -0.2916173040866852, | ||||
|     -0.8924556970596313, | ||||
|     0.8785552978515625, | ||||
|     -0.34576427936553955, | ||||
|     0.5742510557174683, | ||||
|     -0.04222835972905159, | ||||
|     -0.137906014919281 | ||||
|     -0.8924556970596313 | ||||
| ] | ||||
| ``` | ||||
							
								
								
									
										4
									
								
								pom.xml
									
									
									
									
									
								
							
							
						
						
									
										4
									
								
								pom.xml
									
									
									
									
									
								
							| @@ -63,6 +63,10 @@ | ||||
|                 <groupId>org.apache.maven.plugins</groupId> | ||||
|                 <artifactId>maven-javadoc-plugin</artifactId> | ||||
|                 <version>3.5.0</version> | ||||
|                 <configuration> | ||||
|                     <!-- to disable the "missing" warnings. Remove the doclint to enable warnings--> | ||||
|                     <doclint>all,-missing</doclint> | ||||
|                 </configuration> | ||||
|                 <executions> | ||||
|                     <execution> | ||||
|                         <id>attach-javadocs</id> | ||||
|   | ||||
| @@ -1,14 +1,14 @@ | ||||
| package io.github.ollama4j; | ||||
|  | ||||
| import io.github.ollama4j.exceptions.OllamaBaseException; | ||||
| import io.github.ollama4j.exceptions.RoleNotFoundException; | ||||
| import io.github.ollama4j.exceptions.ToolInvocationException; | ||||
| import io.github.ollama4j.exceptions.ToolNotFoundException; | ||||
| import io.github.ollama4j.models.chat.OllamaChatMessage; | ||||
| import io.github.ollama4j.models.chat.OllamaChatRequest; | ||||
| import io.github.ollama4j.models.chat.OllamaChatRequestBuilder; | ||||
| import io.github.ollama4j.models.chat.OllamaChatResult; | ||||
| import io.github.ollama4j.models.chat.*; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbedRequestModel; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbeddingResponseModel; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbeddingsRequestModel; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbedResponseModel; | ||||
| import io.github.ollama4j.models.generate.OllamaGenerateRequest; | ||||
| import io.github.ollama4j.models.generate.OllamaStreamHandler; | ||||
| import io.github.ollama4j.models.ps.ModelsProcessResponse; | ||||
| @@ -36,7 +36,7 @@ import java.util.*; | ||||
| /** | ||||
|  * The base Ollama API class. | ||||
|  */ | ||||
| @SuppressWarnings("DuplicatedCode") | ||||
| @SuppressWarnings({"DuplicatedCode", "resource"}) | ||||
| public class OllamaAPI { | ||||
|  | ||||
|     private static final Logger logger = LoggerFactory.getLogger(OllamaAPI.class); | ||||
| @@ -97,12 +97,7 @@ public class OllamaAPI { | ||||
|         HttpClient httpClient = HttpClient.newHttpClient(); | ||||
|         HttpRequest httpRequest = null; | ||||
|         try { | ||||
|             httpRequest = | ||||
|                     getRequestBuilderDefault(new URI(url)) | ||||
|                             .header("Accept", "application/json") | ||||
|                             .header("Content-type", "application/json") | ||||
|                             .GET() | ||||
|                             .build(); | ||||
|             httpRequest = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-type", "application/json").GET().build(); | ||||
|         } catch (URISyntaxException e) { | ||||
|             throw new RuntimeException(e); | ||||
|         } | ||||
| @@ -121,19 +116,17 @@ public class OllamaAPI { | ||||
|     /** | ||||
|      * Provides a list of running models and details about each model currently loaded into memory. | ||||
|      * | ||||
|      * @return ModelsProcessResponse | ||||
|      * @return ModelsProcessResponse containing details about the running models | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      */ | ||||
|     public ModelsProcessResponse ps() throws IOException, InterruptedException, OllamaBaseException { | ||||
|         String url = this.host + "/api/ps"; | ||||
|         HttpClient httpClient = HttpClient.newHttpClient(); | ||||
|         HttpRequest httpRequest = null; | ||||
|         try { | ||||
|             httpRequest = | ||||
|                     getRequestBuilderDefault(new URI(url)) | ||||
|                             .header("Accept", "application/json") | ||||
|                             .header("Content-type", "application/json") | ||||
|                             .GET() | ||||
|                             .build(); | ||||
|             httpRequest = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-type", "application/json").GET().build(); | ||||
|         } catch (URISyntaxException e) { | ||||
|             throw new RuntimeException(e); | ||||
|         } | ||||
| @@ -142,36 +135,30 @@ public class OllamaAPI { | ||||
|         int statusCode = response.statusCode(); | ||||
|         String responseString = response.body(); | ||||
|         if (statusCode == 200) { | ||||
|             return Utils.getObjectMapper() | ||||
|                     .readValue(responseString, ModelsProcessResponse.class); | ||||
|             return Utils.getObjectMapper().readValue(responseString, ModelsProcessResponse.class); | ||||
|         } else { | ||||
|             throw new OllamaBaseException(statusCode + " - " + responseString); | ||||
|         } | ||||
|     } | ||||
|  | ||||
|     /** | ||||
|      * List available models from Ollama server. | ||||
|      * Lists available models from the Ollama server. | ||||
|      * | ||||
|      * @return the list | ||||
|      * @return a list of models available on the server | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws URISyntaxException   if the URI for the request is malformed | ||||
|      */ | ||||
|     public List<Model> listModels() | ||||
|             throws OllamaBaseException, IOException, InterruptedException, URISyntaxException { | ||||
|     public List<Model> listModels() throws OllamaBaseException, IOException, InterruptedException, URISyntaxException { | ||||
|         String url = this.host + "/api/tags"; | ||||
|         HttpClient httpClient = HttpClient.newHttpClient(); | ||||
|         HttpRequest httpRequest = | ||||
|                 getRequestBuilderDefault(new URI(url)) | ||||
|                         .header("Accept", "application/json") | ||||
|                         .header("Content-type", "application/json") | ||||
|                         .GET() | ||||
|                         .build(); | ||||
|         HttpResponse<String> response = | ||||
|                 httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString()); | ||||
|         HttpRequest httpRequest = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-type", "application/json").GET().build(); | ||||
|         HttpResponse<String> response = httpClient.send(httpRequest, HttpResponse.BodyHandlers.ofString()); | ||||
|         int statusCode = response.statusCode(); | ||||
|         String responseString = response.body(); | ||||
|         if (statusCode == 200) { | ||||
|             return Utils.getObjectMapper() | ||||
|                     .readValue(responseString, ListModelsResponse.class) | ||||
|                     .getModels(); | ||||
|             return Utils.getObjectMapper().readValue(responseString, ListModelsResponse.class).getModels(); | ||||
|         } else { | ||||
|             throw new OllamaBaseException(statusCode + " - " + responseString); | ||||
|         } | ||||
| @@ -182,29 +169,24 @@ public class OllamaAPI { | ||||
|      * href="https://ollama.ai/library">available models</a>. | ||||
|      * | ||||
|      * @param modelName the name of the model | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws URISyntaxException   if the URI for the request is malformed | ||||
|      */ | ||||
|     public void pullModel(String modelName) | ||||
|             throws OllamaBaseException, IOException, URISyntaxException, InterruptedException { | ||||
|     public void pullModel(String modelName) throws OllamaBaseException, IOException, URISyntaxException, InterruptedException { | ||||
|         String url = this.host + "/api/pull"; | ||||
|         String jsonData = new ModelRequest(modelName).toString(); | ||||
|         HttpRequest request = | ||||
|                 getRequestBuilderDefault(new URI(url)) | ||||
|                         .POST(HttpRequest.BodyPublishers.ofString(jsonData)) | ||||
|                         .header("Accept", "application/json") | ||||
|                         .header("Content-type", "application/json") | ||||
|                         .build(); | ||||
|         HttpRequest request = getRequestBuilderDefault(new URI(url)).POST(HttpRequest.BodyPublishers.ofString(jsonData)).header("Accept", "application/json").header("Content-type", "application/json").build(); | ||||
|         HttpClient client = HttpClient.newHttpClient(); | ||||
|         HttpResponse<InputStream> response = | ||||
|                 client.send(request, HttpResponse.BodyHandlers.ofInputStream()); | ||||
|         HttpResponse<InputStream> response = client.send(request, HttpResponse.BodyHandlers.ofInputStream()); | ||||
|         int statusCode = response.statusCode(); | ||||
|         InputStream responseBodyStream = response.body(); | ||||
|         String responseString = ""; | ||||
|         try (BufferedReader reader = | ||||
|                      new BufferedReader(new InputStreamReader(responseBodyStream, StandardCharsets.UTF_8))) { | ||||
|         try (BufferedReader reader = new BufferedReader(new InputStreamReader(responseBodyStream, StandardCharsets.UTF_8))) { | ||||
|             String line; | ||||
|             while ((line = reader.readLine()) != null) { | ||||
|                 ModelPullResponse modelPullResponse = | ||||
|                         Utils.getObjectMapper().readValue(line, ModelPullResponse.class); | ||||
|                 ModelPullResponse modelPullResponse = Utils.getObjectMapper().readValue(line, ModelPullResponse.class); | ||||
|                 if (verbose) { | ||||
|                     logger.info(modelPullResponse.getStatus()); | ||||
|                 } | ||||
| @@ -220,17 +202,15 @@ public class OllamaAPI { | ||||
|      * | ||||
|      * @param modelName the model | ||||
|      * @return the model details | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws URISyntaxException   if the URI for the request is malformed | ||||
|      */ | ||||
|     public ModelDetail getModelDetails(String modelName) | ||||
|             throws IOException, OllamaBaseException, InterruptedException, URISyntaxException { | ||||
|     public ModelDetail getModelDetails(String modelName) throws IOException, OllamaBaseException, InterruptedException, URISyntaxException { | ||||
|         String url = this.host + "/api/show"; | ||||
|         String jsonData = new ModelRequest(modelName).toString(); | ||||
|         HttpRequest request = | ||||
|                 getRequestBuilderDefault(new URI(url)) | ||||
|                         .header("Accept", "application/json") | ||||
|                         .header("Content-type", "application/json") | ||||
|                         .POST(HttpRequest.BodyPublishers.ofString(jsonData)) | ||||
|                         .build(); | ||||
|         HttpRequest request = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-type", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData)).build(); | ||||
|         HttpClient client = HttpClient.newHttpClient(); | ||||
|         HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString()); | ||||
|         int statusCode = response.statusCode(); | ||||
| @@ -248,17 +228,15 @@ public class OllamaAPI { | ||||
|      * | ||||
|      * @param modelName     the name of the custom model to be created. | ||||
|      * @param modelFilePath the path to model file that exists on the Ollama server. | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws URISyntaxException   if the URI for the request is malformed | ||||
|      */ | ||||
|     public void createModelWithFilePath(String modelName, String modelFilePath) | ||||
|             throws IOException, InterruptedException, OllamaBaseException, URISyntaxException { | ||||
|     public void createModelWithFilePath(String modelName, String modelFilePath) throws IOException, InterruptedException, OllamaBaseException, URISyntaxException { | ||||
|         String url = this.host + "/api/create"; | ||||
|         String jsonData = new CustomModelFilePathRequest(modelName, modelFilePath).toString(); | ||||
|         HttpRequest request = | ||||
|                 getRequestBuilderDefault(new URI(url)) | ||||
|                         .header("Accept", "application/json") | ||||
|                         .header("Content-Type", "application/json") | ||||
|                         .POST(HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)) | ||||
|                         .build(); | ||||
|         HttpRequest request = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)).build(); | ||||
|         HttpClient client = HttpClient.newHttpClient(); | ||||
|         HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString()); | ||||
|         int statusCode = response.statusCode(); | ||||
| @@ -282,17 +260,15 @@ public class OllamaAPI { | ||||
|      * | ||||
|      * @param modelName         the name of the custom model to be created. | ||||
|      * @param modelFileContents the path to model file that exists on the Ollama server. | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws URISyntaxException   if the URI for the request is malformed | ||||
|      */ | ||||
|     public void createModelWithModelFileContents(String modelName, String modelFileContents) | ||||
|             throws IOException, InterruptedException, OllamaBaseException, URISyntaxException { | ||||
|     public void createModelWithModelFileContents(String modelName, String modelFileContents) throws IOException, InterruptedException, OllamaBaseException, URISyntaxException { | ||||
|         String url = this.host + "/api/create"; | ||||
|         String jsonData = new CustomModelFileContentsRequest(modelName, modelFileContents).toString(); | ||||
|         HttpRequest request = | ||||
|                 getRequestBuilderDefault(new URI(url)) | ||||
|                         .header("Accept", "application/json") | ||||
|                         .header("Content-Type", "application/json") | ||||
|                         .POST(HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)) | ||||
|                         .build(); | ||||
|         HttpRequest request = getRequestBuilderDefault(new URI(url)).header("Accept", "application/json").header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)).build(); | ||||
|         HttpClient client = HttpClient.newHttpClient(); | ||||
|         HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString()); | ||||
|         int statusCode = response.statusCode(); | ||||
| @@ -313,17 +289,15 @@ public class OllamaAPI { | ||||
|      * | ||||
|      * @param modelName          the name of the model to be deleted. | ||||
|      * @param ignoreIfNotPresent ignore errors if the specified model is not present on Ollama server. | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws URISyntaxException   if the URI for the request is malformed | ||||
|      */ | ||||
|     public void deleteModel(String modelName, boolean ignoreIfNotPresent) | ||||
|             throws IOException, InterruptedException, OllamaBaseException, URISyntaxException { | ||||
|     public void deleteModel(String modelName, boolean ignoreIfNotPresent) throws IOException, InterruptedException, OllamaBaseException, URISyntaxException { | ||||
|         String url = this.host + "/api/delete"; | ||||
|         String jsonData = new ModelRequest(modelName).toString(); | ||||
|         HttpRequest request = | ||||
|                 getRequestBuilderDefault(new URI(url)) | ||||
|                         .method("DELETE", HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)) | ||||
|                         .header("Accept", "application/json") | ||||
|                         .header("Content-type", "application/json") | ||||
|                         .build(); | ||||
|         HttpRequest request = getRequestBuilderDefault(new URI(url)).method("DELETE", HttpRequest.BodyPublishers.ofString(jsonData, StandardCharsets.UTF_8)).header("Accept", "application/json").header("Content-type", "application/json").build(); | ||||
|         HttpClient client = HttpClient.newHttpClient(); | ||||
|         HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString()); | ||||
|         int statusCode = response.statusCode(); | ||||
| @@ -342,9 +316,13 @@ public class OllamaAPI { | ||||
|      * @param model  name of model to generate embeddings from | ||||
|      * @param prompt text to generate embeddings for | ||||
|      * @return embeddings | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @deprecated Use {@link #embed(String, List)} instead. | ||||
|      */ | ||||
|     public List<Double> generateEmbeddings(String model, String prompt) | ||||
|             throws IOException, InterruptedException, OllamaBaseException { | ||||
|     @Deprecated | ||||
|     public List<Double> generateEmbeddings(String model, String prompt) throws IOException, InterruptedException, OllamaBaseException { | ||||
|         return generateEmbeddings(new OllamaEmbeddingsRequestModel(model, prompt)); | ||||
|     } | ||||
|  | ||||
| @@ -353,28 +331,69 @@ public class OllamaAPI { | ||||
|      * | ||||
|      * @param modelRequest request for '/api/embeddings' endpoint | ||||
|      * @return embeddings | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @deprecated Use {@link #embed(OllamaEmbedRequestModel)} instead. | ||||
|      */ | ||||
|     @Deprecated | ||||
|     public List<Double> generateEmbeddings(OllamaEmbeddingsRequestModel modelRequest) throws IOException, InterruptedException, OllamaBaseException { | ||||
|         URI uri = URI.create(this.host + "/api/embeddings"); | ||||
|         String jsonData = modelRequest.toString(); | ||||
|         HttpClient httpClient = HttpClient.newHttpClient(); | ||||
|         HttpRequest.Builder requestBuilder = | ||||
|                 getRequestBuilderDefault(uri) | ||||
|                         .header("Accept", "application/json") | ||||
|                         .POST(HttpRequest.BodyPublishers.ofString(jsonData)); | ||||
|         HttpRequest.Builder requestBuilder = getRequestBuilderDefault(uri).header("Accept", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData)); | ||||
|         HttpRequest request = requestBuilder.build(); | ||||
|         HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString()); | ||||
|         int statusCode = response.statusCode(); | ||||
|         String responseBody = response.body(); | ||||
|         if (statusCode == 200) { | ||||
|             OllamaEmbeddingResponseModel embeddingResponse = | ||||
|                     Utils.getObjectMapper().readValue(responseBody, OllamaEmbeddingResponseModel.class); | ||||
|             OllamaEmbeddingResponseModel embeddingResponse = Utils.getObjectMapper().readValue(responseBody, OllamaEmbeddingResponseModel.class); | ||||
|             return embeddingResponse.getEmbedding(); | ||||
|         } else { | ||||
|             throw new OllamaBaseException(statusCode + " - " + responseBody); | ||||
|         } | ||||
|     } | ||||
|  | ||||
|     /** | ||||
|      * Generate embeddings for a given text from a model | ||||
|      * | ||||
|      * @param model  name of model to generate embeddings from | ||||
|      * @param inputs text/s to generate embeddings for | ||||
|      * @return embeddings | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaEmbedResponseModel embed(String model, List<String> inputs) throws IOException, InterruptedException, OllamaBaseException { | ||||
|         return embed(new OllamaEmbedRequestModel(model, inputs)); | ||||
|     } | ||||
|  | ||||
|     /** | ||||
|      * Generate embeddings using a {@link OllamaEmbedRequestModel}. | ||||
|      * | ||||
|      * @param modelRequest request for '/api/embed' endpoint | ||||
|      * @return embeddings | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaEmbedResponseModel embed(OllamaEmbedRequestModel modelRequest) throws IOException, InterruptedException, OllamaBaseException { | ||||
|         URI uri = URI.create(this.host + "/api/embed"); | ||||
|         String jsonData = Utils.getObjectMapper().writeValueAsString(modelRequest); | ||||
|         HttpClient httpClient = HttpClient.newHttpClient(); | ||||
|  | ||||
|         HttpRequest request = HttpRequest.newBuilder(uri).header("Accept", "application/json").POST(HttpRequest.BodyPublishers.ofString(jsonData)).build(); | ||||
|  | ||||
|         HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString()); | ||||
|         int statusCode = response.statusCode(); | ||||
|         String responseBody = response.body(); | ||||
|  | ||||
|         if (statusCode == 200) { | ||||
|             return Utils.getObjectMapper().readValue(responseBody, OllamaEmbedResponseModel.class); | ||||
|         } else { | ||||
|             throw new OllamaBaseException(statusCode + " - " + responseBody); | ||||
|         } | ||||
|     } | ||||
|  | ||||
|     /** | ||||
|      * Generate response for a question to a model running on Ollama server. This is a sync/blocking | ||||
| @@ -387,9 +406,11 @@ public class OllamaAPI { | ||||
|      *                      details on the options</a> | ||||
|      * @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false. | ||||
|      * @return OllamaResult that includes response text and time taken for response | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaResult generate(String model, String prompt, boolean raw, Options options, OllamaStreamHandler streamHandler) | ||||
|             throws OllamaBaseException, IOException, InterruptedException { | ||||
|     public OllamaResult generate(String model, String prompt, boolean raw, Options options, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException { | ||||
|         OllamaGenerateRequest ollamaRequestModel = new OllamaGenerateRequest(model, prompt); | ||||
|         ollamaRequestModel.setRaw(raw); | ||||
|         ollamaRequestModel.setOptions(options.getOptionsMap()); | ||||
| @@ -406,13 +427,14 @@ public class OllamaAPI { | ||||
|      * @param raw     In some cases, you may wish to bypass the templating system and provide a full prompt. In this case, you can use the raw parameter to disable templating. Also note that raw mode will not return a context. | ||||
|      * @param options Additional options or configurations to use when generating the response. | ||||
|      * @return {@link OllamaResult} | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaResult generate(String model, String prompt, boolean raw, Options options) | ||||
|             throws OllamaBaseException, IOException, InterruptedException { | ||||
|     public OllamaResult generate(String model, String prompt, boolean raw, Options options) throws OllamaBaseException, IOException, InterruptedException { | ||||
|         return generate(model, prompt, raw, options, null); | ||||
|     } | ||||
|  | ||||
|  | ||||
|     /** | ||||
|      * Generates response using the specified AI model and prompt (in blocking mode), and then invokes a set of tools | ||||
|      * on the generated response. | ||||
| @@ -421,13 +443,11 @@ public class OllamaAPI { | ||||
|      * @param prompt  The input text or prompt to provide to the AI model. | ||||
|      * @param options Additional options or configurations to use when generating the response. | ||||
|      * @return {@link OllamaToolsResult} An OllamaToolsResult object containing the response from the AI model and the results of invoking the tools on that output. | ||||
|      * @throws OllamaBaseException  If there is an error related to the Ollama API or service. | ||||
|      * @throws IOException          If there is an error related to input/output operations. | ||||
|      * @throws InterruptedException If the method is interrupted while waiting for the AI model | ||||
|      *                              to generate the response or for the tools to be invoked. | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaToolsResult generateWithTools(String model, String prompt, Options options) | ||||
|             throws OllamaBaseException, IOException, InterruptedException, ToolInvocationException { | ||||
|     public OllamaToolsResult generateWithTools(String model, String prompt, Options options) throws OllamaBaseException, IOException, InterruptedException, ToolInvocationException { | ||||
|         boolean raw = true; | ||||
|         OllamaToolsResult toolResult = new OllamaToolsResult(); | ||||
|         Map<ToolFunctionCallSpec, Object> toolResults = new HashMap<>(); | ||||
| @@ -448,7 +468,6 @@ public class OllamaAPI { | ||||
|         return toolResult; | ||||
|     } | ||||
|  | ||||
|  | ||||
|     /** | ||||
|      * Generate response for a question to a model running on Ollama server and get a callback handle | ||||
|      * that can be used to check for status and get the response from the model later. This would be | ||||
| @@ -462,9 +481,7 @@ public class OllamaAPI { | ||||
|         OllamaGenerateRequest ollamaRequestModel = new OllamaGenerateRequest(model, prompt); | ||||
|         ollamaRequestModel.setRaw(raw); | ||||
|         URI uri = URI.create(this.host + "/api/generate"); | ||||
|         OllamaAsyncResultStreamer ollamaAsyncResultStreamer = | ||||
|                 new OllamaAsyncResultStreamer( | ||||
|                         getRequestBuilderDefault(uri), ollamaRequestModel, requestTimeoutSeconds); | ||||
|         OllamaAsyncResultStreamer ollamaAsyncResultStreamer = new OllamaAsyncResultStreamer(getRequestBuilderDefault(uri), ollamaRequestModel, requestTimeoutSeconds); | ||||
|         ollamaAsyncResultStreamer.start(); | ||||
|         return ollamaAsyncResultStreamer; | ||||
|     } | ||||
| @@ -481,10 +498,11 @@ public class OllamaAPI { | ||||
|      *                      details on the options</a> | ||||
|      * @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false. | ||||
|      * @return OllamaResult that includes response text and time taken for response | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaResult generateWithImageFiles( | ||||
|             String model, String prompt, List<File> imageFiles, Options options, OllamaStreamHandler streamHandler) | ||||
|             throws OllamaBaseException, IOException, InterruptedException { | ||||
|     public OllamaResult generateWithImageFiles(String model, String prompt, List<File> imageFiles, Options options, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException { | ||||
|         List<String> images = new ArrayList<>(); | ||||
|         for (File imageFile : imageFiles) { | ||||
|             images.add(encodeFileToBase64(imageFile)); | ||||
| @@ -498,10 +516,12 @@ public class OllamaAPI { | ||||
|      * Convenience method to call Ollama API without streaming responses. | ||||
|      * <p> | ||||
|      * Uses {@link #generateWithImageFiles(String, String, List, Options, OllamaStreamHandler)} | ||||
|      * | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaResult generateWithImageFiles( | ||||
|             String model, String prompt, List<File> imageFiles, Options options) | ||||
|             throws OllamaBaseException, IOException, InterruptedException { | ||||
|     public OllamaResult generateWithImageFiles(String model, String prompt, List<File> imageFiles, Options options) throws OllamaBaseException, IOException, InterruptedException { | ||||
|         return generateWithImageFiles(model, prompt, imageFiles, options, null); | ||||
|     } | ||||
|  | ||||
| @@ -517,10 +537,12 @@ public class OllamaAPI { | ||||
|      *                      details on the options</a> | ||||
|      * @param streamHandler optional callback consumer that will be applied every time a streamed response is received. If not set, the stream parameter of the request is set to false. | ||||
|      * @return OllamaResult that includes response text and time taken for response | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws URISyntaxException   if the URI for the request is malformed | ||||
|      */ | ||||
|     public OllamaResult generateWithImageURLs( | ||||
|             String model, String prompt, List<String> imageURLs, Options options, OllamaStreamHandler streamHandler) | ||||
|             throws OllamaBaseException, IOException, InterruptedException, URISyntaxException { | ||||
|     public OllamaResult generateWithImageURLs(String model, String prompt, List<String> imageURLs, Options options, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException, URISyntaxException { | ||||
|         List<String> images = new ArrayList<>(); | ||||
|         for (String imageURL : imageURLs) { | ||||
|             images.add(encodeByteArrayToBase64(Utils.loadImageBytesFromUrl(imageURL))); | ||||
| @@ -534,14 +556,16 @@ public class OllamaAPI { | ||||
|      * Convenience method to call Ollama API without streaming responses. | ||||
|      * <p> | ||||
|      * Uses {@link #generateWithImageURLs(String, String, List, Options, OllamaStreamHandler)} | ||||
|      * | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      * @throws URISyntaxException   if the URI for the request is malformed | ||||
|      */ | ||||
|     public OllamaResult generateWithImageURLs(String model, String prompt, List<String> imageURLs, | ||||
|                                               Options options) | ||||
|             throws OllamaBaseException, IOException, InterruptedException, URISyntaxException { | ||||
|     public OllamaResult generateWithImageURLs(String model, String prompt, List<String> imageURLs, Options options) throws OllamaBaseException, IOException, InterruptedException, URISyntaxException { | ||||
|         return generateWithImageURLs(model, prompt, imageURLs, options, null); | ||||
|     } | ||||
|  | ||||
|  | ||||
|     /** | ||||
|      * Ask a question to a model based on a given message stack (i.e. a chat history). Creates a synchronous call to the api | ||||
|      * 'api/chat'. | ||||
| @@ -552,6 +576,9 @@ public class OllamaAPI { | ||||
|      * @throws OllamaBaseException  any response code than 200 has been returned | ||||
|      * @throws IOException          in case the responseStream can not be read | ||||
|      * @throws InterruptedException in case the server is not reachable or network issues happen | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaChatResult chat(String model, List<OllamaChatMessage> messages) throws OllamaBaseException, IOException, InterruptedException { | ||||
|         OllamaChatRequestBuilder builder = OllamaChatRequestBuilder.getInstance(model); | ||||
| @@ -568,6 +595,9 @@ public class OllamaAPI { | ||||
|      * @throws OllamaBaseException  any response code than 200 has been returned | ||||
|      * @throws IOException          in case the responseStream can not be read | ||||
|      * @throws InterruptedException in case the server is not reachable or network issues happen | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaChatResult chat(OllamaChatRequest request) throws OllamaBaseException, IOException, InterruptedException { | ||||
|         return chat(request, null); | ||||
| @@ -584,6 +614,9 @@ public class OllamaAPI { | ||||
|      * @throws OllamaBaseException  any response code than 200 has been returned | ||||
|      * @throws IOException          in case the responseStream can not be read | ||||
|      * @throws InterruptedException in case the server is not reachable or network issues happen | ||||
|      * @throws OllamaBaseException  if the response indicates an error status | ||||
|      * @throws IOException          if an I/O error occurs during the HTTP request | ||||
|      * @throws InterruptedException if the operation is interrupted | ||||
|      */ | ||||
|     public OllamaChatResult chat(OllamaChatRequest request, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException { | ||||
|         OllamaChatEndpointCaller requestCaller = new OllamaChatEndpointCaller(host, basicAuth, requestTimeoutSeconds, verbose); | ||||
| @@ -601,6 +634,37 @@ public class OllamaAPI { | ||||
|         toolRegistry.addFunction(toolSpecification.getFunctionName(), toolSpecification.getToolDefinition()); | ||||
|     } | ||||
|  | ||||
|     /** | ||||
|      * Adds a custom role. | ||||
|      * | ||||
|      * @param roleName the name of the custom role to be added | ||||
|      * @return the newly created OllamaChatMessageRole | ||||
|      */ | ||||
|     public OllamaChatMessageRole addCustomRole(String roleName) { | ||||
|         return OllamaChatMessageRole.newCustomRole(roleName); | ||||
|     } | ||||
|  | ||||
|     /** | ||||
|      * Lists all available roles. | ||||
|      * | ||||
|      * @return a list of available OllamaChatMessageRole objects | ||||
|      */ | ||||
|     public List<OllamaChatMessageRole> listRoles() { | ||||
|         return OllamaChatMessageRole.getRoles(); | ||||
|     } | ||||
|  | ||||
|     /** | ||||
|      * Retrieves a specific role by name. | ||||
|      * | ||||
|      * @param roleName the name of the role to retrieve | ||||
|      * @return the OllamaChatMessageRole associated with the given name | ||||
|      * @throws RoleNotFoundException if the role with the specified name does not exist | ||||
|      */ | ||||
|     public OllamaChatMessageRole getRole(String roleName) throws RoleNotFoundException { | ||||
|         return OllamaChatMessageRole.getRole(roleName); | ||||
|     } | ||||
|  | ||||
|  | ||||
|     // technical private methods // | ||||
|  | ||||
|     private static String encodeFileToBase64(File file) throws IOException { | ||||
| @@ -611,11 +675,8 @@ public class OllamaAPI { | ||||
|         return Base64.getEncoder().encodeToString(bytes); | ||||
|     } | ||||
|  | ||||
|     private OllamaResult generateSyncForOllamaRequestModel( | ||||
|             OllamaGenerateRequest ollamaRequestModel, OllamaStreamHandler streamHandler) | ||||
|             throws OllamaBaseException, IOException, InterruptedException { | ||||
|         OllamaGenerateEndpointCaller requestCaller = | ||||
|                 new OllamaGenerateEndpointCaller(host, basicAuth, requestTimeoutSeconds, verbose); | ||||
|     private OllamaResult generateSyncForOllamaRequestModel(OllamaGenerateRequest ollamaRequestModel, OllamaStreamHandler streamHandler) throws OllamaBaseException, IOException, InterruptedException { | ||||
|         OllamaGenerateEndpointCaller requestCaller = new OllamaGenerateEndpointCaller(host, basicAuth, requestTimeoutSeconds, verbose); | ||||
|         OllamaResult result; | ||||
|         if (streamHandler != null) { | ||||
|             ollamaRequestModel.setStream(true); | ||||
| @@ -633,10 +694,7 @@ public class OllamaAPI { | ||||
|      * @return HttpRequest.Builder | ||||
|      */ | ||||
|     private HttpRequest.Builder getRequestBuilderDefault(URI uri) { | ||||
|         HttpRequest.Builder requestBuilder = | ||||
|                 HttpRequest.newBuilder(uri) | ||||
|                         .header("Content-Type", "application/json") | ||||
|                         .timeout(Duration.ofSeconds(requestTimeoutSeconds)); | ||||
|         HttpRequest.Builder requestBuilder = HttpRequest.newBuilder(uri).header("Content-Type", "application/json").timeout(Duration.ofSeconds(requestTimeoutSeconds)); | ||||
|         if (isBasicAuthCredentialsSet()) { | ||||
|             requestBuilder.header("Authorization", getBasicAuthHeaderValue()); | ||||
|         } | ||||
| @@ -662,7 +720,6 @@ public class OllamaAPI { | ||||
|         return basicAuth != null; | ||||
|     } | ||||
|  | ||||
|  | ||||
|     private Object invokeTool(ToolFunctionCallSpec toolFunctionCallSpec) throws ToolInvocationException { | ||||
|         try { | ||||
|             String methodName = toolFunctionCallSpec.getName(); | ||||
|   | ||||
| @@ -0,0 +1,8 @@ | ||||
| package io.github.ollama4j.exceptions; | ||||
|  | ||||
| public class RoleNotFoundException extends Exception { | ||||
|  | ||||
|     public RoleNotFoundException(String s) { | ||||
|         super(s); | ||||
|     } | ||||
| } | ||||
| @@ -1,19 +1,53 @@ | ||||
| package io.github.ollama4j.models.chat; | ||||
|  | ||||
| import com.fasterxml.jackson.annotation.JsonValue; | ||||
| import io.github.ollama4j.exceptions.RoleNotFoundException; | ||||
| import lombok.Getter; | ||||
|  | ||||
| import java.util.ArrayList; | ||||
| import java.util.List; | ||||
|  | ||||
| /** | ||||
|  * Defines the possible Chat Message roles. | ||||
|  */ | ||||
| public enum OllamaChatMessageRole { | ||||
|     SYSTEM("system"), | ||||
|     USER("user"), | ||||
|     ASSISTANT("assistant"); | ||||
| @Getter | ||||
| public class OllamaChatMessageRole { | ||||
|     private static final List<OllamaChatMessageRole> ROLES = new ArrayList<>(); | ||||
|  | ||||
|     public static final OllamaChatMessageRole SYSTEM = new OllamaChatMessageRole("system"); | ||||
|     public static final OllamaChatMessageRole USER = new OllamaChatMessageRole("user"); | ||||
|     public static final OllamaChatMessageRole ASSISTANT = new OllamaChatMessageRole("assistant"); | ||||
|     public static final OllamaChatMessageRole TOOL = new OllamaChatMessageRole("tool"); | ||||
|  | ||||
|     @JsonValue | ||||
|     private String roleName; | ||||
|     private final String roleName; | ||||
|  | ||||
|     private OllamaChatMessageRole(String roleName){ | ||||
|     private OllamaChatMessageRole(String roleName) { | ||||
|         this.roleName = roleName; | ||||
|         ROLES.add(this); | ||||
|     } | ||||
|  | ||||
|     public static OllamaChatMessageRole newCustomRole(String roleName) { | ||||
|         OllamaChatMessageRole customRole = new OllamaChatMessageRole(roleName); | ||||
|         ROLES.add(customRole); | ||||
|         return customRole; | ||||
|     } | ||||
|  | ||||
|     public static List<OllamaChatMessageRole> getRoles() { | ||||
|         return new ArrayList<>(ROLES); | ||||
|     } | ||||
|  | ||||
|     public static OllamaChatMessageRole getRole(String roleName) throws RoleNotFoundException { | ||||
|         for (OllamaChatMessageRole role : ROLES) { | ||||
|             if (role.roleName.equals(roleName)) { | ||||
|                 return role; | ||||
|             } | ||||
|         } | ||||
|         throw new RoleNotFoundException("Invalid role name: " + roleName); | ||||
|     } | ||||
|  | ||||
|     @Override | ||||
|     public String toString() { | ||||
|         return roleName; | ||||
|     } | ||||
| } | ||||
|   | ||||
| @@ -8,12 +8,11 @@ import io.github.ollama4j.models.response.OllamaResult; | ||||
|  * Specific chat-API result that contains the chat history sent to the model and appends the answer as {@link OllamaChatResult} given by the | ||||
|  * {@link OllamaChatMessageRole#ASSISTANT} role. | ||||
|  */ | ||||
| public class OllamaChatResult extends OllamaResult{ | ||||
| public class OllamaChatResult extends OllamaResult { | ||||
|  | ||||
|     private List<OllamaChatMessage> chatHistory; | ||||
|  | ||||
|     public OllamaChatResult(String response, long responseTime, int httpStatusCode, | ||||
|             List<OllamaChatMessage> chatHistory) { | ||||
|     public OllamaChatResult(String response, long responseTime, int httpStatusCode, List<OllamaChatMessage> chatHistory) { | ||||
|         super(response, responseTime, httpStatusCode); | ||||
|         this.chatHistory = chatHistory; | ||||
|         appendAnswerToChatHistory(response); | ||||
| @@ -21,12 +20,10 @@ public class OllamaChatResult extends OllamaResult{ | ||||
|  | ||||
|     public List<OllamaChatMessage> getChatHistory() { | ||||
|         return chatHistory; | ||||
|     }  | ||||
|     } | ||||
|  | ||||
|     private void appendAnswerToChatHistory(String answer){ | ||||
|     private void appendAnswerToChatHistory(String answer) { | ||||
|         OllamaChatMessage assistantMessage = new OllamaChatMessage(OllamaChatMessageRole.ASSISTANT, answer); | ||||
|         this.chatHistory.add(assistantMessage); | ||||
|     } | ||||
|      | ||||
|      | ||||
| } | ||||
|   | ||||
| @@ -0,0 +1,41 @@ | ||||
| package io.github.ollama4j.models.embeddings; | ||||
|  | ||||
| import com.fasterxml.jackson.annotation.JsonProperty; | ||||
| import com.fasterxml.jackson.core.JsonProcessingException; | ||||
| import lombok.Data; | ||||
| import lombok.NoArgsConstructor; | ||||
| import lombok.NonNull; | ||||
| import lombok.RequiredArgsConstructor; | ||||
|  | ||||
| import java.util.List; | ||||
| import java.util.Map; | ||||
|  | ||||
| import static io.github.ollama4j.utils.Utils.getObjectMapper; | ||||
|  | ||||
| @Data | ||||
| @RequiredArgsConstructor | ||||
| @NoArgsConstructor | ||||
| public class OllamaEmbedRequestModel { | ||||
|     @NonNull | ||||
|     private String model; | ||||
|  | ||||
|     @NonNull | ||||
|     private List<String> input; | ||||
|  | ||||
|     private Map<String, Object> options; | ||||
|  | ||||
|     @JsonProperty(value = "keep_alive") | ||||
|     private String keepAlive; | ||||
|  | ||||
|     @JsonProperty(value = "truncate") | ||||
|     private Boolean truncate = true; | ||||
|  | ||||
|     @Override | ||||
|     public String toString() { | ||||
|         try { | ||||
|             return getObjectMapper().writerWithDefaultPrettyPrinter().writeValueAsString(this); | ||||
|         } catch (JsonProcessingException e) { | ||||
|             throw new RuntimeException(e); | ||||
|         } | ||||
|     } | ||||
| } | ||||
| @@ -0,0 +1,25 @@ | ||||
| package io.github.ollama4j.models.embeddings; | ||||
|  | ||||
| import com.fasterxml.jackson.annotation.JsonProperty; | ||||
| import lombok.Data; | ||||
|  | ||||
| import java.util.List; | ||||
|  | ||||
| @SuppressWarnings("unused") | ||||
| @Data | ||||
| public class OllamaEmbedResponseModel { | ||||
|     @JsonProperty("model") | ||||
|     private String model; | ||||
|  | ||||
|     @JsonProperty("embeddings") | ||||
|     private List<List<Double>> embeddings; | ||||
|  | ||||
|     @JsonProperty("total_duration") | ||||
|     private long totalDuration; | ||||
|  | ||||
|     @JsonProperty("load_duration") | ||||
|     private long loadDuration; | ||||
|  | ||||
|     @JsonProperty("prompt_eval_count") | ||||
|     private int promptEvalCount; | ||||
| } | ||||
| @@ -10,12 +10,9 @@ package io.github.ollama4j.types; | ||||
| public class OllamaModelType { | ||||
|     public static final String GEMMA = "gemma"; | ||||
|     public static final String GEMMA2 = "gemma2"; | ||||
|  | ||||
|  | ||||
|     public static final String LLAMA2 = "llama2"; | ||||
|     public static final String LLAMA3 = "llama3"; | ||||
|     public static final String LLAMA3_1 = "llama3.1"; | ||||
|  | ||||
|     public static final String MISTRAL = "mistral"; | ||||
|     public static final String MIXTRAL = "mixtral"; | ||||
|     public static final String LLAVA = "llava"; | ||||
| @@ -35,7 +32,6 @@ public class OllamaModelType { | ||||
|     public static final String ZEPHYR = "zephyr"; | ||||
|     public static final String OPENHERMES = "openhermes"; | ||||
|     public static final String QWEN = "qwen"; | ||||
|  | ||||
|     public static final String QWEN2 = "qwen2"; | ||||
|     public static final String WIZARDCODER = "wizardcoder"; | ||||
|     public static final String LLAMA2_CHINESE = "llama2-chinese"; | ||||
|   | ||||
| @@ -1,5 +1,6 @@ | ||||
| package io.github.ollama4j.utils; | ||||
|  | ||||
| import java.io.IOException; | ||||
| import java.util.HashMap; | ||||
|  | ||||
| /** Builder class for creating options for Ollama model. */ | ||||
| @@ -207,6 +208,34 @@ public class OptionsBuilder { | ||||
|     return this; | ||||
|   } | ||||
|  | ||||
|   /** | ||||
|    * Alternative to the top_p, and aims to ensure a balance of qualityand variety. The parameter p | ||||
|    * represents the minimum probability for a token to be considered, relative to the probability | ||||
|    * of the most likely token. For example, with p=0.05 and the most likely token having a | ||||
|    * probability of 0.9, logits with a value less than 0.045 are filtered out. (Default: 0.0) | ||||
|    */ | ||||
|   public OptionsBuilder setMinP(float value) { | ||||
|     options.getOptionsMap().put("min_p", value); | ||||
|     return this; | ||||
|   } | ||||
|  | ||||
|   /** | ||||
|    * Allows passing an option not formally supported by the library | ||||
|    * @param name The option name for the parameter. | ||||
|    * @param value The value for the "{name}" parameter. | ||||
|    * @return The updated OptionsBuilder. | ||||
|    * @throws IllegalArgumentException if parameter has an unsupported type | ||||
|    */ | ||||
|   public OptionsBuilder setCustomOption(String name, Object value) throws IllegalArgumentException { | ||||
|     if (!(value instanceof Integer || value instanceof Float || value instanceof String)) { | ||||
|       throw new IllegalArgumentException("Invalid type for parameter. Allowed types are: Integer, Float, or String."); | ||||
|     } | ||||
|     options.getOptionsMap().put(name, value); | ||||
|     return this; | ||||
|   } | ||||
|  | ||||
|  | ||||
|  | ||||
|   /** | ||||
|    * Builds the options map. | ||||
|    * | ||||
| @@ -215,4 +244,6 @@ public class OptionsBuilder { | ||||
|   public Options build() { | ||||
|     return options; | ||||
|   } | ||||
|  | ||||
|  | ||||
| } | ||||
|   | ||||
| @@ -2,6 +2,10 @@ package io.github.ollama4j.unittests; | ||||
|  | ||||
| import io.github.ollama4j.OllamaAPI; | ||||
| import io.github.ollama4j.exceptions.OllamaBaseException; | ||||
| import io.github.ollama4j.exceptions.RoleNotFoundException; | ||||
| import io.github.ollama4j.models.chat.OllamaChatMessageRole; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbedRequestModel; | ||||
| import io.github.ollama4j.models.embeddings.OllamaEmbedResponseModel; | ||||
| import io.github.ollama4j.models.response.ModelDetail; | ||||
| import io.github.ollama4j.models.response.OllamaAsyncResultStreamer; | ||||
| import io.github.ollama4j.models.response.OllamaResult; | ||||
| @@ -14,7 +18,9 @@ import java.io.IOException; | ||||
| import java.net.URISyntaxException; | ||||
| import java.util.ArrayList; | ||||
| import java.util.Collections; | ||||
| import java.util.List; | ||||
|  | ||||
| import static org.junit.jupiter.api.Assertions.*; | ||||
| import static org.mockito.Mockito.*; | ||||
|  | ||||
| class TestMockedAPIs { | ||||
| @@ -97,6 +103,34 @@ class TestMockedAPIs { | ||||
|         } | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|     void testEmbed() { | ||||
|         OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class); | ||||
|         String model = OllamaModelType.LLAMA2; | ||||
|         List<String> inputs = List.of("some prompt text"); | ||||
|         try { | ||||
|             when(ollamaAPI.embed(model, inputs)).thenReturn(new OllamaEmbedResponseModel()); | ||||
|             ollamaAPI.embed(model, inputs); | ||||
|             verify(ollamaAPI, times(1)).embed(model, inputs); | ||||
|         } catch (IOException | OllamaBaseException | InterruptedException e) { | ||||
|             throw new RuntimeException(e); | ||||
|         } | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|     void testEmbedWithEmbedRequestModel() { | ||||
|         OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class); | ||||
|         String model = OllamaModelType.LLAMA2; | ||||
|         List<String> inputs = List.of("some prompt text"); | ||||
|         try { | ||||
|             when(ollamaAPI.embed(new OllamaEmbedRequestModel(model, inputs))).thenReturn(new OllamaEmbedResponseModel()); | ||||
|             ollamaAPI.embed(new OllamaEmbedRequestModel(model, inputs)); | ||||
|             verify(ollamaAPI, times(1)).embed(new OllamaEmbedRequestModel(model, inputs)); | ||||
|         } catch (IOException | OllamaBaseException | InterruptedException e) { | ||||
|             throw new RuntimeException(e); | ||||
|         } | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|     void testAsk() { | ||||
|         OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class); | ||||
| @@ -161,4 +195,68 @@ class TestMockedAPIs { | ||||
|         ollamaAPI.generateAsync(model, prompt, false); | ||||
|         verify(ollamaAPI, times(1)).generateAsync(model, prompt, false); | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|     void testAddCustomRole() { | ||||
|         OllamaAPI ollamaAPI = mock(OllamaAPI.class); | ||||
|         String roleName = "custom-role"; | ||||
|         OllamaChatMessageRole expectedRole = OllamaChatMessageRole.newCustomRole(roleName); | ||||
|         when(ollamaAPI.addCustomRole(roleName)).thenReturn(expectedRole); | ||||
|         OllamaChatMessageRole customRole = ollamaAPI.addCustomRole(roleName); | ||||
|         assertEquals(expectedRole, customRole); | ||||
|         verify(ollamaAPI, times(1)).addCustomRole(roleName); | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|     void testListRoles() { | ||||
|         OllamaAPI ollamaAPI = Mockito.mock(OllamaAPI.class); | ||||
|         OllamaChatMessageRole role1 = OllamaChatMessageRole.newCustomRole("role1"); | ||||
|         OllamaChatMessageRole role2 = OllamaChatMessageRole.newCustomRole("role2"); | ||||
|         List<OllamaChatMessageRole> expectedRoles = List.of(role1, role2); | ||||
|         when(ollamaAPI.listRoles()).thenReturn(expectedRoles); | ||||
|         List<OllamaChatMessageRole> actualRoles = ollamaAPI.listRoles(); | ||||
|         assertEquals(expectedRoles, actualRoles); | ||||
|         verify(ollamaAPI, times(1)).listRoles(); | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|     void testGetRoleNotFound() { | ||||
|         OllamaAPI ollamaAPI = mock(OllamaAPI.class); | ||||
|         String roleName = "non-existing-role"; | ||||
|         try { | ||||
|             when(ollamaAPI.getRole(roleName)).thenThrow(new RoleNotFoundException("Role not found")); | ||||
|         } catch (RoleNotFoundException exception) { | ||||
|             throw new RuntimeException("Failed to run test: testGetRoleNotFound"); | ||||
|         } | ||||
|         try { | ||||
|             ollamaAPI.getRole(roleName); | ||||
|             fail("Expected RoleNotFoundException not thrown"); | ||||
|         } catch (RoleNotFoundException exception) { | ||||
|             assertEquals("Role not found", exception.getMessage()); | ||||
|         } | ||||
|         try { | ||||
|             verify(ollamaAPI, times(1)).getRole(roleName); | ||||
|         } catch (RoleNotFoundException exception) { | ||||
|             throw new RuntimeException("Failed to run test: testGetRoleNotFound"); | ||||
|         } | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|     void testGetRoleFound() { | ||||
|         OllamaAPI ollamaAPI = mock(OllamaAPI.class); | ||||
|         String roleName = "existing-role"; | ||||
|         OllamaChatMessageRole expectedRole = OllamaChatMessageRole.newCustomRole(roleName); | ||||
|         try { | ||||
|             when(ollamaAPI.getRole(roleName)).thenReturn(expectedRole); | ||||
|         } catch (RoleNotFoundException exception) { | ||||
|             throw new RuntimeException("Failed to run test: testGetRoleFound"); | ||||
|         } | ||||
|         try { | ||||
|             OllamaChatMessageRole actualRole = ollamaAPI.getRole(roleName); | ||||
|             assertEquals(expectedRole, actualRole); | ||||
|             verify(ollamaAPI, times(1)).getRole(roleName); | ||||
|         } catch (RoleNotFoundException exception) { | ||||
|             throw new RuntimeException("Failed to run test: testGetRoleFound"); | ||||
|         } | ||||
|     } | ||||
| } | ||||
|   | ||||
| @@ -1,6 +1,7 @@ | ||||
| package io.github.ollama4j.unittests.jackson; | ||||
|  | ||||
| import static org.junit.jupiter.api.Assertions.assertEquals; | ||||
| import static org.junit.jupiter.api.Assertions.assertThrowsExactly; | ||||
|  | ||||
| import java.io.File; | ||||
| import java.util.List; | ||||
| @@ -59,6 +60,10 @@ public class TestChatRequestSerialization extends AbstractSerializationTest<Olla | ||||
|             .withOptions(b.setSeed(1).build()) | ||||
|             .withOptions(b.setTopK(1).build()) | ||||
|             .withOptions(b.setTopP(1).build()) | ||||
|             .withOptions(b.setMinP(1).build()) | ||||
|             .withOptions(b.setCustomOption("cust_float", 1.0f).build()) | ||||
|             .withOptions(b.setCustomOption("cust_int", 1).build()) | ||||
|             .withOptions(b.setCustomOption("cust_str", "custom").build()) | ||||
|             .build(); | ||||
|  | ||||
|         String jsonRequest = serialize(req); | ||||
| @@ -72,6 +77,20 @@ public class TestChatRequestSerialization extends AbstractSerializationTest<Olla | ||||
|         assertEquals(1, deserializeRequest.getOptions().get("seed")); | ||||
|         assertEquals(1, deserializeRequest.getOptions().get("top_k")); | ||||
|         assertEquals(1.0, deserializeRequest.getOptions().get("top_p")); | ||||
|         assertEquals(1.0, deserializeRequest.getOptions().get("min_p")); | ||||
|         assertEquals(1.0, deserializeRequest.getOptions().get("cust_float")); | ||||
|         assertEquals(1, deserializeRequest.getOptions().get("cust_int")); | ||||
|         assertEquals("custom", deserializeRequest.getOptions().get("cust_str")); | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|     public void testRequestWithInvalidCustomOption() { | ||||
|         OptionsBuilder b = new OptionsBuilder(); | ||||
|         assertThrowsExactly(IllegalArgumentException.class, () -> { | ||||
|                 OllamaChatRequest req = builder.withMessage(OllamaChatMessageRole.USER, "Some prompt") | ||||
|                 .withOptions(b.setCustomOption("cust_obj", new Object()).build()) | ||||
|                 .build(); | ||||
|         }); | ||||
|     } | ||||
|  | ||||
|     @Test | ||||
|   | ||||
		Reference in New Issue
	
	Block a user