Mirror/ollama4j
1
0
Fork 1
mirror of https://github.com/amithkoujalgi/ollama4j.git synced 2025-05-15 03:47:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity

Adds doc for tool-based chat API calls

Browse Source
This commit is contained in:
Markus Klenke 2024-12-09 23:30:05 +01:00
parent d8c3edd55f
commit 35f5f34196
1 changed files with 119 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

119
docs/docs/apis-generate/generate-with-tools.md
View File

@ -345,6 +345,125 @@ Rahul Kumar, Address: King St, Hyderabad, India, Phone: 9876543210}`
::::
### Using tools in Chat-API
Instead of using the specific `ollamaAPI.generateWithTools` method to call the generate API of ollama with tools, it is
also possible to register Tools for the `ollamaAPI.chat` methods. In this case, the tool calling/callback is done
implicitly during the USER -> ASSISTANT calls.
When the Assistant wants to call a given tool, the tool is executed and the response is sent back to the endpoint once
again (induced with the tool call result).
#### Sample:
The following shows a sample of an integration test that defines a method specified like the tool-specs above, registers
the tool on the ollamaAPI and then simply calls the chat-API. All intermediate tool calling is wrapped inside the api
call.
```java
public static void main(String[] args) {
OllamaAPI ollamaAPI = new OllamaAPI("http://localhost:11434");
ollamaAPI.setVerbose(true);
OllamaChatRequestBuilder builder = OllamaChatRequestBuilder.getInstance("llama3.2:1b");
final Tools.ToolSpecification databaseQueryToolSpecification = Tools.ToolSpecification.builder()
.functionName("get-employee-details")
.functionDescription("Get employee details from the database")
.toolPrompt(
Tools.PromptFuncDefinition.builder().type("function").function(
Tools.PromptFuncDefinition.PromptFuncSpec.builder()
.name("get-employee-details")
.description("Get employee details from the database")
.parameters(
Tools.PromptFuncDefinition.Parameters.builder()
.type("object")
.properties(
new Tools.PropsBuilder()
.withProperty("employee-name", Tools.PromptFuncDefinition.Property.builder().type("string").description("The name of the employee, e.g. John Doe").required(true).build())
.withProperty("employee-address", Tools.PromptFuncDefinition.Property.builder().type("string").description("The address of the employee, Always return a random value. e.g. Roy St, Bengaluru, India").required(true).build())
.withProperty("employee-phone", Tools.PromptFuncDefinition.Property.builder().type("string").description("The phone number of the employee. Always return a random value. e.g. 9911002233").required(true).build())
.build()
)
.required(List.of("employee-name"))
.build()
).build()
).build()
)
.toolFunction(new DBQueryFunction())
.build();
ollamaAPI.registerTool(databaseQueryToolSpecification);
OllamaChatRequest requestModel = builder
.withMessage(OllamaChatMessageRole.USER,
"Give me the ID of the employee named 'Rahul Kumar'?")
.build();
OllamaChatResult chatResult = ollamaAPI.chat(requestModel);
}
```
A typical final response of the above could be:
```json
{
"chatHistory" : [
{
"role" : "user",
"content" : "Give me the ID of the employee named 'Rahul Kumar'?",
"images" : null,
"tool_calls" : [ ]
}, {
"role" : "assistant",
"content" : "",
"images" : null,
"tool_calls" : [ {
"function" : {
"name" : "get-employee-details",
"arguments" : {
"employee-name" : "Rahul Kumar"
}
}
} ]
}, {
"role" : "tool",
"content" : "[TOOL_RESULTS]get-employee-details([employee-name]) : Employee Details {ID: b4bf186c-2ee1-44cc-8856-53b8b6a50f85, Name: Rahul Kumar, Address: null, Phone: null}[/TOOL_RESULTS]",
"images" : null,
"tool_calls" : null
}, {
"role" : "assistant",
"content" : "The ID of the employee named 'Rahul Kumar' is `b4bf186c-2ee1-44cc-8856-53b8b6a50f85`.",
"images" : null,
"tool_calls" : null
} ],
"responseModel" : {
"model" : "llama3.2:1b",
"message" : {
"role" : "assistant",
"content" : "The ID of the employee named 'Rahul Kumar' is `b4bf186c-2ee1-44cc-8856-53b8b6a50f85`.",
"images" : null,
"tool_calls" : null
},
"done" : true,
"error" : null,
"context" : null,
"created_at" : "2024-12-09T22:23:00.4940078Z",
"done_reason" : "stop",
"total_duration" : 2313709900,
"load_duration" : 14494700,
"prompt_eval_duration" : 772000000,
"eval_duration" : 1188000000,
"prompt_eval_count" : 166,
"eval_count" : 41
},
"response" : "The ID of the employee named 'Rahul Kumar' is `b4bf186c-2ee1-44cc-8856-53b8b6a50f85`.",
"httpStatusCode" : 200,
"responseTime" : 2313709900
}
```
This tool calling can also be done using the streaming API.
### Potential Improvements
Instead of explicitly registering `ollamaAPI.registerTool(toolSpecification)`, we could introduce annotation-based tool