ollama4j/docs/docs/apis-generate/chat-with-tools.md

sidebar_position

sidebar_position
8

import CodeEmbed from '@site/src/components/CodeEmbed';

Chat with Tools

Using Tools in Chat

If you want to have a natural back-and-forth chat experience with tools, you can directly integrate tools into the chat() method, instead of using the generateWithTools() method. This allows you to register tools that are automatically used during the conversation between the user and the assistant, creating a more conversational experience.

When the model determines that a tool should be used, the tool is automatically executed. The result is then seamlessly incorporated back into the conversation, enhancing the interaction with real-world data and actions.

The following example demonstrates usage of a simple tool, registered with the OllamaAPI, and then used within a chat session. The tool invocation and response handling are all managed internally by the API.

::::tip[LLM Response] First answer: 6527fb60-9663-4073-b59e-855526e0a0c2 is the ID of the employee named 'Rahul Kumar'.

Second answer: Kumar is the last name of the employee named 'Rahul Kumar'. ::::

This tool calling can also be done using the streaming API.

Client-managed tool calls (clientHandlesTools)

By default, ollama4j automatically executes tool calls returned by the model during chat, runs the corresponding registered Java methods, and appends the tool results back into the conversation. For some applications, you may want to intercept tool calls and decide yourself when and how to execute them (for example, to queue them, to show a confirmation UI to the user, to run them in a sandbox, or to perform multi‑step orchestration).

To enable this behavior, set the clientHandlesTools flag to true on your OllamaAPI instance. When enabled, ollama4j will stop auto‑executing tools and will instead return tool calls inside the assistant message. You can then inspect the tool calls and execute them manually.

Notes:

• Default value: clientHandlesTools is false for backward compatibility.
• When clientHandlesTools is false, ollama4j auto‑executes tools and loops internally until tools are resolved or max retries is reached.
• When clientHandlesTools is true, ollama4j will not execute tools; you are responsible for invoking tools and passing results back as TOOL messages, then re‑calling chat() to continue.

Annotation-Based Tool Registration

Ollama4j provides a declarative and convenient way to define and register tools using Java annotations and reflection. This approach offers an alternative to the more verbose, explicit tool registration method.

To use a method as a tool within a chat call, follow these steps:

• Annotate the Tool Method:
  • Use the @ToolSpec annotation to mark a method as a tool. This annotation describes the tool's purpose.
  • Use the @ToolProperty annotation to define the input parameters of the tool. The following data types are currently supported:
    • java.lang.String
    • java.lang.Integer
    • java.lang.Boolean
    • java.math.BigDecimal
• Annotate the Ollama Service Class:
  • Annotate the class that interacts with the OllamaAPI client using the @OllamaToolService annotation. Reference the provider class(es) containing the @ToolSpec annotated methods within this annotation.
• Register the Annotated Tools:
  • Before making a chat request with the OllamaAPI, call the OllamaAPI.registerAnnotatedTools() method. This registers the annotated tools, making them available for use during the chat session.

Let's try an example. Consider an OllamaToolService class that needs to ask the LLM a question that can only be answered by a specific tool. This tool is implemented within a GlobalConstantGenerator class. Following is the code that exposes an annotated method as a tool:

The annotated method can then be used as a tool in the chat session:

Running the above would produce a response similar to:

::::tip[LLM Response] First answer: 0.0000112061 is the most important constant in the world using 10 digits, according to my function. This constant is known as Planck's constant and plays a fundamental role in quantum mechanics. It relates energy and frequency in electromagnetic radiation and action (the product of momentum and distance) for particles.

Second answer: 3-digit constant: 8.001 ::::