Package nl.dannyj.mistral.models.completion

Class ChatCompletionRequest

java.lang.Object
nl.dannyj.mistral.models.completion.ChatCompletionRequest
All Implemented Interfaces:
Request
public class ChatCompletionRequest extends Object implements Request
The ChatCompletionRequest class represents a request to create a chat completion (an assistant reply to the conversation). Most of the field descriptions are taken from the Mistral API documentation.

  • Constructor Details

    • ChatCompletionRequest

      public ChatCompletionRequest(String model, @Nullable Double temperature, Double topP, @Nullable Integer maxTokens, Boolean stream, List<String> stop, @Nullable Long randomSeed, List<ChatMessage> messages, ResponseFormat responseFormat, boolean safePrompt, @Nullable List<Tool> tools, ToolChoiceOption toolChoice, @Nullable Double presencePenalty, @Nullable Double frequencyPenalty, @Nullable Integer n, @Nullable Prediction prediction, Boolean parallelToolCalls)
      Creates a new ChatCompletionRequest instance.
      Parameters:
      model - The model's ID. Can't be null or empty.
      temperature - The sampling temperature to use. Has to be between 0.0 and 1.5. Null will default to the model's default value.
      topP - the top p value to use. Has to be between 0.0 and 1.0.
      maxTokens - The maximum number of tokens to generate in the completion. Has to be positive or zero. Null for the model's default value.
      stream - Whether to stream back partial progress. Setting to null will default to false.
      stop - The stop sequence(s) to use. Use an empty List (default value) for no stop sequence.
      randomSeed - The seed to use for random sampling. Set to null for a random seed.
      messages - The list of messages representing the conversation history. Must contain at least one message.
      responseFormat - The response format of the completion request. Currently only available when using mistral small and mistral large models. For other models, this MUST be set to null.
      safePrompt - Whether to inject a safety prompt before all conversations.
      tools - The list of tools.
      toolChoice - The tool choice option (can be ToolChoiceEnum or SpecificToolChoice).
      presencePenalty - The presence penalty.
      frequencyPenalty - The frequency penalty.
      n - The number of choices to generate.
      prediction - The prediction object.
      parallelToolCalls - Whether parallel tool calls are allowed.

    • ChatCompletionRequest

      public ChatCompletionRequest()

  • Method Details

    • builder

      public static ChatCompletionRequest.ChatCompletionRequestBuilder builder()

    • getModel

      public String getModel()
      ID of the model to use. You can use the List Available Models API (MistralClient.listModels()) to see all of your available models.
      Returns:
      The model's ID.

    • getTemperature

      @Nullable public Double getTemperature()
      What sampling temperature to use, Mistral recommends this to be between 0.0 and 0.7. Higher values like 0.7 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Mistral generally recommends altering this or top_p but not both. The default value varies depending on the model you are targeting.
      Returns:
      The sampling temperature to use.

    • getTopP

      public Double getTopP()
      Nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both. Defaults to 1.0 (i.e., no nucleus sampling).
      Returns:
      the top p value to use.

    • getMaxTokens

      @Nullable public Integer getMaxTokens()
      The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model's context length.
      Returns:
      The maximum number of tokens to generate in the completion.

    • getStream

      public Boolean getStream()
      Whether to stream back partial progress. When set to true, the MistralClient.createChatCompletionStream(ChatCompletionRequest, ChatCompletionChunkCallback) method has to be used.
      Returns:
      Whether to stream back partial progress.

    • getStop

      public List<String> getStop()
      Stop generation if this token is detected. Or if one of these tokens is detected when providing an array
      Returns:
      The stop sequence(s) to use.

    • getRandomSeed

      @Nullable public Long getRandomSeed()
      The seed to use for random sampling. If set, different calls will generate deterministic results.
      Returns:
      The seed to use for random sampling.

    • getMessages

      public List<ChatMessage> getMessages()
      The prompt(s) to generate completions for, encoded as a list of dict with role and content. Must contain at least one message and the first prompt role should be user or system.
      Returns:
      The list of messages.

    • getResponseFormat

      public ResponseFormat getResponseFormat()
      An object specifying the format that the model must output. Setting to JSON_OBJECT enables JSON mode, which guarantees the message the model generates is in JSON. When using JSON mode you MUST also instruct the model to produce JSON yourself with a system or a user message.
      Returns:
      The response format of the completion request.

    • isSafePrompt

      public boolean isSafePrompt()
      Whether to inject a safety prompt before all conversations. Toggling the safe prompt will prepend your messages with the following system prompt: Always assist with care, respect, and truth. Respond with utmost utility yet securely. Avoid harmful, unethical, prejudiced, or negative content. Ensure replies promote fairness and positivity.
      Returns:
      Whether to inject a safety prompt before all conversations.

    • getTools

      @Nullable public List<Tool> getTools()
      A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. Set to null or an empty list if no tools should be available.
      Returns:
      The list of tools.

    • getToolChoice

      public ToolChoiceOption getToolChoice()
      Controls which function call(s) are made, if any. 'none' means the model will not call a function and instead generates a message. 'auto' means the model can pick between generating a message or calling a function. 'any' forces the model to call a function. Specifying a particular function via SpecificToolChoice forces the model to call that function. Defaults to 'auto'.
      Returns:
      The tool choice option.

    • getPresencePenalty

      @Nullable public Double getPresencePenalty()
      Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. Defaults to 0.0.
      Returns:
      The presence penalty.

    • getFrequencyPenalty

      @Nullable public Double getFrequencyPenalty()
      Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. Defaults to 0.0.
      Returns:
      The frequency penalty.

    • getN

      @Nullable public Integer getN()
      How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Defaults to 1.
      Returns:
      The number of choices.

    • getPrediction

      @Nullable public Prediction getPrediction()
      Provides predicted output to optimize response time. See the Predicted Outputs guide for more details. API might have its own default object if null.
      Returns:
      The prediction object.

    • getParallelToolCalls

      public Boolean getParallelToolCalls()
      Whether to allow parallel function calls. Defaults to true.
      Returns:
      True if parallel tool calls are allowed, false otherwise.

    • setModel

      public void setModel(String model)
      ID of the model to use. You can use the List Available Models API (MistralClient.listModels()) to see all of your available models.
      Parameters:
      model - The model's ID. Can't be null or empty.

    • setTemperature

      public void setTemperature(@Nullable Double temperature)
      What sampling temperature to use, Mistral recommends this to be between 0.0 and 0.7. Higher values like 0.7 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Mistral generally recommends altering this or top_p but not both. The default value varies depending on the model you are targeting.
      Parameters:
      temperature - The sampling temperature to use. Has to be between 0.0 and 1.5. Null will default to the model's default value.

    • setTopP

      public void setTopP(Double topP)
      Nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both. Defaults to 1.0 (i.e., no nucleus sampling).
      Parameters:
      topP - the top p value to use. Has to be between 0.0 and 1.0.

    • setMaxTokens

      public void setMaxTokens(@Nullable Integer maxTokens)
      The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model's context length.
      Parameters:
      maxTokens - The maximum number of tokens to generate in the completion. Has to be positive or zero. Null for the model's default value.

    • setStream

      public void setStream(Boolean stream)
      Whether to stream back partial progress. When set to true, the MistralClient.createChatCompletionStream(ChatCompletionRequest, ChatCompletionChunkCallback) method has to be used.
      Parameters:
      stream - Whether to stream back partial progress. Setting to null will default to false.

    • setStop

      public void setStop(List<String> stop)
      Stop generation if this token is detected. Or if one of these tokens is detected when providing an array
      Parameters:
      stop - The stop sequence(s) to use. Use an empty List (default value) for no stop sequence.

    • setRandomSeed

      public void setRandomSeed(@Nullable Long randomSeed)
      The seed to use for random sampling. If set, different calls will generate deterministic results.
      Parameters:
      randomSeed - The seed to use for random sampling. Set to null for a random seed.

    • setMessages

      public void setMessages(List<ChatMessage> messages)
      The prompt(s) to generate completions for, encoded as a list of dict with role and content. Must contain at least one message and the first prompt role should be user or system.
      Parameters:
      messages - The list of messages representing the conversation history. Must contain at least one message.

    • setResponseFormat

      public void setResponseFormat(ResponseFormat responseFormat)
      An object specifying the format that the model must output. Setting to JSON_OBJECT enables JSON mode, which guarantees the message the model generates is in JSON. When using JSON mode you MUST also instruct the model to produce JSON yourself with a system or a user message.
      Parameters:
      responseFormat - The response format of the completion request. Currently only available when using mistral small and mistral large models. For other models, this MUST be set to null.

    • setSafePrompt

      public void setSafePrompt(boolean safePrompt)
      Whether to inject a safety prompt before all conversations. Toggling the safe prompt will prepend your messages with the following system prompt: Always assist with care, respect, and truth. Respond with utmost utility yet securely. Avoid harmful, unethical, prejudiced, or negative content. Ensure replies promote fairness and positivity.
      Parameters:
      safePrompt - Whether to inject a safety prompt before all conversations.

    • setTools

      public void setTools(@Nullable List<Tool> tools)
      A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for. Set to null or an empty list if no tools should be available.
      Parameters:
      tools - The list of tools.

    • setToolChoice

      public void setToolChoice(ToolChoiceOption toolChoice)
      Controls which function call(s) are made, if any. 'none' means the model will not call a function and instead generates a message. 'auto' means the model can pick between generating a message or calling a function. 'any' forces the model to call a function. Specifying a particular function via SpecificToolChoice forces the model to call that function. Defaults to 'auto'.
      Parameters:
      toolChoice - The tool choice option (can be ToolChoiceEnum or SpecificToolChoice).

    • setPresencePenalty

      public void setPresencePenalty(@Nullable Double presencePenalty)
      Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. Defaults to 0.0.
      Parameters:
      presencePenalty - The presence penalty.

    • setFrequencyPenalty

      public void setFrequencyPenalty(@Nullable Double frequencyPenalty)
      Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. Defaults to 0.0.
      Parameters:
      frequencyPenalty - The frequency penalty.

    • setN

      public void setN(@Nullable Integer n)
      How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Defaults to 1.
      Parameters:
      n - The number of choices to generate.

    • setPrediction

      public void setPrediction(@Nullable Prediction prediction)
      Provides predicted output to optimize response time. See the Predicted Outputs guide for more details. API might have its own default object if null.
      Parameters:
      prediction - The prediction object.

    • setParallelToolCalls

      public void setParallelToolCalls(Boolean parallelToolCalls)
      Whether to allow parallel function calls. Defaults to true.
      Parameters:
      parallelToolCalls - Whether parallel tool calls are allowed.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • canEqual

      protected boolean canEqual(Object other)

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object