Class ExternalAPI

java.lang.Object
edu.uky.cs.nil.tt.ExternalAPI

public class ExternalAPI extends Object
The external API is how a client contacts remote services to perform computations the client cannot perform on its own.

An external API typically requires an API key, which is usually a long string that acts as both a user name and password. An API key allows this client to authenticate itself with the external service and allows that service to limit what functions and resources this client can access.

This object represents a client's connection to the external API for that client's lifetime, which is one session. Limits on the resources this API can use exist only for that session; there is no common budget that clients share.

Author:
Stephen G. Ware
  • Field Details

    • EMBED_TOKENS_LIMIT

      public static final int EMBED_TOKENS_LIMIT
      The maximum number of tokens that can be used by the text embedding API during this client's session
      See Also:
    • COMPLETE_IN_TOKENS_LIMIT

      public static final int COMPLETE_IN_TOKENS_LIMIT
      The maximum number of input tokens that can be used by the text completion API during this client's session
      See Also:
    • COMPLETE_OUT_TOKENS_LIMIT

      public static final int COMPLETE_OUT_TOKENS_LIMIT
      The maximum number of output tokens that can be used by the text completion API during this client's session
      See Also:
  • Constructor Details

    • ExternalAPI

      public ExternalAPI(String key)
      Constructs an interface to the external API using an API key.
      Parameters:
      key - a string used by the client to authenticate with the external API, or null if the API does not require a key
  • Method Details

    • hasKey

      public boolean hasKey()
      Returns true if this interface to the external API has a key that can be used to authenticate with the external service.
      Returns:
      true if this interface has an API key, false otherwise
    • getEmbedTokensUsed

      public int getEmbedTokensUsed()
      Returns the number of tokens used for text embedding so far during this session.
      Returns:
      the number of embedding tokens used
      See Also:
    • getEmbedTokensRemaining

      public int getEmbedTokensRemaining()
      Returns the number of tokens that can be used for text embedding before the limit is exceeded.
      Returns:
      the number of tokens that can be used for text embedding
      See Also:
    • getEmbedDimensions

      public Integer getEmbedDimensions()
      Returns the number of dimensions that will be used when embedding a string (i.e. the length of the vector returned by embed(String)). A null value means this setting is not being used and the external API will decide the length of the vector.
      Returns:
      the number of dimensions used for string embedding, or null if this setting is not being used
      See Also:
    • setEmbedDimensions

      public void setEmbedDimensions(Integer value)
      Sets the number of dimensions that will be used when embedding a string (i.e. the length of the vector returned by embed(String)). A null value means this setting will not be used and the external API will decide the length of the vector.
      Parameters:
      value - the new number of dimensions to use for string embedding
      Throws:
      IllegalArgumentException - if the value is less than 1
      See Also:
    • embed

      public float[] embed(String string) throws IOException
      Queries the external text embedding API to convert a string into a high-dimensional numeric vector. The number of dimensions that the resulting vector will have can be set via setEmbedDimensions(Integer).

      Each time a string is embedded it is broken down into tokens. This API limits the total number of tokens that can be used during a client's session to EMBED_TOKENS_LIMIT. You can check the number of embedding tokens used this session with getEmbedTokensUsed() and the number of embedding tokens remaining for this session with getEmbedTokensRemaining().

      Parameters:
      string - the string to convert to a vector
      Returns:
      an array of floating point values corresponding to that string's position in the embedding's latent space
      Throws:
      IllegalStateException - if the client does not have an API key
      RuntimeException - if this client has exceeded the number of tokens that can be used for text embedding
      NullPointerException - if the input string is null
      IOException - if an error occurs while calling the external API
      See Also:
    • getCompleteInTokensUsed

      public int getCompleteInTokensUsed()
      Returns the number of input tokens used for text completion so far during this session.
      Returns:
      the number of input text completion tokens used
      See Also:
    • getCompleteInTokensRemaining

      public int getCompleteInTokensRemaining()
      Returns the number of input tokens that can be used for text completion before the limit is exceeded.
      Returns:
      the number of input tokens that can be used for text completion
      See Also:
    • getCompleteOutTokensUsed

      public int getCompleteOutTokensUsed()
      Returns the number of output tokens used for text completion so far during this session.
      Returns:
      the number of output text completion tokens used
      See Also:
    • getCompleteOutTokensRemaining

      public int getCompleteOutTokensRemaining()
      Returns the number of output tokens that can be used for text completion before the limit is exceeded.
      Returns:
      the number of output tokens that can be used for text completion
      See Also:
    • getCompleteMaxOutTokens

      public Integer getCompleteMaxOutTokens()
      Returns the maximum number of output tokens that can be used when reasoning and generating a single response during text completion. A null value means this setting is not being used and the external API will decide how many tokens to use.

      Reasoning models generate output tokens during the reasoning phase and before generating the text they will return. This limit covers both the tokens generated during reasoning and the actual output tokens. complete(String...) may return an empty output if it uses all tokens during the reasoning phase.

      Returns:
      the max tokens that can be used to generate a single response during text completion, or null if this setting is not being used
      See Also:
    • setCompleteMaxOutTokens

      public void setCompleteMaxOutTokens(Integer value)
      Sets the maximum number of output tokens that can be used when reasoning and generating a single response during text completion. A null value means this setting will not be used and the external API will decide how many tokens to use.
      Parameters:
      value - the new max tokens that can be used to generate a single response during text completion, or null if this setting should not be used
      Throws:
      IllegalArgumentException - if the value is less than 1
      See Also:
    • getCompleteReasoningEffort

      public ExternalAPI.ReasoningEffort getCompleteReasoningEffort()
      Returns the level of reasoning effort the text completion model will use when generating its response. A null value means this setting will not be used and the external API will decide what level of reasoning to use.
      Returns:
      the level of reasoning done during text completion, or null if this setting is not being used
      See Also:
    • setCompleteReasoningEffort

      public void setCompleteReasoningEffort(ExternalAPI.ReasoningEffort value)
      Sets the level of reasoning effort the text completion model will use when generating its response. A null value means this setting should not be used and the external API will decide what level of reasoning to use.

      Reasoning models generate more tokens to check and refine their responses. Lower reasoning uses fewer tokens, while higher reasoning may generate better quality responses.

      Parameters:
      value - the new level of reasoning to do during text completion, or null if this setting should not be used
      See Also:
    • getCompleteTemperature

      public Float getCompleteTemperature()
      Returns the temperature used during text completion. A null value means this setting is not being used and the external API will decide what temperature to use.
      Returns:
      the temperature used during text completion, or null if this setting is not being used
      See Also:
    • setCompleteTemperature

      public void setCompleteTemperature(Float value)
      Sets the temperature to be used during text completion. A null value means this setting should not be used and the external API will decide what temperature to use.

      Temperature affects the variety of the words used in the output. A low temperature will produce predictable and repeatable responses, whereas a high temperature will produce less predictable responses.

      Parameters:
      value - the new temperature to use during text completion, or null if this setting should not be used
      Throws:
      IllegalArgumentException - if the value is less than 0.0 or more than 2.0
      See Also:
    • getCompleteTopK

      public Integer getCompleteTopK()
      Returns the top k setting used during text completion. A null value means this setting is not used and the external API should decide how to choose its next tokens.
      Returns:
      the top k setting used during text completion, or null if this setting is not being used
      See Also:
    • setCompleteTopK

      public void setCompleteTopK(Integer value)
      Sets the top k setting used during text completion. A null value means this setting should not be used and the external API should decide how to choose its next tokens.

      When choosing its next output token during text generation, this setting forces the model to choose only among the k most likely tokens. A value of 1 means the model will always choose the most likely token. Higher values allow the model to choose from a wider variety of tokens. Not using this setting means the model can choose from all tokens.

      Parameters:
      value - the new top k value to use during text completion, or null if this setting should not be used
      Throws:
      IllegalArgumentException - if the value is less than 1
      See Also:
    • getCompleteTopP

      public Float getCompleteTopP()
      Returns the top p setting used during text completion. A null value means this setting is not used and the external API should decide how to choose its next tokens.
      Returns:
      the top p setting used during text completion, or null if this setting is not being used
      See Also:
    • setCompleteTopP

      public void setCompleteTopP(Float value)
      Sets the top p setting used during text completion. A null value means this setting should not be used and the external API should decide how to choose its next tokens.

      When choosing its next output token during text generation, this setting forces the model to choose only among the most likely tokens whose likelihoods sum to p. A low value means the model will generate more predictable text, whereas a higher value means the model will generate less predictable text. Not using this setting means the model can choose from all tokens.

      Parameters:
      value - the new top p value to use during text completion, or null if this setting should not be used
      Throws:
      IllegalArgumentException - if the value is less than 0 or more than 1
      See Also:
    • getCompleteMinP

      public Float getCompleteMinP()
      Returns the min p setting used during text completion. A null value means this setting is not used and the external API should decide how to choose its next tokens.
      Returns:
      the min p setting used during text completion, or null if this setting is not being used
      See Also:
    • setCompleteMinP

      public void setCompleteMinP(Float value)
      Sets the min p setting used during text completion. A null value means this setting should not be used and the external API should decide how to choose its next tokens.

      When choosing its next output token during text generation, this setting forces the model to ignore tokens whose likelihood is below a threshold. The threshold is the product of the likelihood of the best token and p. For example, if p is set to 0.1, then the model may only consider tokens whose likelihood is one tenth as likely as the best token. A low value means the model will have many tokens to choose from and will generate less predictable text, whereas a higher value means the model will have fewer tokens to choose from and will generate more predictable text. Not using this setting means the model can choose from all tokens.

      Parameters:
      value - the new min p value to use during text completion, or null if this setting should not be used
      Throws:
      IllegalArgumentException - if the value is less 0 or more than 1
      See Also:
    • complete

      public String complete(String... conversation) throws IOException
      Queries the external text completion API to generate text which is a fitting continuation of a series of messages. If multiple messages are given as input, then message 0 is treated as the system prompt (the instructions to the text completion agent), odd numbered messages (1, 3, etc.) are treated as messages from the user, and even numbered messages (2, 4, etc.) are treated as responses from the agent. If only a single string is given as input it is treated as a message from the user and the text completion model will use its default system prompt.

      Text completion works by breaking down the input text into tokens, called input tokens. The response text is generated one token at a time, called output tokens. When choosing the next token to add to the output, the model assigns a probability to every possible token and chooses the next token probabilistically based on the likelihood of each token. Several parameters can affect how the model chooses its next tokens:

      • Set the reasoning effort to control how much additional text the model generates to verify and refine its output. Reasoning text is not returned as part of the response, but higher levels of reasoning should produce better responses.
      • Set the temperature to control the level of variation in how output tokens are chosen. A low temperature means the model is more likely to choose the most likely tokens, resulting in more predictable output. A higher temperature results in less predictable responses.
      • Set top k to force the model to choose only from the k most likely output tokens. A low value results in more predictable text. A high value results in less predictable text.
      • Set top p to force the model to choose only from the most likely output tokens whose probabilities add up to p. A low value results in more predictable text. A higher value results in less predictable text.
      • Set min p to force the model to ignore tokens whose likelihood is lower than the likelihood of the best token times p. A low value results in less predictable text. A higher value results in more predictable text.
      This API limits the total number of input and output tokens that can be used during a client's session to COMPLETE_IN_TOKENS_LIMIT and COMPLETE_OUT_TOKENS_LIMIT respectively. You can check the number of input and output tokens used this session with getCompleteInTokensUsed() and getCompleteOutTokensUsed(). You can also limit how many output tokens a model uses by setting setCompleteMaxOutTokens(Integer).
      Parameters:
      conversation - a series of messages
      Returns:
      text that is a fitting continuation of the input messages
      Throws:
      IllegalStateException - if the client does not have an API key
      RuntimeException - if this client has exceeded the number of input or output tokens that can be used for text completion
      NullPointerException - if the input conversation or any of its elements are null
      IllegalArgumentException - if the conversation has more than one message but an odd number of messages
      IOException - if an error occurs while calling the external API
      See Also: