Class ExternalAPI
client contacts remote services to
perform computations the client cannot perform on its own.
An external API typically requires an API key, which is usually a long string that acts as both a user name and password. An API key allows this client to authenticate itself with the external service and allows that service to limit what functions and resources this client can access.
This object represents a client's connection to the external API for that client's lifetime, which is one session. Limits on the resources this API can use exist only for that session; there is no common budget that clients share.
- Author:
- Stephen G. Ware
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic enumRepresents the different levels of effort that can be used by a reasoning model duringtext completion. -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final intThe maximum number of input tokens that can be used by the text completion API during this client's sessionstatic final intThe maximum number of output tokens that can be used by the text completion API during this client's sessionstatic final intThe maximum number of tokens that can be used by the text embedding API during this client's session -
Constructor Summary
ConstructorsConstructorDescriptionExternalAPI(String key) Constructs an interface to the external API using an API key. -
Method Summary
Modifier and TypeMethodDescriptionQueries the external text completion API to generate text which is a fitting continuation of a series of messages.float[]Queries the external text embedding API to convert a string into a high-dimensional numeric vector.intReturns the number of input tokens that can be used fortext completionbeforethe limitis exceeded.intReturns the number of input tokens used fortext completionso far during this session.Returns the maximum number of output tokens that can be used when reasoning and generating a single response duringtext completion.Returns the min p setting used duringtext completion.intReturns the number of output tokens that can be used fortext completionbeforethe limitis exceeded.intReturns the number of output tokens used fortext completionso far during this session.Returns thelevel of reasoning effortthetext completionmodel will use when generating its response.Returns the temperature used duringtext completion.Returns the top k setting used duringtext completion.Returns the top p setting used duringtext completion.Returns the number of dimensions that will be used whenembedding a string(i.e. the length of the vector returned byembed(String)).intReturns the number of tokens that can be used fortext embeddingbeforethe limitis exceeded.intReturns the number of tokens used fortext embeddingso far during this session.booleanhasKey()Returns true if this interface to the external API has a key that can be used to authenticate with the external service.voidsetCompleteMaxOutTokens(Integer value) Sets the maximum number of output tokens that can be used when reasoning and generating a single response duringtext completion.voidsetCompleteMinP(Float value) Sets the min p setting used duringtext completion.voidSets thelevel of reasoning effortthetext completionmodel will use when generating its response.voidsetCompleteTemperature(Float value) Sets the temperature to be used duringtext completion.voidsetCompleteTopK(Integer value) Sets the top k setting used duringtext completion.voidsetCompleteTopP(Float value) Sets the top p setting used duringtext completion.voidsetEmbedDimensions(Integer value) Sets the number of dimensions that will be used whenembedding a string(i.e. the length of the vector returned byembed(String)).
-
Field Details
-
EMBED_TOKENS_LIMIT
public static final int EMBED_TOKENS_LIMITThe maximum number of tokens that can be used by the text embedding API during this client's session- See Also:
-
COMPLETE_IN_TOKENS_LIMIT
public static final int COMPLETE_IN_TOKENS_LIMITThe maximum number of input tokens that can be used by the text completion API during this client's session- See Also:
-
COMPLETE_OUT_TOKENS_LIMIT
public static final int COMPLETE_OUT_TOKENS_LIMITThe maximum number of output tokens that can be used by the text completion API during this client's session- See Also:
-
-
Constructor Details
-
ExternalAPI
Constructs an interface to the external API using an API key.- Parameters:
key- a string used by the client to authenticate with the external API, or null if the API does not require a key
-
-
Method Details
-
hasKey
public boolean hasKey()Returns true if this interface to the external API has a key that can be used to authenticate with the external service.- Returns:
- true if this interface has an API key, false otherwise
-
getEmbedTokensUsed
public int getEmbedTokensUsed()Returns the number of tokens used fortext embeddingso far during this session.- Returns:
- the number of embedding tokens used
- See Also:
-
getEmbedTokensRemaining
public int getEmbedTokensRemaining()Returns the number of tokens that can be used fortext embeddingbeforethe limitis exceeded.- Returns:
- the number of tokens that can be used for text embedding
- See Also:
-
getEmbedDimensions
Returns the number of dimensions that will be used whenembedding a string(i.e. the length of the vector returned byembed(String)). A null value means this setting is not being used and the external API will decide the length of the vector.- Returns:
- the number of dimensions used for string embedding, or null if this setting is not being used
- See Also:
-
setEmbedDimensions
Sets the number of dimensions that will be used whenembedding a string(i.e. the length of the vector returned byembed(String)). A null value means this setting will not be used and the external API will decide the length of the vector.- Parameters:
value- the new number of dimensions to use for string embedding- Throws:
IllegalArgumentException- if the value is less than 1- See Also:
-
embed
Queries the external text embedding API to convert a string into a high-dimensional numeric vector. The number of dimensions that the resulting vector will have can be set viasetEmbedDimensions(Integer).Each time a string is embedded it is broken down into tokens. This API limits the total number of tokens that can be used during a client's session to
EMBED_TOKENS_LIMIT. You can check the number of embedding tokens used this session withgetEmbedTokensUsed()and the number of embedding tokens remaining for this session withgetEmbedTokensRemaining().- Parameters:
string- the string to convert to a vector- Returns:
- an array of floating point values corresponding to that string's position in the embedding's latent space
- Throws:
IllegalStateException- if the client does not have an API keyRuntimeException- if this client has exceeded the number of tokens that can be used for text embeddingNullPointerException- if the input string is nullIOException- if an error occurs while calling the external API- See Also:
-
getCompleteInTokensUsed
public int getCompleteInTokensUsed()Returns the number of input tokens used fortext completionso far during this session.- Returns:
- the number of input text completion tokens used
- See Also:
-
getCompleteInTokensRemaining
public int getCompleteInTokensRemaining()Returns the number of input tokens that can be used fortext completionbeforethe limitis exceeded.- Returns:
- the number of input tokens that can be used for text completion
- See Also:
-
getCompleteOutTokensUsed
public int getCompleteOutTokensUsed()Returns the number of output tokens used fortext completionso far during this session.- Returns:
- the number of output text completion tokens used
- See Also:
-
getCompleteOutTokensRemaining
public int getCompleteOutTokensRemaining()Returns the number of output tokens that can be used fortext completionbeforethe limitis exceeded.- Returns:
- the number of output tokens that can be used for text completion
- See Also:
-
getCompleteMaxOutTokens
Returns the maximum number of output tokens that can be used when reasoning and generating a single response duringtext completion. A null value means this setting is not being used and the external API will decide how many tokens to use.Reasoning models generate output tokens during the reasoning phase and before generating the text they will return. This limit covers both the tokens generated during reasoning and the actual output tokens.
complete(String...)may return an empty output if it uses all tokens during the reasoning phase.- Returns:
- the max tokens that can be used to generate a single response during text completion, or null if this setting is not being used
- See Also:
-
setCompleteMaxOutTokens
Sets the maximum number of output tokens that can be used when reasoning and generating a single response duringtext completion. A null value means this setting will not be used and the external API will decide how many tokens to use.- Parameters:
value- the new max tokens that can be used to generate a single response during text completion, or null if this setting should not be used- Throws:
IllegalArgumentException- if the value is less than 1- See Also:
-
getCompleteReasoningEffort
Returns thelevel of reasoning effortthetext completionmodel will use when generating its response. A null value means this setting will not be used and the external API will decide what level of reasoning to use.- Returns:
- the level of reasoning done during text completion, or null if this setting is not being used
- See Also:
-
setCompleteReasoningEffort
Sets thelevel of reasoning effortthetext completionmodel will use when generating its response. A null value means this setting should not be used and the external API will decide what level of reasoning to use.Reasoning models generate more tokens to check and refine their responses. Lower reasoning uses fewer tokens, while higher reasoning may generate better quality responses.
- Parameters:
value- the new level of reasoning to do during text completion, or null if this setting should not be used- See Also:
-
getCompleteTemperature
Returns the temperature used duringtext completion. A null value means this setting is not being used and the external API will decide what temperature to use.- Returns:
- the temperature used during text completion, or null if this setting is not being used
- See Also:
-
setCompleteTemperature
Sets the temperature to be used duringtext completion. A null value means this setting should not be used and the external API will decide what temperature to use.Temperature affects the variety of the words used in the output. A low temperature will produce predictable and repeatable responses, whereas a high temperature will produce less predictable responses.
- Parameters:
value- the new temperature to use during text completion, or null if this setting should not be used- Throws:
IllegalArgumentException- if the value is less than 0.0 or more than 2.0- See Also:
-
getCompleteTopK
Returns the top k setting used duringtext completion. A null value means this setting is not used and the external API should decide how to choose its next tokens.- Returns:
- the top k setting used during text completion, or null if this setting is not being used
- See Also:
-
setCompleteTopK
Sets the top k setting used duringtext completion. A null value means this setting should not be used and the external API should decide how to choose its next tokens.When choosing its next output token during text generation, this setting forces the model to choose only among the k most likely tokens. A value of 1 means the model will always choose the most likely token. Higher values allow the model to choose from a wider variety of tokens. Not using this setting means the model can choose from all tokens.
- Parameters:
value- the new top k value to use during text completion, or null if this setting should not be used- Throws:
IllegalArgumentException- if the value is less than 1- See Also:
-
getCompleteTopP
Returns the top p setting used duringtext completion. A null value means this setting is not used and the external API should decide how to choose its next tokens.- Returns:
- the top p setting used during text completion, or null if this setting is not being used
- See Also:
-
setCompleteTopP
Sets the top p setting used duringtext completion. A null value means this setting should not be used and the external API should decide how to choose its next tokens.When choosing its next output token during text generation, this setting forces the model to choose only among the most likely tokens whose likelihoods sum to p. A low value means the model will generate more predictable text, whereas a higher value means the model will generate less predictable text. Not using this setting means the model can choose from all tokens.
- Parameters:
value- the new top p value to use during text completion, or null if this setting should not be used- Throws:
IllegalArgumentException- if the value is less than 0 or more than 1- See Also:
-
getCompleteMinP
Returns the min p setting used duringtext completion. A null value means this setting is not used and the external API should decide how to choose its next tokens.- Returns:
- the min p setting used during text completion, or null if this setting is not being used
- See Also:
-
setCompleteMinP
Sets the min p setting used duringtext completion. A null value means this setting should not be used and the external API should decide how to choose its next tokens.When choosing its next output token during text generation, this setting forces the model to ignore tokens whose likelihood is below a threshold. The threshold is the product of the likelihood of the best token and p. For example, if p is set to 0.1, then the model may only consider tokens whose likelihood is one tenth as likely as the best token. A low value means the model will have many tokens to choose from and will generate less predictable text, whereas a higher value means the model will have fewer tokens to choose from and will generate more predictable text. Not using this setting means the model can choose from all tokens.
- Parameters:
value- the new min p value to use during text completion, or null if this setting should not be used- Throws:
IllegalArgumentException- if the value is less 0 or more than 1- See Also:
-
complete
Queries the external text completion API to generate text which is a fitting continuation of a series of messages. If multiple messages are given as input, then message 0 is treated as the system prompt (the instructions to the text completion agent), odd numbered messages (1, 3, etc.) are treated as messages from the user, and even numbered messages (2, 4, etc.) are treated as responses from the agent. If only a single string is given as input it is treated as a message from the user and the text completion model will use its default system prompt.Text completion works by breaking down the input text into tokens, called input tokens. The response text is generated one token at a time, called output tokens. When choosing the next token to add to the output, the model assigns a probability to every possible token and chooses the next token probabilistically based on the likelihood of each token. Several parameters can affect how the model chooses its next tokens:
- Set the
reasoning effortto control how much additional text the model generates to verify and refine its output. Reasoning text is not returned as part of the response, but higher levels of reasoning should produce better responses. - Set the
temperatureto control the level of variation in how output tokens are chosen. A low temperature means the model is more likely to choose the most likely tokens, resulting in more predictable output. A higher temperature results in less predictable responses. - Set
top kto force the model to choose only from the k most likely output tokens. A low value results in more predictable text. A high value results in less predictable text. - Set
top pto force the model to choose only from the most likely output tokens whose probabilities add up to p. A low value results in more predictable text. A higher value results in less predictable text. - Set
min pto force the model to ignore tokens whose likelihood is lower than the likelihood of the best token times p. A low value results in less predictable text. A higher value results in more predictable text.
COMPLETE_IN_TOKENS_LIMITandCOMPLETE_OUT_TOKENS_LIMITrespectively. You can check the number of input and output tokens used this session withgetCompleteInTokensUsed()andgetCompleteOutTokensUsed(). You can also limit how many output tokens a model uses by settingsetCompleteMaxOutTokens(Integer).- Parameters:
conversation- a series of messages- Returns:
- text that is a fitting continuation of the input messages
- Throws:
IllegalStateException- if the client does not have an API keyRuntimeException- if this client has exceeded the number of input or output tokens that can be used for text completionNullPointerException- if the input conversation or any of its elements are nullIllegalArgumentException- if the conversation has more than one message but an odd number of messagesIOException- if an error occurs while calling the external API- See Also:
- Set the
-