Set a cut-off limit for the responses of the GPT functions in the current spreadsheet. This won't shape the response, but if the response goes beyond this limit, it will be truncated.
|Tokens can be thought of as pieces of words. During processing, the language model breaks down both the input (prompt) and the result (completion) texts into smaller units called tokens. Tokens generally correspond to ~4 characters of common English text. So 100 tokens are approximately worth 75 words. See how text is split into tokens.
|Token limit is the maximum total number of tokens that can be used in both the input (prompt) and the result (completion) when interacting with a language model.
|Cut-off limit of the result measured in tokens. If the result is larger than this limit, it will be truncated. Helps control cost and speed. The maximum value for Max tokens is set 200 tokens below the model Token limit. For example, gpt-3.5-turbo has a capacity of 4096 tokens, so you can only set Max tokens up to 3896. This provides room for your input and prevents hitting the max token limit due to an excessively high Max tokens setting.
In the GPT for Sheets sidebar, click Settings.
Set Max tokens as follows:
- Auto: The model will automatically use the remaining available tokens (with Safe mode only).
- Fixed: The maximum number of tokens is set manually using the slider. The value must be lower than the model limit to make room for input tokens.
- If you expect short responses, lower the limit to get faster responses
- If you expect long responses, increase the limit to make sure they are not truncated
- If your response is truncated, increase the limit
The cut-off limit you have set is now the default value for all GPT executions from Google Sheets. If you make new executions or regenerate existing GPT formulas, the output will take this new value into account.
Select other settings to customize how the language model operates.