Input & Output Parameters (Fields of the WorkFlow module)
- Source Field : The field to be used as a request for completion.
- JSON Request Field : The field in which the complete JSON request is saved.
- Destination Field : The field in which the response is saved.
- JSON Response Field: The field in which the complete JSON response is saved.
Request Body Parameters Description & Info
Summary
There are several parameters to consider in a chat completion request:
- Model: Defines the GPT-3 model, e.g. “text-davinci-003.”
- Max Tokens: Limits the response length.
- Temperature: Influences response creativity. Low (0.2) for conservative, high (0.8) for creative responses.
- Top P: Controls token selection. Low (0.1) for probable, high (1.0) for diverse answers.
- Frequency Penalty: Influences the repetition of tokens. Low (0) encourages, high (0.8) avoids.
- Presence Penalty: A higher value (e.g. 1.5) reduces the probability of repetitions in the model’s responses.
Description
Model : The model parameter in a chat completion request defines the specific GPT-3 model to be used to generate the response. The available models may change, so check the latest OpenAI documentation or API reference to make sure you are using the latest model name. Usually you will use models like “text-davinci-003”, but there are also specialized models for specific applications.
Max tokens : The max_tokens parameter in a chat completion request limits the maximum number of tokens generated in the response. You can use it to control the length of the generated response. You can adjust this value as needed to control the desired length of the response. Note, however, that too low a number may make the response incomplete or incoherent.
Temperature (a number between 0 and 2.0; default value 1) : The temperature parameter in a chat completion request affects the creativity of the responses generated. A lower value (e.g. 0.2) generates more conservative and predictable answers, while a higher value (e.g. 0.8) leads to more random and creative answers.
Top P (a number between 0 and 1.0; default value 1) : top_p is a probability limit for token selection in the chat model. A low value (e.g. 0.1) limits the selection to the most probable tokens. Higher values (e.g. 1.0) allow more diversity. Nucleus sampling is a sampling technique in text generation. Only the most probable tokens whose cumulative probability does not exceed the predefined threshold (often referred to as top_p) are selected. This allows a certain degree of control over text diversity, as less probable tokens are excluded. It helps to achieve coherent yet varied results.
Frequency Penalty (a number between -2.0 and 2.0; default value 0) : frequency_penalty in the chat completion request controls how much the model tries to repeat previously used tokens. A low value (e.g. 0) encourages the model to repeat frequently used words, while a higher value (e.g. 0.8) tends to avoid the use of frequent tokens.
Presence Penalty (a number between -2.0 and 2.0; default value 0) : The presence_penalty parameter in a chat completion request influences how much the model tends to repeat previously used tokens in the conversation. A higher presence_penalty value (usually greater than 1) reduces the probability of reusing frequently used words.
A web-based platform for experimenting with different models of OpenAI
Example
What is an iPhone? (assuming all other parameters have the default values) model”: “gpt-3.5-turbo”
Temperature : 0,2
An iPhone is a smartphone developed and manufactured by Apple. It is a cell phone that has a variety of functions, including making phone calls, sending text messages, checking emails, surfing the Internet, taking photos and videos, playing music and downloading and using various apps. The iPhone is known for its elegant design, ease of use and integration with other Apple products and services. There are different models and generations of the iPhone, which are regularly updated and improved.
Temperature : 0,8
An iPhone is a series of smartphones developed and manufactured by the company Apple. It was first introduced in 2007 and has since spawned various models and generations. The iPhone is characterized by its elegant design, touch screen, powerful hardware and iOS operating system. It allows users to make calls, send text messages, browse the internet, take photos and videos, play music and download and use a variety of apps. The iPhone has become one of the best-known and most popular smartphones in the world.
Top P : 0,1
An iPhone is a smartphone developed and manufactured by Apple. It is a cell phone that offers various functions such as calling, text messaging, internet access, email, music and video playback, photography and much more. The iPhone uses Apple’s iOS operating system and is known for its ease of use, design and high quality. There are different models and generations of the iPhone, which are regularly updated.
Top P : 1,0
The iPhone is a smartphone developed and marketed by Apple Inc. It was first introduced in 2007 and has since gone through various models and generations. The iPhone is characterized by its elegant design, ease of use and ecosystem of apps and services. It offers a wide range of functions, including telephony, text messaging, Internet access, camera, music and video playback as well as access to various applications and services.
Frequency Penalty : -1,9
An iPhone is a cell phone developed and marketed by Apple. It is based on Apple’s iOS operating system and has various functions such as calls, text messages, internet access, camera, music, apps, storage
Frequency Penalty : 1,9
An iPhone is a smartphone developed by Apple. It combines the functions of a cell phone with those of a tablet and offers various apps, Internet access and communication options. The first iPhone was introduced in 2007 and since then several models have been launched and regularly updated. iPhones use Apple’s iOS operating system. They are considered high-quality devices with a user-friendly design and a wide range of functions for users of all kinds.
Presence Penalty : -1,8
An iPhone is a smartphone developed and manufactured by Apple. It is a combination of cell phone, music player and Internet device. The iPhone has numerous functions, such as surfing the internet, sending and receiving emails, playing music and videos, taking photos and videos, playing games and downloading apps. It runs on the iOS operating system and is regularly updated with new functions and software updates. There are different models and generations of the iPhone, such as the iPhone 12, iPhone 11 and iPhone SE.
Presence Penalty : 1,8
An iPhone is a series of smartphones developed and manufactured by Apple Inc. It is one of the most popular mobile devices on the market and runs on Apple’s iOS operating system. The iPhone has various functions such as making phone calls, text messaging, surfing the internet, taking photos, playing videos, listening to music and many other applications via the App Store. There are different models of the iPhone, which are regularly introduced with new functions and improvements.
Tips for setting the parameters
Experiment carefully:
– Start with small values and increase them gradually to understand the effects. Especially with temperature and top_p, extreme values can lead to unexpected results.
Mutual influence:
– Changes in one parameter can have effects on others. Experiment with different combinations to find the desired balance between creativity, length, repetition and variety.
Consider safety:
– Review and filter the generated text to ensure it meets the desired standards. Especially with lower values for top_p and temperature, there is a risk of unwanted content.
Consider scaling:
– Consider the potential cost and time required to process requests with different parameters.
Caution – examples of extreme values
- A temperature of 2, for example, leads to very random and unpredictable responses. This is because the temperature affects the influence of the probability distribution on the selection of the next token. At higher temperatures, probabilities are more spread out, which leads to more randomness. The probabilities are greatly attenuated, and a kind of “entropy” occurs, where the selection of next words is less predictable. This can lead to seemingly random or nonsensical answers, as the model is less focused on context or probability. If you want more stable and coherent answers, you should set the temperature to a lower value, for example between 0.2 and 0.8.
-
A frequency_penalty of -2 causes an increased repetition of already used tokens. At such an extremely low value, the model is strongly encouraged to repeat frequently used words and this can lead to unusual or even incoherent results. Negative values for frequency_penalty can lead to the model tending to focus too much on certain words instead of generating a diverse and meaningful response. It is advisable to use moderate values for frequency_penalty, usually between 0 and -1, to achieve an appropriate balance between repetition and diversity.