Input & Output Parameters (Fields of the WorkFlow module)
- Source : The field to be used as a request (prompt).
- JSON Request : The field in which the complete JSON request is saved.
- Destination : The field in which the answer is saved.
- JSON Response: The field in which the complete JSON response is saved.
Request Body Parameters Description & Info
Summary
There are several parameters to consider in a chat completion request:
- Model: Name of the deployment. The values are currently entered in the “Azure OpenAI Settings” in the Marini Systems Admin area.
- Max Tokens: Limits the response length.
- Temperature: Influences response creativity. Low (0.2) for conservative, high (0.8) for creative responses.
- Top P: Controls token selection. Low (0.1) for probable, high (1.0) for diverse answers.
- Frequency Penalty: Influences the repetition of tokens. Low (0) encourages, high (0.8) avoids.
- Presence Penalty: A higher value (e.g. 1.5) reduces the probability of repetitions in the model’s responses.
- API version: The version of the API that should be used. The values are currently entered in the “Azure OpenAI Settings” in the Marini Systems Admin area.
Description
Model : Name of the deployment you want to forward the request to. Azure OpenAI Service is based on a variety of models with different features and price points. Model availability varies by region. Supported for endpoints that support multiple deployments.
Max Tokens : The maximum number of tokens that can be generated in chat completion. The total length of input tokens and generated tokens is limited by the context length of the model. Passing NULL causes the model to use its maximum context length.
Temperature (a number between 0 and 2.0; default value 1) : Non-negative number Returns 422 if the value is not supported by the model.
Top P (a number between 0 and 1.0; given value 1) :An alternative to sampling with temperature, the so-called nucleus sampling, where the model considers the results of the tokens with the probability mass “top_p”. Therefore, 0.1 means that only the tokens that comprise the top 10% of the probability mass are taken into account.
Frequency Penalty (a number between -2.0 and 2.0; default value 0) : Prevents word repetition by reducing the probability that a word will be selected if it has already been used. The higher the frequency penalty, the less likely the model is to repeat the same words in its output. Returns a 422 error if the value or parameter is not supported by the model.
Presence Penalty (a number between -2.0 and 2.0; default value 0) : Prevents the same topics from being repeated by penalizing a word if it is already present in the completion, even if it is only once. Returns a 422 error if the value or parameter is not supported by the model.
API version: The version of the API in the format “YYYY-MM-DD” or “YYYY-MM-DD-Preview”.n.
Examples
What is a robot? (Assuming all other parameters have the default values) model”:”gpt-35-turbo”
Temperature : 0,2 A robot is a mechanical or virtual artificial agent that is designed to perform tasks autonomously or with human guidance. It can be programmed to carry out specific actions or functions, often mimicking human behavior or intelligence. Robots can be found in various forms, such as industrial robots used in manufacturing, humanoid robots that resemble humans, robotic pets, or even software robots that perform automated tasks on computers.
Temperature : 0,8 A robot is a mechanical or virtual artificial agent that is designed to perform tasks autonomously or semi-autonomously, typically through programming or remote control. Robots are often equipped with sensors, actuators, and processors to perceive their environment, make decisions, and manipulate objects. They can be found in various forms, including industrial machines, humanoid robots, drones, automated vehicles, and virtual assistants. The field of robotics involves the study, design, and development of robots, aiming to create intelligent and capable machines that can assist or replace humans in various tasks.
Top P : 0,1 A robot is a mechanical or virtual artificial agent that is programmed to perform tasks autonomously or with human guidance. It can be designed to interact with its environment, gather information, make decisions, and carry out specific actions. Robots can come in various forms, ranging from industrial robots used in manufacturing processes to humanoid robots that resemble humans and are used in research, entertainment, or assistance roles. They can be controlled by computer programs or artificial intelligence algorithms, and their capabilities can vary greatly depending on their design and purpose.
Top P : 1,0 A robot is a programmable machine designed to carry out specific tasks autonomously or under human control. It is usually equipped with sensors, actuators, and a control system to interact with its environment and perform various functions. Robots can be found in various industries, including manufacturing, healthcare, space exploration, and entertainment. They come in different forms, such as industrial robots, service robots, humanoid robots, and autonomous vehicles.
Frequency Penalty : -1 A robot is a mechanical device that is designed to perform tasks autonomously or under remote control, typically programmed to perform a specific set of actions or functions. It is typically equipped with sensors, actuators, and a control system to interact with and manipulate its environment. Robots can range from simple machines programmed to perform a single task to advanced systems with artificial intelligence that can learn, adapt, and perform complex tasks independently. They can be found in various industries and applications, such as manufacturing, healthcare, space exploration, and household chores, among others
Frequency Penalty : 1,9 A robot is a mechanical or virtual artificial agent programmed to perform tasks autonomously or with some level of human supervision. It can be designed to interact with the physical world, manipulate objects, carry out repetitive tasks, gather data, analyze information, and make decisions based on pre-determined rules or algorithms. Robots can take various forms such as humanoid robots resembling humans in appearance and behavior or specialized machines for specific purposes like industrial robots used in manufacturing processes.
Presence Penalty : -1,8 A robot is a mechanical or virtual artificial agent that is designed to perform tasks autonomously or with minimal human intervention. It is typically programmed to carry out specific tasks or functions efficiently and accurately. Robots can be found in various forms and sizes, with capabilities that can range from simple mechanical tasks to complex cognitive functions. They can be used in various industries, such as manufacturing, healthcare, transportation, and exploration, to name a few.
Presence Penalty : 1,8 A robot is a programmable machine designed to automatically carry out a series of tasks or actions. It can be operated by remote control, computer programs, or artificial intelligence. Robots are typically created to perform tasks that are difficult, dangerous, monotonous, or repetitive for humans. They can take various forms and have different levels of complexity, ranging from simple machines with basic functions to highly advanced humanoid robots capable of imitating human behavior and interacting with their environment.
Tips for setting the parameters
Experiment carefully:
– Start with small values and increase them gradually to understand the effects. Especially with temperature and top_p, extreme values can lead to unexpected results.
Mutual influence:
– Changes in one parameter can have effects on others. Experiment with different combinations to find the desired balance between creativity, length, repetition and variety.
Consider safety:
– Review and filter the generated text to ensure it meets the desired standards. Especially with lower values for top_p and temperature, there is a risk of unwanted content.
Consider scaling:
– Consider the potential cost and time required to process requests with different parameters.
Caution – examples of extreme values
- A temperature of 2, for example, leads to very random and unpredictable responses. This is because the temperature affects the influence of the probability distribution on the selection of the next token. At higher temperatures, probabilities are more spread out, which leads to more randomness. The probabilities are greatly weakened, and a kind of “entropy” occurs, where the selection of the next words is less predictable. This can lead to seemingly random or nonsensical answers, as the model is less focused on context or probability. If you want more stable and coherent answers, you should set the temperature to a lower value, for example between 0.2 and 0.8.
- A frequency_penalty of -2 causes an increased repetition of already used tokens. With such an extremely low value, the model is strongly encouraged to repeat frequently used words and this can lead to unusual or even incoherent results. Negative values for frequency_penalty can lead to the model tending to focus too much on certain words instead of generating a diverse and meaningful response. It is advisable to use moderate values for frequency_penalty, usually between 0 and -1, to achieve an appropriate balance between repetition and diversity.