LLM Resource
The llm
resource facilitates the creation of a Large Language Model (LLM) session to interact with AI models effectively.
Multiple LLM models can be declared and used across multiple LLM resource. For more information, see the Workflow documentation.
Creating a New LLM Resource
To create a new llm
chat resource, you can either generate a new AI agent using the kdeps new
command or scaffold the resource directly.
Here’s how to scaffold an llm
resource:
kdeps scaffold [aiagent] llm
This command will add an llm
resource to the aiagent/resources
folder, generating the following folder structure:
aiagent
└── resources
└── llm.pkl
The file includes essential metadata and common configurations, such as Skip Conditions and Preflight Validations. For more details, refer to the Common Resource Configurations documentation.
Chat Block
Within the file, you’ll find the chat
block, structured as follows:
chat {
model = "tinydolphin" // Specifies the LLM model to use, defined in the workflow.
prompt = "Who is @(request.data())?"
// Determines if the LLM response should be a structured JSON.
JSONResponse = true
// If JSONResponse is true, the structured JSON will include the following keys:
JSONResponseKeys {
"first_name"
"last_name"
"parents"
"address"
"famous_quotes"
"known_for"
}
// Specify the files that this LLM will process.
files {
// "@(request.files()[0])"
}
// Timeout duration in seconds, specifying when to terminate the LLM session.
timeoutSeconds = 60
}
Key Elements of the chat
Block
model
: Specifies the LLM model to be used.prompt
: The input prompt sent to the model.files
: List all the files for use by the LLM model. This feature is particularly beneficial for vision-based LLM models.JSONResponse
: Indicates if the response should be structured as JSON.JSONResponseKeys
: Lists the required keys for the structured JSON response. To ensure the output conforms to specific data types, you can define the keys with their corresponding types. For example:first_name__string
,famous_quotes__array
,details__markdown
, orage__integer
.timeoutSeconds
: Sets the execution timeout in seconds, after which the session is terminated.
When the resource is executed, you can leverage LLM functions like llm.response("id")
to retrieve the generated response. For further details, refer to the LLM Functions documentation.