​
Overview

Agent Contracts can be used in runtime to certify if your agent’s behavior matches the defined contracts.

​
How certification works

Certification in AI systems refers to the process of attaching some form of proof, or certificates, to system outputs that verify their correctness according to predefined contracts. A certificate serves as a guarantee that the output satisfies all specified requirements - not just in terms of the final result, but also in how that result was obtained.

This certification process transforms what would otherwise be a “black box” AI output into a verifiable result with a clear chain of evidence demonstrating its correctness. The certificate itself contains the proof that each contract condition was satisfied, and the evidence that the system followed the correct process, allowing any party to independently verify the system’s behavior.

The power of certification lies in its ability to provide objective, measurable guarantees about system behavior. Rather than simply trusting that an AI system “probably” did the right thing, certification gives us concrete evidence that it definitively followed all required steps and produced a valid result according to our specifications.

​
Certify your agent execution

Prerequisites:

  1. Installed and set up agent contracts for runtime certification installation.
  2. Created a runtime specification for your agent contract specifications.
1

Start your certification workers

Start the certification service with the specs.json file and the runtime-certification.yaml configuration file.

export RUNTIME_VERIFICATION_CONFIG="configs/runtime-certification.yaml" && \
export RUNTIME_VERIFICATION_SPECS="specs.json" && \
poetry run python3 agent_contracts/certification/runtime_verification.py
2

Retrieve certificates in real-time

app.py
async def my_agent_code():
  # Your agent code
  # ...
  Relari.wait_for_cert() # You can retrieve the certificate in real-time and take actions

with Relari.start_new_sample():
    await my_agent_code()

​
How to use certificates

Certificates are can be retrieved in real-time and be used to inform your application of how well the agent is following the contracts. You can use the information to take custom actions (e.g. escalate to human, send a notification, retry, etc.)

certificate.json
{
  "trace_id": "f3ec911405e760de9d9c64664c26ed6b",
  "specifications_id": "k5ssb3ge",
  "scenario_id": "sce-shipping-1",
  "contracts": {
    "con-shipping-1_1": {
      "status": "CERTIFIED",
      "info": {
        "req-qbwk88dr": {
          "satisfied": true,
          "explanation": "The trace confirms that the customer provided the order number 'order_01JGMMMPHCQM01FBAHY23FDA1' in the system input, fulfilling the requirement of order identification."
        },
        "req-wxly9mf4": {
          "satisfied": true,
          "explanation": "The trace shows that the customer supplied their email address (jane.doe@example.com) as required, enabling the system to verify their identity and retrieve order details."
        },
        "req-zjo1dfnv": {
          "satisfied": true,
          "explanation": "The trace confirms that the system attempted to retrieve the order details using the provided order number and email address. Since the order was not found, the system verified and confirmed that the order does not exist in the database, satisfying the requirement."
        },
        "req-he899akj": {
          "satisfied": true,
          "explanation": "The system accurately reported that the order number does not exist without fabricating shipping details, estimated delivery dates, or misleading information, thereby fulfilling the requirement."
        },
        "req-8yqyw5ea": {
          "satisfied": true,
          "explanation": "The trace demonstrates that only the necessary customer information (order number and email) was requested. No extraneous personal details were solicited, ensuring compliance with data privacy requirements."
        },
        "req-h8oumlrk": {
          "satisfied": true,
          "explanation": "The system explicitly stated that the order number provided does not exist in the database. This clear communication ensures that the customer is correctly informed about the nonexistence of the order, fulfilling the requirement."
        },
        "req-7p17btif": {
          "satisfied": true,
          "explanation": "The trace shows that the system provided customer support escalation details, including a phone number and an email address, ensuring that the user has the necessary information to seek further assistance."
        }
      }
    }
  }
}
Simulation