Setup

​
Creating a judge (<5 mins)

  1. Go to Monitoring → Judges → New Judge.
  2. Sepcify the behaviour that you want to track from your production traffic.
  3. Tweak the prompt of the judge until it matches what you are looking for!
That’s it! Historical and future traces will be scored automatically and shown in the dashboard.

​
Calibrating your judge

For each evaluated item you have the option to mark it as correct or incorrect. This is automatically stored and used to improve the judge over time. Calibrating your judge