ZeroEval’s autotune feature allows you to continuously improve your prompts and automatically deploy the best-performing models. The setup is simple and powerful.
Getting started (<5 mins)
Replace hardcoded prompts with ze.prompt() and include the name of the specific part of your agent that you want to tune.
# Before
prompt = "You are a helpful assistant"
# After - with autotune
prompt = ze.prompt(
name="assistant",
content="You are a helpful assistant"
)
ZeroEval › Tuning › [task_name].
Auto-tune behavior: When you provide content, ZeroEval automatically uses the latest optimized version from your dashboard if one exists. The content parameter serves as a fallback for when no optimized versions are available yet. This means you can hardcode a default prompt in your code, but ZeroEval will seamlessly swap in tuned versions without any code changes.To explicitly use the hardcoded content and bypass auto-optimization, use from_="explicit":prompt = ze.prompt(
name="assistant",
from_="explicit",
content="You are a helpful assistant"
)
Pushing models to production
Once you see a model that performs well, you can send it to production with a single click, as seen below.
Your specified model gets replaced automatically any time you use the prompt from ze.prompt(), as seen below.
# You write this
response = client.chat.completions.create(
model="gpt-4", # ← Gets replaced!
messages=[{"role": "system", "content": prompt}]
)
Example
Here’s autotune in action for a simple customer support bot:
import zeroeval as ze
from openai import OpenAI
ze.init()
client = OpenAI()
# Define your prompt with version tracking
system_prompt = ze.prompt(
name="support-bot",
content="""You are a customer support agent for {{company}}.
Be helpful, concise, and professional.""",
variables={"company": "TechCorp"}
)
# Use it normally - model gets patched automatically
response = client.chat.completions.create(
model="gpt-4", # This might run claude-3-sonnet in production!
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": "I need help with my order"}
]
)
Understanding Prompt Versions
ZeroEval automatically manages prompt versions for you. When you use ze.prompt() with content, the SDK will:
- Check for optimized versions: First, it tries to fetch the latest optimized version from your dashboard
- Fall back to your content: If no optimized versions exist yet, it uses the
content you provided
- Create a version: Your provided content is stored as the initial version for this task
This means you get the best of both worlds: hardcoded fallback prompts in your code, with automatic optimization in production.
# This will use the latest optimized version if one exists in your dashboard
# Otherwise, it uses the content you provide here
prompt = ze.prompt(
name="customer-support",
content="You are a helpful assistant."
)
Explicit version control
If you need more control over which version to use:
# Always use the latest optimized version (fails if none exists)
prompt = ze.prompt(
name="customer-support",
from_="latest"
)
# Always use the hardcoded content (bypass auto-optimization)
prompt = ze.prompt(
name="customer-support",
from_="explicit",
content="You are a helpful assistant."
)
# Use a specific version by its content hash
prompt = ze.prompt(
name="customer-support",
from_="a1b2c3d4..." # 64-character SHA-256 hash
)
When to use each mode
| Mode | Use Case | Behavior |
|---|
content only | Recommended for most cases | Auto-optimization with fallback |
from_="explicit" | Testing, debugging, or A/B testing specific prompts | Always use hardcoded content |
from_="latest" | Production where optimization is required | Fail if no optimized version exists |
from_="<hash>" | Pinning to specific tested versions | Use exact version by hash |
Best practice: Use content parameter alone for local development and production. ZeroEval will automatically use optimized versions when available. Only use from_="explicit" when you specifically need to test or debug the hardcoded content.
