Skip to main content

Installation

npm install zeroeval

Core Functions

init()

Initializes the ZeroEval SDK. Must be called before using any other SDK features.
function init(opts?: InitOptions): void

Parameters

OptionTypeDefaultDescription
apiKeystringZEROEVAL_API_KEY envYour ZeroEval API key
apiUrlstringhttps://api.zeroeval.comCustom API URL
flushIntervalnumber10Interval in seconds to flush spans
maxSpansnumber100Maximum spans to buffer before flushing
collectCodeDetailsbooleantrueCapture source code context
integrationsRecord<string, boolean>Enable/disable specific integrations
debugbooleanfalseEnable debug logging

Example

import * as ze from 'zeroeval';

ze.init({
  apiKey: 'your-api-key',
  debug: true
});

Wrapper Functions

wrap()

Wraps a supported AI client to automatically trace all API calls.
function wrap<T extends object>(client: T): WrappedClient<T>

Supported Clients

  • OpenAI SDK (openai package)
  • Vercel AI SDK (ai package)

Examples

// OpenAI
import { OpenAI } from 'openai';
import * as ze from 'zeroeval';

const openai = ze.wrap(new OpenAI());

// Vercel AI SDK
import * as ai from 'ai';
import * as ze from 'zeroeval';

const wrappedAI = ze.wrap(ai);

Spans API

withSpan()

Wraps a function execution in a span, automatically capturing timing and errors.
function withSpan<T>(
  opts: SpanOptions,
  fn: () => Promise<T> | T
): Promise<T> | T

SpanOptions

OptionTypeRequiredDescription
namestringYesName of the span
sessionIdstringNoSession ID to associate with the span
sessionNamestringNoHuman-readable session name
tagsRecord<string, string>NoTags to attach to the span
attributesRecord<string, unknown>NoAdditional attributes
inputDataanyNoManual input data override
outputDataanyNoManual output data override

Example

import * as ze from 'zeroeval';

const result = await ze.withSpan(
  { name: 'fetch-user-data' },
  async () => {
    const user = await fetchUser(userId);
    return user;
  }
);

@span Decorator

Decorator for class methods to automatically create spans.
span(opts: SpanOptions): MethodDecorator

Example

import * as ze from 'zeroeval';

class UserService {
  @ze.span({ name: 'get-user' })
  async getUser(id: string): Promise<User> {
    return await db.users.findById(id);
  }
}
Requires experimentalDecorators: true in your tsconfig.json.

Context Functions

getCurrentSpan()

Returns the currently active span, if any.
function getCurrentSpan(): Span | undefined

getCurrentTrace()

Returns the current trace ID.
function getCurrentTrace(): string | undefined

getCurrentSession()

Returns the current session ID.
function getCurrentSession(): string | undefined

setTag()

Sets tags on a span, trace, or session.
function setTag(
  target: Span | string | undefined,
  tags: Record<string, string>
): void

Parameters

ParameterDescription
SpanSets tags on the specific span
stringSets tags on the trace or session by ID
undefinedSets tags on the current span

Prompts API

prompt()

Creates or fetches versioned prompts from the Prompt Library. Returns decorated content for downstream LLM calls.
async function prompt(options: PromptOptions): Promise<string>

PromptOptions

OptionTypeRequiredDescription
namestringYesTask name associated with the prompt
contentstringNoRaw prompt content (used as fallback or for explicit mode)
variablesRecord<string, string>NoTemplate variables to interpolate {{variable}} tokens
fromstringNoVersion control: "latest", "explicit", or a 64-char SHA-256 hash

Behavior

  • Auto-optimization (default): If content is provided without from, tries to fetch the latest optimized version first, falls back to provided content
  • Explicit mode (from: "explicit"): Always uses provided content, bypasses auto-optimization
  • Latest mode (from: "latest"): Requires an optimized version to exist, fails if none found
  • Hash mode (from: "<hash>"): Fetches a specific version by its 64-character SHA-256 content hash

Examples

import * as ze from 'zeroeval';

// Auto-optimization mode (recommended)
const prompt = await ze.prompt({
  name: 'customer-support',
  content: 'You are a helpful {{role}} assistant.',
  variables: { role: 'customer service' }
});

// Explicit mode - bypass auto-optimization
const prompt = await ze.prompt({
  name: 'customer-support',
  content: 'You are a helpful assistant.',
  from: 'explicit'
});

// Latest mode - require optimized version
const prompt = await ze.prompt({
  name: 'customer-support',
  from: 'latest'
});

// Hash mode - specific version
const prompt = await ze.prompt({
  name: 'customer-support',
  from: 'a1b2c3d4e5f6...'  // 64-char SHA-256 hash
});

Return Value

Returns a decorated prompt string with metadata header used by integrations:
<zeroeval>{"task":"...", "prompt_version": 1, ...}</zeroeval>Your prompt content here

Errors

ErrorWhen
ErrorBoth content and from provided (except from: "explicit"), or neither
PromptRequestErrorfrom: "latest" but no versions exist
PromptNotFoundErrorfrom is a hash that does not exist

sendFeedback()

Sends feedback for a completion to enable prompt optimization.
async function sendFeedback(options: SendFeedbackOptions): Promise<PromptFeedbackResponse>

SendFeedbackOptions

OptionTypeRequiredDescription
promptSlugstringYesThe slug of the prompt (task name)
completionIdstringYesUUID of the span/completion
thumbsUpbooleanYestrue for positive, false for negative
reasonstringNoExplanation of the feedback
expectedOutputstringNoWhat the expected output should be
metadataRecord<string, unknown>NoAdditional metadata
judgeIdstringNoJudge automation ID for judge feedback
expectedScorenumberNoExpected score for scored judges
scoreDirection'too_high' | 'too_low'NoScore direction for scored judges

Example

import * as ze from 'zeroeval';

await ze.sendFeedback({
  promptSlug: 'support-bot',
  completionId: '550e8400-e29b-41d4-a716-446655440000',
  thumbsUp: false,
  reason: 'Response was too verbose',
  expectedOutput: 'A concise 2-3 sentence response'
});

Signals API

sendSignal()

Send a signal to a specific entity.
async function sendSignal(
  entityType: 'session' | 'trace' | 'span' | 'completion',
  entityId: string,
  name: string,
  value: string | boolean | number,
  signalType?: 'boolean' | 'numerical'
): Promise<void>

sendTraceSignal()

Send a signal to the current trace.
function sendTraceSignal(
  name: string,
  value: string | boolean | number,
  signalType?: 'boolean' | 'numerical'
): void

sendSessionSignal()

Send a signal to the current session.
function sendSessionSignal(
  name: string,
  value: string | boolean | number,
  signalType?: 'boolean' | 'numerical'
): void

sendSpanSignal()

Send a signal to the current span.
function sendSpanSignal(
  name: string,
  value: string | boolean | number,
  signalType?: 'boolean' | 'numerical'
): void

getEntitySignals()

Retrieve signals for a specific entity.
async function getEntitySignals(
  entityType: 'session' | 'trace' | 'span' | 'completion',
  entityId: string
): Promise<Signal[]>

Example

import * as ze from 'zeroeval';

await ze.withSpan({ name: 'process-request' }, async () => {
  // Process something...
  
  // Send signals
  ze.sendSpanSignal('success', true);
  ze.sendSpanSignal('latency_ms', 150);
  ze.sendTraceSignal('user_satisfied', true);
});

Utility Functions

renderTemplate()

Render a template string with variable substitution.
function renderTemplate(
  template: string,
  variables: Record<string, string | number | boolean>,
  options?: { ignoreMissing?: boolean }
): string

extractVariables()

Extract variable names from a template string.
function extractVariables(template: string): Set<string>

sha256Hex()

Compute SHA-256 hash of text.
async function sha256Hex(text: string): Promise<string>

normalizePromptText()

Normalize prompt text for consistent hashing.
function normalizePromptText(text: string): string

Error Classes

PromptNotFoundError

Thrown when a specific prompt version (by hash) is not found.
class PromptNotFoundError extends Error {
  constructor(message: string)
}

PromptRequestError

Thrown when a prompt request fails (e.g., no versions exist for from: "latest").
class PromptRequestError extends Error {
  constructor(message: string, statusCode?: number)
}

Types

Prompt

interface Prompt {
  id: string;
  prompt_id: string;
  content: string;
  content_hash: string;
  version: number;
  model_id?: string;
}

PromptMetadata

interface PromptMetadata {
  task: string;
  prompt_slug?: string;
  prompt_version?: number;
  prompt_version_id?: string;
  content_hash?: string;
  variables?: Record<string, string>;
}

Signal

interface Signal {
  value: string | boolean | number;
  type: 'boolean' | 'numerical';
}
Need help? Check out our GitHub examples or reach out on Discord.