Skip to main content
weave / EvaluationLogger EvaluationLogger enables incremental logging of predictions and scores. Unlike the traditional Evaluation class which requires upfront dataset and batch processing, EvaluationLogger allows you to log predictions as they happen, with flexible scoring. Example
const ev = new EvaluationLogger({name: 'my-eval', dataset: 'my-dataset'});

for (const example of streamingData) {
  const output = await myModel.predict(example);
  const pred = await ev.logPrediction(example, output);

  if (shouldScore(output)) {
    await pred.logScore("accuracy", calculateAccuracy(output));
  }
  await pred.finish();
}

await ev.logSummary();

Table of contents

Constructors

Methods

Constructors

constructor

new EvaluationLogger(options): EvaluationLogger

Parameters

NameType
optionsEvaluationLoggerOptions

Returns

EvaluationLogger

Defined in

evaluationLogger.ts:502

Methods

logPrediction

logPrediction(inputs, output): Promise<ScoreLogger> Log a prediction with its input and output. Creates a predict_and_score call (with child predict call). Returns a ScoreLogger for adding scores.

Parameters

NameType
inputsRecord<string, any>
outputany

Returns

Promise<ScoreLogger>

Defined in

evaluationLogger.ts:579

logSummary

logSummary(summary?): Promise<void> Log a summary and finalize the evaluation. Creates a summarize call and finishes the evaluate call.

Parameters

NameType
summary?Record<string, any>

Returns

Promise<void>

Defined in

evaluationLogger.ts:669
I