sagax
Introduction

Tasks

The primary way you will interact with sagax is defining tasks. Tasks can be anything: an agent evaluation, tests, benchmarks, etc. You write code driving the task and later configure how you want to run it. In a way, it's analogous to defining an request handler in frameworks like Axum. As such, it's designed to feel similar.

Here is a basic hello world task:

use sagax::{TaskConfig, TaskContext, TaskError, TaskResult};
use serde::{Deserialize, Serialize};
use serde_json::json;

#[derive(Deserialize, Serialize)]
struct PrintConfig {
    message: String,
}

async fn print_task(
    context: TaskContext,
    TaskConfig(config): TaskConfig<PrintConfig>,
) -> Result<TaskResult, TaskError> {
    println!("[trial {}] {}", context.trial_index, config.message);

    Ok(TaskResult {
        output: json!({
          "success": true,
        }),
    })
}

This defines a task printing a message provided by task configuration.

Without a frontend, you can programatically drive the task:

use futures_util::StreamExt;
use sagax::{
    EngineError, Planner, ResolvedExecutionPlan, ResolvedJobSpec, Scheduler, TaskExtensions,
    TaskRegistry, TrialRecord,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut tasks = TaskRegistry::new();
    let extensions = TaskExtensions::new();

    let print_factory = tasks.register("print", print_task)?;

    let planner = Planner::new(tasks, extensions);
    let plan = ResolvedExecutionPlan::new(vec![
        ResolvedJobSpec::new(
            "print",
            print_factory.clone(),
            PrintConfig {
                message: "hello world".to_string(),
            },
        )?
        .trials(5),
    ]);
    let trials = planner.plan(&plan)?;

    let scheduler = Scheduler::default();
    let results: Vec<Result<TrialRecord, EngineError>> =
        scheduler.run(trials).await.collect().await;

    println!("{:?}", results);
    Ok(())
}

This will concurrently invoke the configured job five times, with each invocation printing "hello world". The planner combines the task specification and configuration into a job, then creates trials representing invocations. Each trial is an independently schedulable unit of compute (future) that the scheduler runs until completion.