Agent Maturity Compass API - v1.2.0
    Preparing search index...

    Function runExperiment

    • Compares a baseline run against a candidate run.

      Both sides must be real: casebook results are computed from evidence in the ledger, so the candidate configuration has to have actually run and produced evidence. Previously the candidate was never executed — its outcomes were the baseline's plus deterministic jitter — so every "candidate improves on baseline by X%" figure was invented.

      Parameters

      • params: {
            agentId?: string;
            candidateAgentId?: string;
            candidateWindow?: string;
            experimentId: string;
            mode: "sandbox" | "supervise";
            workspace: string;
        }
        • OptionalagentId?: string
        • OptionalcandidateAgentId?: string

          Agent id the candidate ran under, when it differs from the baseline.

        • OptionalcandidateWindow?: string

          Evidence window covering the candidate configuration's own run (e.g. "7d"). Required: without it there is nothing real to compare.

        • experimentId: string
        • mode: "sandbox" | "supervise"
        • workspace: string

      Returns { jsonPath: string; mdPath: string; report: ExperimentReport }

      when no candidate evidence window is supplied, because AMC cannot execute a candidate configuration itself.

    Settings
    Member Visibility