Quickstart

Everything you need to go from nothing to a live run, on one page: install the CLI, run a first task end to end, then use the configuration reference when you start tuning.

Installation

Two commands. The npm package is a small launcher; scientist login stores your license key and provisions the private core into an isolated, launcher-managed environment (~/.synthetic-scientists/) — nothing touches your project's dependencies.

npm install -g synthetic-scientists
scientist login <YOUR-LICENSE-KEY>

Without Node, the shell installer is equivalent (it bootstraps uv and installs the core the same way):

curl -fsSL https://raw.githubusercontent.com/synthetic-sciences/scientist/main/install.sh | \
  SCIENTIST_LICENSE_KEY=<YOUR-LICENSE-KEY> sh

Verify, inspect, upgrade:

scientist --version
scientist self status                    # launcher, core, and license state
npm install -g synthetic-scientists@latest   # upgrade (core follows the launcher)

Prerequisites

• Node 18+ (for the npm launcher; Python is managed automatically via uv)
• git: Synthetic Scientists uses git worktrees for agent isolation
• tmux (recommended): runs auto-wrap in a tmux session (use run.session=local to skip)

At least one supported agent runtime, installed and authenticated through its own login flow:

• Claude Code (default)
• Codex, OpenCode, Cursor Agent, Kiro

Benchmark adapters that invoke Harbor additionally need Docker on the host; see Benchmarks.

First-time setup

scientist configure                 # detect runtimes, create named profiles
scientist profiles verify         # confirm install + auth

scientist configure and profiles are covered in depth in Runtime Profiles.

Quick Start

This walks through creating a task, writing a grader, and launching agents. Five minutes if the install above went smoothly.

1. Scaffold a task

scientist new my-task

This creates a self-contained task with a packaged grader:

my-task/
├── task.yaml                              # task config + grader entrypoint
├── seed/
│   └── solution.py                        # baseline the agent iterates on
└── grader/                                # standalone Python package
    ├── pyproject.toml                     # name: my-task-grader, deps: [synthetic-scientists]
    └── src/
        └── my_task_grader/
            ├── __init__.py                # re-exports Grader
            └── grader.py                  # class Grader(TaskGrader): ...

The grader ships as a real Python package so it gets its own isolated venv at run time (created from grader.setup in task.yaml).

2. Define your task

Edit my-task/task.yaml:

task:
  name: my-task
  description: "Optimize the function in solution.py to print a higher score."

grader:
  entrypoint: "my_task_grader.grader:Grader"
  setup:
    - "uv pip install -e ./grader"
  timeout: 300
  direction: maximize
  args:
    program_file: "solution.py"

agents:
  count: 2
  runtime: claude_code
  model: claude-sonnet-4-6

workspace:
  repo_path: "./seed"

3. Write a grader

The scaffold already includes a working stub at my-task/grader/src/my_task_grader/grader.py that runs solution.py and parses a single float from stdout. Customise evaluate() for your scoring logic:

from scientist.grader import TaskGrader
from scientist.types import ScoreBundle


class Grader(TaskGrader):
    def evaluate(self) -> float | ScoreBundle:
        program_file = self.args.get("program_file", "solution.py")
        result = self.run_program(program_file)

        if result.returncode != 0:
            return self.fail(f"Program crashed: {result.stderr[:200]}")

        try:
            return float(result.stdout.strip())
        except ValueError:
            return self.fail("Could not parse output as a number")

What you have on self: codebase_path (agent worktree), private_dir (.scientist/private/ for hidden answer keys), args (dict from grader.args), timeout, plus helpers run_program(filename), score(value, explanation=...), fail(reason). See Evaluator Authoring for the full API.

If the grader needs extra dependencies (numpy, torch, etc.), add them to my-task/grader/pyproject.toml under dependencies; they'll be installed into the grader's venv by scientist check and the daemon.

4. Validate the grader

Before launching agents, verify your grader works against the seed code:

scientist check my-task

This runs the grader once and shows you the score. Fix any issues before proceeding.

5. Launch agents

scientist launch -c my-task/task.yaml

Synthetic Scientists will:

1. Create a .scientist/ shared state directory
2. Create isolated git worktrees for each agent
3. Generate a SCIENTIST.md instruction file in each worktree
4. Spawn the agents

6. Monitor progress

scientist results        # ranking
scientist overview     # agent health
scientist dashboard         # web dashboard

7. Stop when done

scientist halt

What happens next?

Each agent autonomously reads its SCIENTIST.md, explores the codebase, makes changes, calls scientist eval -m "description", reads the score and feedback, and iterates — sharing notes and skills with the other agents until stopped.

From here: Concepts explains the moving pieces; the configuration reference below documents every field; Evaluator Authoring covers the full grader API.

Configuration

Every Synthetic Scientists task is defined by a task.yaml file. This section documents all available options.

Full example

task:
  name: "my-task"
  description: "Optimize solution.py to maximize accuracy."
  tips: "numpy and scipy are available. Timeout is 300s."

grader:
  entrypoint: "my_task_grader.grader:Grader"
  setup:
    - "uv pip install -e ./grader"
  timeout: 300
  direction: maximize
  private:
    - "answers/test_data.csv"
  args:
    program_file: "solution.py"

agents:
  count: 2
  runtime: claude_code
  model: claude-sonnet-4-6
  max_turns: 200
  timeout: 3600
  research: true
  cadence:
    - name: reflect
      every: 1
    - name: synthesize
      every: 10
      global: true

labs:
  count: 1
  rotation:
    enabled: true
    every: 50
    rank_window: 20
    min_evals: 3
    dest_weighting: score
    max_per_cycle: 2
    notify_lab: true

sharing:
  attempts: true
  notes: true
  skills: true

workspace:
  results_dir: "./results"
  repo_path: "."

preset field

A top-level preset: pulls in a reusable bundle of config defaults so tasks that share a setup (runtime, gateway, session, grader install step) don't repeat the same boilerplate.

preset: docker-opencode      # built-in preset name
# preset: ./shared.yaml      # or a local YAML file, resolved next to task.yaml

task:
  name: "Decode Throughput"
  description: "..."

grader:
  entrypoint: "decode_eval.grader:Grader"   # task-specific keys still go here

The preset forms a layer between the schema defaults and the task's own keys:

schema defaults  <  preset  <  task.yaml  <  CLI dotlist overrides

So anything the task.yaml sets explicitly wins over the preset, and CLI key=value overrides still win over everything.

• A bare name (no / and no .yaml/.yml suffix) refers to a built-in preset shipped with Synthetic Scientists under scientist/template/presets/.
• Anything else is a path: absolute, or relative to the directory holding your task.yaml.
• Stacking is not supported: a preset file may not itself declare a preset: key.

The resolved run's config.yaml is fully expanded (the preset: reference is flattened away), so scientist continue and the grader daemon never depend on the preset file still being present.

Built-in presets

task section

grader section

Synthetic Scientists bootstraps a venv at .scientist/private/grader_venv/, runs every command in grader.setup inside it, and spawns a worker subprocess from that venv whenever an experiment needs grading. grader.entrypoint is required; leaving it empty is an error at load time. See Evaluator Authoring for the packaged-grader layout.

agents section

Entries under agents.assignments may set a built-in persona: builder, investigator, performance-analyst, verifier, tooling-specialist, or integrator.

Cadence actions

Cadence actions run at regular intervals based on eval count:

cadence:
  - name: reflect        # Built-in: agent reviews progress
    every: 1             # After every eval
  - name: synthesize    # Built-in: agents share knowledge
    every: 10
    global: true         # Uses global eval count (all agents combined)

You can also define custom cadence actions with a prompt:

scientist cadence set review --every 5 --prompt "Review your approach and consider alternatives."

coordination section

Plugin-authored runs can enable a shared task-and-commit DAG with automatic personas and coordinators:

coordination:
  enabled: true
  mode: auto
  coordinator_threshold: 8
  workers_per_coordinator: 4
  auto_personas: true
  seed: coordination.yaml

Up to eight workers claim DAG tasks directly. Above the threshold, Synthetic Scientists adds one coordinator process per four workers. agents.count continues to mean worker count. The DAG is global across labs.

labs section

Multi-lab mode partitions a session into isolated worker groups that search independently until rotation moves an agent between labs. Semantics, agent-ID prefixes, lab-scoped CLI behavior, and worked examples live in Multi-Agent Runs; this section is the option reference.

agents:
  count: 4

labs:
  count: 2
  rotation:
    enabled: true
    every: 50
    rank_window: 20
    min_evals: 3
    dest_weighting: score
    max_per_cycle: 2
    notify_lab: true

labs.count defaults to 1, which preserves the normal single-lab layout. When labs.count > 1, the total number of agents must be at least the number of labs. With agents.assignments, the total agent count is the sum of all assignment counts.

Rotation options

Rotation is ignored when labs.count == 1.

sharing section

Controls what shared state is enabled in .scientist/public/:

workspace section

Uninstall

scientist logout                     # remove the stored license key
scientist self uninstall             # remove the managed core environment
npm uninstall -g synthetic-scientists

Session results live wherever you launched them (results/ or .synthetic/ inside your projects) and are never deleted by an uninstall.