THE ORCHEMIST MUSEUM · 2026

Where chaos
becomes craft.

Orchemist was an orchestration engine for AI agents: every task through fixed steps, a separate agent for each, and every result scored before anything was accepted. I built it in 2026 to learn how language models behave at work. It is paused indefinitely, and this is its museum.

pipeline.yaml
phases:
  spec          # define what to build
  behavioral    # write contracts
  adversary     # challenge them
  tests         # prove it works before it exists
  implement     # write the code
  review        # score it
  test          # verify it

transitions:
  review.fail: implement  # loop back, not forward

Exhibit 1 · pipeline.yaml · YAML, spring 2026 — the file that defined a run: the phases in order, and where a failed review went back to.

ROOM 1 · THE IDEA

AI is probabilistic.
Your pipeline shouldn't be.

AI agents are coin flips wrapped in confidence. Every run produces different output. Every output needs verification. Most teams either trust blindly or review manually. Neither scales.

CURATOR’S NOTE
That was the bet, as written in spring 2026: agents were confident and inconsistent, so the checking had to live outside them.

ROOM 2 · THE FIVE RULES

From probabilistic
to verifiable.

Orchemist was not another orchestrator. It was a trust engine, built on five rules.

01

Specs before code

Define what success looks like before writing a single line.

02

Adversarial review

Challenge assumptions before you build on them.

03

Tests before implementation

Prove it works before it exists.

04

Scoring and routing

Quantify confidence. Gate on thresholds. Route on results.

05

Bounded self-healing

Retry with feedback, not blindly. Surface blockers, don't hide them.

ROOM 3 · THE MACHINE

Any workload. Any sequence.
One engine.

A pipeline was a YAML state machine: phases, transitions and quality gates. The engine followed the graph. A failed gate sent the work back with feedback, or up to a person; phases, models and gates could change without touching code.

SPEC Define gates BUILD Execute phase SCORE Evaluate output GATE OK start done scored pass fail / retry with feedback escalate / re-spec

ROOM 4 · THE EXHIBITS

Pipelines for real work.

Four pipelines, as shown in spring 2026. Each card opens its exhibit.

ROOM 5 · THE PITCH, SPRING 2026

They orchestrate agents.
We orchestrate trust.

The orchestration frameworks of the time competed on how elegantly they routed agents. The pitch was that none of them checked whether the agents’ work could be trusted: they solved communication, and Orchemist would solve verification.

WHAT THEY DO
WHAT WE ADD
Agent routing
Behavioral acceptance tests
Prompt chaining
Confidence scoring with routing
Multi-model support
Adversarial spec review
Workflow graphs
Bounded self-healing
—
Full audit trail

CURATOR’S NOTE

What held up: checking mattered more than routing, and keeping the writer, the checker and the critic apart worked. What didn’t: the idea that this needed an engine of its own.

ROOM 6 · WHAT IT FOUND

Where agentic programming
broke.

I built Orchemist to learn how the models really behave. These are the three things it taught me.

Agents forget.

Every session started from zero. What an agent had learned the week before was gone, unless it was written down and handed back to it.

Groundhog Day, AI edition ↗

Self-checks are too kind.

A model checking its own work found reasons it was right. The checker had to be a different agent, one that never saw the builder's reasoning.

It Scored Lower Because It Stopped Lying ↗

Confident is not correct.

Early versions produced confident, wrong content. Every claim needed a source, and every result a check that did not come from the agent that made it.

I Published AI Content Without Challenging It ↗

WHAT HAPPENED NEXT

Then the models changed. Newer ones report their own mistakes more honestly, and the tools around them now come with planning, review and parallel agents built in. Much of what Orchemist added from the outside now comes from the inside, which is why I stopped using it.

The checks didn’t retire. Their job moved up: from does the report match reality? to was this the right work?

THE NUMBERS

As it stood in spring 2026.

7,000+
Tests passing
11
Pipeline phases
4
Execution modes
0.13.1
Last release, 10 June 2026

ROOM 7 · THE ARCHIVE

Written along the way.

The essays on connylazo.com, in the order they came out, and the releases.

  1. 16 Feb 2026

    The Orchestra series, in five parts: running a team of agents, before Orchemist. Setting Up Your Stage

  2. 28 Feb 2026

    The engine, three and a half weeks in. Building Your Own AI Orchestration Engine

  3. 10 Mar 2026

    The launch series, part 1: why it had to exist. I Published AI Content Without Challenging It

  4. 20 Mar 2026

    Part 2: not only code. A Trust Factory for Anything AI Touches

  5. 21 Mar 2026

    Part 3: specs and tests before the code. When AI Writes the Code, Who Checks the Homework?

  6. 30 Mar 2026

    The first release on PyPI, 0.8.0.

  7. 18 Apr 2026

    Part 4: running it. Getting Orchemist Running in 10 Minutes

  8. 25 Apr 2026

    Part 5: the roadmap. From Factory to Dark Factory

  9. 7 May 2026

    What agents don't know they don't know. The Agent Memory Wall

  10. 22 May 2026

    Forgetting as the operating condition. Groundhog Day, AI edition

  11. 6 Jun 2026

    The models got more honest, and two years of my own systems became half-obsolete. It Scored Lower Because It Stopped Lying

  12. 10 Jun 2026

    The last release, 0.13.1.

  13. Then

    Paused indefinitely. This site became its museum on 11 November 2026.

THE LAST RELEASE

Orchemist 0.13.1, released on 10 June 2026, is still on PyPI under the MIT licence. It is not maintained: no fixes and no support. The source repositories are private.