THE ORCHEMIST MUSEUM · 2026
Orchemist was an orchestration engine for AI agents: every task through fixed steps, a separate agent for each, and every result scored before anything was accepted. I built it in 2026 to learn how language models behave at work. It is paused indefinitely, and this is its museum.
phases:
spec # define what to build
behavioral # write contracts
adversary # challenge them
tests # prove it works before it exists
implement # write the code
review # score it
test # verify it
transitions:
review.fail: implement # loop back, not forward Exhibit 1 · pipeline.yaml · YAML, spring 2026 — the file that defined a run: the phases in order, and where a failed review went back to.
ROOM 1 · THE IDEA
AI agents are coin flips wrapped in confidence. Every run produces different output. Every output needs verification. Most teams either trust blindly or review manually. Neither scales.
CURATOR’S NOTE
That was the bet, as written in spring 2026: agents were confident and inconsistent, so the checking had to
live outside them.
ROOM 2 · THE FIVE RULES
Orchemist was not another orchestrator. It was a trust engine, built on five rules.
Define what success looks like before writing a single line.
Challenge assumptions before you build on them.
Prove it works before it exists.
Quantify confidence. Gate on thresholds. Route on results.
Retry with feedback, not blindly. Surface blockers, don't hide them.
ROOM 3 · THE MACHINE
A pipeline was a YAML state machine: phases, transitions and quality gates. The engine followed the graph. A failed gate sent the work back with feedback, or up to a person; phases, models and gates could change without touching code.
ROOM 4 · THE EXHIBITS
Four pipelines, as shown in spring 2026. Each card opens its exhibit.
Spec, adversary, acceptance tests, implement, review, score. Every phase gated. Every output graded.
Built-in anti-hallucination. Every claim gets a source. Red-team review before publish.
Competitive analysis, structured intelligence gathering. From raw data to actionable insight.
Sequential rewrite with flow review and consistency checks. Voice preservation across revisions.
ROOM 5 · THE PITCH, SPRING 2026
The orchestration frameworks of the time competed on how elegantly they routed agents. The pitch was that none of them checked whether the agents’ work could be trusted: they solved communication, and Orchemist would solve verification.
CURATOR’S NOTE
What held up: checking mattered more than routing, and keeping the writer, the checker and the critic apart worked. What didn’t: the idea that this needed an engine of its own.
ROOM 6 · WHAT IT FOUND
I built Orchemist to learn how the models really behave. These are the three things it taught me.
Every session started from zero. What an agent had learned the week before was gone, unless it was written down and handed back to it.
Groundhog Day, AI edition ↗A model checking its own work found reasons it was right. The checker had to be a different agent, one that never saw the builder's reasoning.
It Scored Lower Because It Stopped Lying ↗Early versions produced confident, wrong content. Every claim needed a source, and every result a check that did not come from the agent that made it.
I Published AI Content Without Challenging It ↗WHAT HAPPENED NEXT
Then the models changed. Newer ones report their own mistakes more honestly, and the tools around them now come with planning, review and parallel agents built in. Much of what Orchemist added from the outside now comes from the inside, which is why I stopped using it.
The checks didn’t retire. Their job moved up: from does the report match reality? to was this the right work?
THE NUMBERS
ROOM 7 · THE ARCHIVE
The essays on connylazo.com, in the order they came out, and the releases.
16 Feb 2026
The Orchestra series, in five parts: running a team of agents, before Orchemist. Setting Up Your Stage
28 Feb 2026
The engine, three and a half weeks in. Building Your Own AI Orchestration Engine
10 Mar 2026
The launch series, part 1: why it had to exist. I Published AI Content Without Challenging It
20 Mar 2026
Part 2: not only code. A Trust Factory for Anything AI Touches
21 Mar 2026
Part 3: specs and tests before the code. When AI Writes the Code, Who Checks the Homework?
30 Mar 2026
The first release on PyPI, 0.8.0.
18 Apr 2026
Part 4: running it. Getting Orchemist Running in 10 Minutes
25 Apr 2026
Part 5: the roadmap. From Factory to Dark Factory
7 May 2026
What agents don't know they don't know. The Agent Memory Wall
22 May 2026
Forgetting as the operating condition. Groundhog Day, AI edition
6 Jun 2026
The models got more honest, and two years of my own systems became half-obsolete. It Scored Lower Because It Stopped Lying
10 Jun 2026
The last release, 0.13.1.
Then
Paused indefinitely. This site became its museum on 11 November 2026.
THE LAST RELEASE
Orchemist 0.13.1, released on 10 June 2026, is still on PyPI under the MIT licence. It is not maintained: no fixes and no support. The source repositories are private.