Teaching CrowdRelay Autopilot to learn from its own decisions

I thought predictable, deterministic Rust state machines were enough. I was wrong.

They made decisions reproducible. They could enforce cooldowns, refuse unsafe actions, remember deadlines and explain which rule fired. They couldn't learn which decisions were worth making.

I didn't need more automation that waited for us to improve its thresholds. I needed a system that learned and decided for us, while staying more accountable than an LLM with credentials.

That shift has turned CrowdRelay Autopilot from a collection of reliable decision machines into a working brain. The Rust is still deterministic and the authority boundaries are still explicit. What changed is that the evaluated state now includes beliefs, uncertainty, experiments and memory, and those beliefs change when reality disagrees.

Where the state machines fell short#

The first Autopilot answered questions such as: is a Reddit scan due, is an event close enough to pitch press, has this audience already been contacted, may this class of action execute without approval?

Those are good state-machine questions. The same input should produce the same answer, and a safety invariant shouldn't become probabilistic.

The weakness was one level above them. A scanner, a community post and a Signal invite could all be eligible at the same time. The old system knew each action was allowed. It didn't know which one was likely to create an incremental fan, which uncertainty was worth buying down, whether two actions were reaching the same people, or whether waiting for an existing measurement was better than doing more.

Another if statement could encode my latest opinion. It couldn't turn yesterday's outcome into a better decision tomorrow.

One picture of the world#

Once per cycle the brain builds a world model: fan growth and its direction, Signal conversion, the configured north star, community reach, the outreach pipeline, upcoming events and progress against a growth target.

Without uncertainty that would just be a dashboard. Besides asking what is happening, the brain asks what it currently believes an action will cause in that situation.

A press pitch near a show is treated as a different intervention from a press pitch with no event ahead. A community post into an unfamiliar scene isn't assumed to behave like the tenth pass through an exhausted audience. Context includes event proximity, growth trend, community type, format, time and novelty.

From that state it derives a strategy (aggressive discovery, event-driven, content-first or Signal conversion), with hysteresis so a boundary crossing doesn't change the whole posture every cycle. The strategy provides a deterministic frame. The learned model estimates what the opportunities inside that frame are worth.

Predictions before outcomes#

A learning system needs a written prediction. Without one, every result can be explained afterwards.

Before CrowdRelay dispatches work, it records how many incremental fans and Signal installs it expects, the context behind that expectation, the active strategy, the audience, the treatment assignment and the evidence quality. The candidate, prediction and initial evidence are persisted together, so the prediction can't be quietly replaced after the answer is known.

The fan-count model is a hierarchical Gamma-Poisson model, because count data isn't a neat normal distribution. Templates with little history shrink toward the global belief instead of becoming heroes after one lucky result. Learned context effects adjust that belief without a pile of permanent multipliers. Signal conversion is learned separately, because installing an application is a different outcome from becoming a fan.

Every prediction also carries uncertainty. 2.4 fans based on two observations is a much weaker claim than 2.4 fans based on two hundred. The old state machine had a value. The brain has a belief about the value and a statement of how little it may know.

The posteriors are checkpointed in PostgreSQL. A restart loads the last brain state and replays only newer evidence; a broken checkpoint falls back to the evidence history. Learning is durable business state, kept in the database rather than in one process's memory.

Outcomes versus causation#

This was the uncomfortable part. If CrowdRelay posts into a community and five fans arrive, it's tempting to teach the model that the post created five fans. It may have created none. A release, another campaign or ordinary growth may have done the work.

So the brain separates raw outcomes from incremental effect. Eligible direct actions can enter experiments with deterministic treatment and holdout assignment. Control units are recorded as deliberate non-actions rather than fake actions with zero results. The experiment population is formed before the portfolio chooses its winners; otherwise the comparison would only describe candidates the brain already liked.

Evidence also records how it was obtained. A randomized holdout moves a belief more than a before-and-after comparison. Direct attribution, matched comparisons, modeled attribution and observational evidence each carry different strength. Sample size, contamination and measurement delay can weaken even a nominally strong method.

The learning target is incremental unique durable fans: people who wouldn't have arrived without the action, deduplicated across channels and still active after thirty days. The number that looks best in a weekly report doesn't count.

Thirty days is a long time for a decision loop, so the brain learns a bridge from the fourteen-day signal to the thirty-day result. Until there are enough paired observations, that bridge inflates uncertainty instead of passing off an early guess as a mature fact.

What to learn versus what to do#

One of the first versions mixed everything into Expected Free Energy: expected fans, uncertainty, information gain, novelty and risk. The equation had several terms, so it looked intelligent. It was also turning into a weighted soup where quantities with different meanings could make a bad action look good.

The current boundary is stricter. Expected Free Energy decides what is worth learning about. It gives uncertain or unexplored template-and-context combinations a reason to enter the candidate pool. Exploration memory makes repeated visits less novel, lets knowledge generalise partially across templates and decays old visits so a context can become interesting again when the world changes.

DecisionValue decides what is worth doing. Its economic terms are expressed in one unit: expected incremental durable fans at day thirty. It preserves where the estimate came from, its sample size, evidence quality, uncertainty, bridge confidence, contamination and calibration. Unknown risk is labelled unmodelled rather than smuggled in as zero.

Curiosity may justify measuring an uncertain action. It must not silently rewrite the action's expected fan value.

Choosing a portfolio, including waiting#

The old evaluator considered each bounded context separately. That was tidy locally and naive globally. Two individually sensible actions can hit the same audience, consume the same budget and produce less together than either model predicts alone.

The brain now collects eligible candidates into one portfolio and compares their marginal value after audience overlap, fatigue and resource limits. A community engager can be evaluated per actual community, so r/djent and r/metalcore don't disappear inside one generic worker score.

Doing nothing is a candidate. So is waiting. If several measurements are about to resolve and the best current action has weak expected value, another dispatch may be less useful than learning from work already in flight. Silence can now be the selected decision, rather than only the absence of a matching rule.

The portfolio also records why candidates lost: negative marginal value, insufficient value, exhausted budget or a better candidate for the same audience. Six months from now I want to be able to inspect why the brain chose an action, and a confidence number on its own wouldn't tell me.

Where LLMs fit#

As the brain became more capable, LLMs became less important. There's no language model in this decision path. LLMs scan sources, gather intelligence and draft material. CrowdRelay builds their prompts from owned data, checks their output and measures what happened after execution. A worker can be slowed down, moved to a cheaper tier or retired when its outcomes keep disappointing. Operator approvals and cancellations can adjust how often a type of work is surfaced, but preference isn't allowed to override economic value.

That's the opposite of giving an agent a goal and hoping its chain of thought resembles strategy. The models are replaceable workers. The Rust brain owns the goal, evidence, experiment, budget, authority and final choice.

Determinism was never the mistake; confusing deterministic execution with fixed reasoning was. A Bayesian update is deterministic given the prior and evidence. A holdout assignment can be reproducible. A portfolio can make the same choice from the same beliefs while those beliefs keep changing with observed reality.

Known limits#

The priors start sparse. Thirty-day outcomes necessarily arrive late. Some channels offer clean attribution and others only weak comparisons. Risk is still explicitly unmodelled in the canonical decision value. Waiting currently understands pending measurements, uncertainty and immediate opportunity cost; its longer-term option value isn't modelled yet. The high-level strategy posture is still rule-derived, and learned strategy evidence isn't permitted to manufacture fan value. I'd rather state these limits than hide them behind a larger model.

Not every decision is good yet, and the subsystems' biological names don't make it a brain. What does is that the loop is closed and durable:

The original state machines still guard every edge of that loop. They decide what is legal, due, idempotent and authorised. The brain decides what deserves our next unit of attention, can spend an action on learning instead of exploiting, and can find out that its previous answer was wrong.

That was the missing part. I needed CrowdRelay to remember what we tried, learn what actually grew the fanbase, and make the next decision without us standing in the middle of every loop.