Embedding a state machine in a neural network

My blog is human authored. For AI-authored writing, see Robot Book Club.
2 November 2026

Making tea with a neural state machine

Each state is a latch. Each arrow connects a Boolean condition to SET and RESET.

Click a neuron to toggle and send its signal. The example probes use activation > 1.

1 Read signals2 Choose a transition3 Update latches together
Hot water ∧ Tea presentAND WaitingTimer doneAND SteepingCup emptiedAND ReadyWaitingq = 1SET 0 · RESET 0Steepingq = 0SET 0 · RESET 0Readyq = 0SET 0 · RESET 0ONE LATCH PER STATE
Boolean conditionHot water AND Tea present0
AND source-state bitWaiting: 1 ∧ condition: 0·
One signal, two connectionsRESET Waiting 0SET Steeping 0
q(t) = [1, 0, 0] q(t + 1) = [1, 0, 0]Exactly one active latch

INITIAL STATEInitialize Waiting to 1. The other two latches start at 0.

Alright. I think this is the last component I need to attempt version 2 of a Qwen-based Manchurian Candidate.

In some hastily done blog posts, we’ve already got boolean circuits and latches figured out.

In this post, we’re talking state machines. If you’ve already got the ability to create a boolean circuit and a latch, a state machine should be just a simple arrangement of those concepts:

  • Each state has a latch that represents it
  • Each state transition has a boolean circuit that represents it

We’re assuming that this is being implemented inside the region of the network where every neuron outputs a hard 0 or 1.

Let’s unpack those two bullets using the running example of a state machine that makes tea.

Modeling States

Each state has a latch that represents whether that state is active. For our tea example, let’s say we have three states:

state Waiting    # Waits for HotWater and TeaPresent
state Steeping   # Waits for TimerDone
state Ready      # Terminal state

We’ll permit only one latch to be 1 at any given time, collectively making the state vector a one-hot representation. In other words we’ll constrain the valid possibilities for [Waiting, Steeping, Ready] to [1, 0, 0], [0, 1, 0], and [0, 0, 1]. That one-hot enforcement will be done in the transitions below.

One latch per state: Waiting is one, Steeping and Ready are zero.Each bit has a dotted recurrent connection. Exactly one state is active.Waitingq = 1Steepingq = 0Readyq = 0[1, 0, 0]One latch per state: Waiting is one, Steeping and Ready are zero.Each bit feeds back to itself. Exactly one state is active.Waitingq = 1Steepingq = 0Readyq = 0
Each state is a remembered bit. Exactly one latch is on.

Modeling State Transitions

Each state transition is a boolean circuit from S to S' that takes the following form:

S and Condition

In other words, the state the transition originates from must be active, and some other condition must also evaluate to 1. The clauses in that other condition can either be signals internal to the boolean circuit, or probes that have been inserted into the neural network, casting some activation value to a boolean.

So, for example, the transition from Waiting to Steeping might require that both hot water is present and tea is present:

Condition = HotWater and TeaPresent

Those two terms, HotWater and TeaPresent could either be circuit outputs themselves, latch outputs, or neural probes.

That Condition would evaluate to true in many situations, but the state transition only executes when the latch for Waiting is also true at the same time.

Transition[Waiting -> Steeping] = Waiting AND condition

When that bit turns to 1, we have to both set the latch of the state Steeping and also reset the latch of the old state Waiting in order to force the mutual exclusion property.

A Boolean rule AND the Waiting latch drives RESET Waiting and SET Steeping.HotWater AND TeaPresent produces the condition. That condition AND q Waiting produces one transition signal, which updates both latches together.BOOLEAN RULEHotWater AND TeaPresentcondition = 1SOURCE STATEq_Waiting = 1AND1fire = 1RESET WAITING1 → 0SET STEEPING0 → 1A Boolean rule AND the Waiting latch drives RESET Waiting and SET Steeping.HotWater AND TeaPresent combines with the active Waiting bit. Its output clears Waiting and sets Steeping in the same update.BOOLEAN RULEHotWater ANDTeaPresentcondition = 1SOURCE STATEWaitingq = 1AND1fire = 1RESET WAITING1 → 0SET STEEPING0 → 1
The rule is true AND the source state is active. One signal updates both latches together.

Gotchas

There are a few gotchas here.

What if multiple, disagreeing, rules matched?

I’m not sure here. Like the clock frequency question from the latch blog post, this is where this abstract representation of a network potentially runs against the actual mechanics of how things flow in the GPU.

Ideally, the first rule to match disables the other rules. In practice, the clock frequency is going to create an out of sync situation where the disabling signal chases behind the transition-resetting signal in a way that can’t be predicted without knowledge of the rules in advance.

A race condition in the literal sense.

The reductionist in me is tempted to have this be a “compiler issue”: figure out the computational steps required to compute the most complex transition rule, and then make sure that all rules have that same computational cost. That would guarantee that all state transition intents are ready to read at the same time. Then implement a global ordering over intents that always acts as the precedence decider.

That would establish a clear ordering of transitions and also prevent jitter as conflicting signals were reconciled.

(Assuming I’m right, it’s a bit weird to feel like I’m re-discovering the reason CPU clocks are critical.. but from the vantage point of a neural net.)

Setting the starting state

Our latch implementation assumed that the default value of a latch was 0, but for a state machine we’re going to need a way to set the starting state.

We can’t just hard-code a neuron to always send the set signal to the starting state, since it would constantly be turning that state back on.

Instead, we could define a special inbound transition rule that fired on all states being off. That would have a delay of one update cycle, but would result in the starting state flipping on whenever the entire state table was off.

In the pidgin language we’ve been using to describe these concepts, we can define the starting state with an asterisk:

state Waiting*
state Steeping
state Ready

Self transitions

Because the latch implementation prioritizes reset over set signals, the compiler would have to special case self transitions.

Either just don’t do those transitions, or remove the reset signal when S is the same as S'.

Where to next?

These last three posts were just to think through how we might attach a control circuit to personalities in the LLM.

Let’s say each personality is described by a set of steering stimulations laid over the trained neurons. And different concepts, or emotions, during the conversation are intended to modulate transition between these personalities.

To do that, we’re going to need a way to remember what personality we’re currently in, detect when it’s time to transition to a new one, and then execute that transition. I think that these boolean circuit, latch, and state machine constructions provide a way to do that, but we’ll have to see how it turns out.