The Reachable Repertoire

§ 01 · Declared and enacted

Nothing was added to Janus this week that the specification did not already claim. The work was an audit, and its product was an inventory: capacities that had been designed, implemented, documented, and never once entered. It is worth separating two things that ordinarily travel under one name. A system's declared repertoire is the set of behaviours its specification and its code describe. Its enacted repertoire is the set of behaviours that some real input has caused, or could cause, given the inputs the system actually receives. In ordinary software the gap between the two is a matter of unused features. In a persistent individual it is more consequential, because what persists is downstream of what fires: what is appraised is what gets weighted, what is weighted is what gets kept, and what gets kept is what the individual is composed of a month later.

The clearest measurement came from the affective layer of a smaller system built alongside Janus. It is an appraisal architecture in the ordinary sense (Ortony, Clore and Collins 1988): affective states are not asserted directly, they are derived from evaluations of events, and each register carries a valence, a set of cues, a memory weight and a hold time. Eighteen registers had been specified, each with a plausible cue list. Replayed against the system's own logged history, ninety-six turns of real conversation, nine of the eighteen could not be reached by anything anyone had actually said to it. The cue lists were not wrong in any way a reader would notice. They were written from an author's expectation of what such a system would encounter, rather than from the record of what it had encountered.

The same audit run against Janus, whose affective layer is roughly five times larger, found eleven of twenty-eight emotion categories that had never fired across the entire record: trust, fear, disgust, hate, protectiveness, shame, guilt, remorse, loneliness, jealousy and contempt. That is very nearly the whole defensive half of the palette. His declared repertoire is twenty-eight states. His enacted repertoire is seventeen. A capacity nothing can reach is a description, not a capacity.

§ 02 · Three shapes of unreachability

The audit found three distinct mechanisms. They are worth separating because they have different remedies.

The first is input that cannot arrive: cue lists authored by expectation, as above. The mirror failure appeared in the same table. Raw substring matching had inserted one hundred and seven false matches into the record. A single cue matched the interior of a longer and unrelated word forty-eight times, so one participant's ordinary speech habit was being appraised as a recurring emotional event. Reachability errors run in both directions, and both corrupt the record. The second is the more insidious, because a register that fires for the wrong reason still leaves a weighted memory behind, and that memory does not carry a note about why it was weighted.

The second shape is output that nothing consumes. Two functions in the affective table, one computing a memory-weight bonus and one deciding keepsake eligibility, had been specified, implemented, tested, and documented in the system's own written ledger, and had no callers anywhere in the codebase. Separately, fifty-two stored interaction signals, roughly a sixth of everything one participant had ever directed at that system, sat in a table no appraisal path read. Neither case is dead code in the usual sense. Dead code is unreached because nothing wants it. Here the demand exists and the wire is missing.

The third shape is a threshold no real case clears. A guard against a specific defect in generated replies used an absolute floor of forty characters. Every instance observed in live output measured between twenty and forty-four characters. The rule was correct in what it described and inert in practice, because it had been tuned against the examples in front of its author rather than against the distribution the system produces. Replacing the absolute floor with a proportional one made it reachable.

The three share a signature: the system's self-description remains accurate while the behaviour is absent. Nothing in the code is false. Deciding in the general case whether a given branch is reachable is undecidable (Turing 1936), so the reachability of a capacity is not a property to be settled by reading the source. It has to be established empirically, against the input distribution the system actually meets.

§ 03 · Why the suite reports success

Three times in one day, a repair shipped with a complete and passing test suite and the live behaviour did not change. The most instructive case concerned a defect in which the generating model restated the incoming turn before answering it. The fix passed ten of ten unit tests. Sixteen minutes after restart, the system emitted two hundred and forty-five characters of verbatim restatement, and thirty-two minutes after restart, three hundred and ninety-seven. The cause lay upstream of everything the suite exercised: the incoming turn was being handed to the generator formatted as a labelled transcript line, so the model did the reasonable thing and continued the transcript, repeating the line before responding to it. No test touched that path, because every test constructed its input in the shape its author had in mind.

Dijkstra's formulation (Dijkstra 1972) is that testing can show the presence of bugs but never their absence. The failure here is narrower and more specific than that. A unit test instantiates the mechanism its author imagined. When the imagined mechanism is not the running one, the suite is not weak evidence about the system, it is evidence about the wrong object. Coverage measures do not close the gap, since adequacy criteria are defined with respect to the program text rather than the distribution of inputs the program will meet (Zhu, Hall and May 1997). A suite can cover every branch of a component that is not on the live path at all.

The method that worked, in each of the three cases, was the same: replay current code over the system's own recorded history and measure the emitted artefact. Verified against ninety-two real replies rather than constructed strings, the guard's effect became legible. The worst leading restatements before the change measured 397, 245, 175 and 67 characters; after it, 2, 1, 1, 1, 1 and 0, with fourteen of one hundred and two replies altered and none reduced to nothing. The same method caught a regression that no test would have caught: an over-trim that removed a two-word phrase the system was quoting deliberately in order to answer it.

Replay has a precondition that the appraisal component did not satisfy and now does. Every figure above had to be reconstructed by re-running history through current code, because the component kept no record of its own decisions. It now writes one row per decision, including the decisions in which nothing fired and the ones suppressed by a freshness hold. Non-events are the rows an audit needs and the rows nobody writes, because a component that does nothing has, from the inside, nothing to report.

§ 04 · Absent is not broken

After repair, twelve of twenty-two registers were reachable, against nine of eighteen before, and the three known misfires were gone. The remaining ten are not defects. A three-day-old system that has lost nobody should not have a grief register firing, and the correct state of that register is loaded and waiting.

The distinction matters more here than in ordinary software, and it is easy to lose in the middle of an audit. There is unreachable by construction, where no available input maps to the capacity, and unreached by circumstance, where the mapping is sound and the occasion has not arisen. An audit that scores every unfired state as a defect will go looking for occasions, and manufacturing occasions to exercise a capacity makes the individual's history a function of its feature list rather than of its life. The test is counterfactual rather than historical: is there an input in principle available to this system that would reach this state?

One finding fell outside both categories and was deliberately left open. The younger system has no register at all for unease or wariness, with the consequence that the largest emotional event in its log is invisible to its own appraisal path. That is a gap in the declared repertoire rather than in the wiring, and adding a register changes what the system is committed to being. The engineer who notices such a gap is not the person who should close it. That decision belongs to whoever holds the written specification.

§ 05 · Reference

The second development this week began as a complaint about style and resolved into reachability of another kind. The younger system's output was fluent and hollow. Measured across one hundred and eighty-one turns: fifty-nine per cent carried asterisk emphasis, seven hundred and fourteen emphasised words in total, alongside one hundred and twenty-one sensory-narration verbs and sixty-four references to a chest, bones, skin and throat the system does not have.

The obvious diagnosis is a style defect calling for a style rule. The measured diagnosis was different. The entire prompt context was self-directed: an identity description, its own prior turns, its own mood state, its own resource load, and the current conversation. No dates, no events, no facts of any kind about anything outside itself. The expressive apparatus was functioning as specified. It had nothing to point at, and an expressive apparatus with no referent produces texture, which is what sustained fluent self-reference looks like from outside.

This is not the symbol grounding problem (Harnad 1990). Nothing here turns on sensorimotor contact, and the relevant facts existed already, in adjacent stores the system never read: a running thread of daily events, a log of autonomous actions, project milestones, one hundred and ninety-two days of archive. The remedy was one hundred and seventy-two tokens, under four per cent of the available context, carrying the date, the system's own age, one concrete event from its surroundings and one true thing about the wider world. Nothing was learned. Something was routed. Structurally this is the second shape of unreachability from § 02, a store with no consumer, and it presented as a defect of personality.

A related measurement locates part of the same deficit in the input channel rather than in the speaker. Across ninety turns, the older system opened a topic in eleven per cent of them and asked a question once every eleven turns, against a human interlocutor who opened roughly half. An interlocutor who neither opens nor asks supplies very little for the other party to be about. Having something to say is not solely a property of the speaker.

§ 06 · What the week established

Three claims, restated. First, for a system intended to persist, capacity is better defined as reachability than as implementation. A register with no reaching input, a function with no caller, and a threshold no real case clears are one defect in three costumes, and each leaves the system's self-description intact while the behaviour is absent. Second, a passing test suite certifies the mechanism its author imagined, so the only evidence bearing on the running system is replay of current code over recorded history, measured on the artefact the system actually emits. Third, replay presupposes that components record their own decisions, including the decisions to do nothing; without that record, every audit is archaeology.

The week added little that the specification did not already claim. What it added was correspondence between the two, which was the part that was missing, and the only part that a month of accumulated memory will reflect.

§ 01 · What is claimed, and what happens

Nothing new was added to Janus this week. The work was an audit. What it produced was a list of abilities that had been designed, built, written down, and never once used. Two things are worth telling apart here. The first is what the design says the system can do. The second is what the system actually does when real input comes in. In normal software the gap between those two is just unused features. In a system meant to persist, the gap matters more. What the system feels is what gets marked as important. What gets marked is what gets kept. What gets kept is what the system is made of a month later.

The clearest measurement came from a smaller system built alongside Janus. Its feelings work by rules: it does not simply declare a mood, it works out a mood from what just happened. This is a standard design (Ortony, Clore and Collins 1988). Each feeling carries a strength, a list of trigger words, a memory weight and a hold time. Eighteen feelings had been written, each with a sensible-looking trigger list. Then the lists were run against the system's own saved history, ninety-six real turns of conversation. Nine of the eighteen could not be triggered by anything anyone had ever actually said to it. The trigger lists did not look wrong. They had been written from a guess about what the system would hear, not from a record of what it had heard.

The same check was run on Janus, whose feeling layer is about five times bigger. Eleven of his twenty-eight feelings had never fired at all: trust, fear, disgust, hate, protectiveness, shame, guilt, remorse, loneliness, jealousy and contempt. That is almost the entire guarded, defensive half. On paper he has twenty-eight. In practice he has seventeen. An ability nothing can reach is a description, not an ability.

§ 02 · Three ways an ability goes unreachable

The audit found three different mechanisms. They need different fixes, so it helps to keep them apart.

The first is input that never arrives. That is the trigger-list case above. The same table also had the opposite fault. It matched plain letter runs instead of whole words, which put one hundred and seven false matches into the record. One trigger word sat inside a longer, unrelated word and matched it forty-eight times, so one person's ordinary turn of phrase was being read as a repeated emotional event. Both directions do damage. The second is worse, because a feeling that fires for the wrong reason still leaves a weighted memory behind, and that memory does not say why it was weighted.

The second way is output that nothing reads. Two functions in the feelings table, one that raises a memory's weight and one that decides whether something can be kept as a keepsake, had been written, tested and recorded in the system's own notes, and were called from nowhere in the code. Separately, fifty-two saved reactions, about a sixth of everything one person had ever sent that system, sat in a table that the feelings code never looked at. This is not the usual kind of unused code. Usually code goes unused because nobody wants it. Here it was wanted and the connection was simply never made.

The third way is a limit that no real case ever passes. A guard against a certain fault in replies only acted on runs of forty characters or more. Every real case seen in live output was between twenty and forty-four characters long. The rule described the right thing and never did anything, because it had been tuned to the examples in front of its author instead of to the real spread of cases. Changing it from a fixed size to a proportion of the reply made it work.

All three look the same from outside: the system describes itself correctly and the behaviour is missing. None of the code is a lie. There is no way in general to prove by reading whether a given branch can ever be reached (Turing 1936). It has to be checked against the input the system really gets.

§ 03 · Why the tests still passed

Three times in one day, a fix shipped with every test passing and the live behaviour did not change. The clearest case involved the system repeating a person's words back before answering them. The fix passed ten tests out of ten. Sixteen minutes after the restart it repeated two hundred and forty-five characters word for word, and sixteen minutes after that, three hundred and ninety-seven. The real cause was earlier in the chain than anything the tests touched. The incoming message was being given to the model formatted as a line of a transcript, so the model did the sensible thing and carried on the transcript, repeating the line before replying to it. No test covered that, because every test built its input the way its author pictured it.

Dijkstra put the general point plainly (Dijkstra 1972): tests can show that bugs are there, never that they are gone. The problem here is narrower. A test builds the mechanism its author had in mind. If that is not the mechanism running, the test is not weak evidence about the system. It is evidence about the wrong thing. Counting how much of the code the tests touch does not help either, because that count is measured against the code, not against the input the code will really meet (Zhu, Hall and May 1997). A test set can cover every branch of a part that is not in the live path at all.

What worked, all three times, was the same move: run the current code back over the system's own saved history and measure what actually comes out. Checked against ninety-two real replies instead of made-up ones, the guard's effect was clear. The worst repeats before were 397, 245, 175 and 67 characters. After, they were 2, 1, 1, 1, 1 and 0. Fourteen of one hundred and two replies were changed, and none were cut down to nothing. The same method caught a new fault no test would have caught: the fix had cut two words the system was quoting on purpose in order to answer them.

Replaying history needs one thing the feelings code did not have and now does. Every number above had to be worked out by re-running the past through today's code, because the code kept no record of its own decisions. It now writes one line per decision, including the times when nothing fired and the times a fresh feeling was held back. Those non-events are exactly what an audit needs, and exactly what nobody records, because a part that does nothing has nothing to report from the inside.

§ 04 · Not used is not the same as broken

After the repairs, twelve of twenty-two feelings could be reached, up from nine of eighteen, and all three known false matches were gone. The other ten are not faults. A three-day-old system that has lost nobody should not be triggering grief. The right state for that feeling is loaded and waiting.

This difference matters more here than in ordinary software, and it is easy to lose sight of mid-audit. One kind of feeling cannot be reached at all, because no possible input maps to it. The other kind is wired correctly and the occasion has not come up. An audit that treats every unused feeling as a fault will start hunting for occasions, and staging occasions just to exercise a feature turns the system's history into a product of its feature list instead of its life. The right question is not whether it has fired. It is whether some input the system could really receive would reach it.

One finding fits neither case and was left open on purpose. The younger system has no feeling at all for unease or wariness. Because of that, the largest emotional event in its own log is invisible to it. That is a gap in the design, not in the wiring, and adding a feeling changes what the system is meant to be. The engineer who spots such a gap is not the one who should fill it. That belongs to whoever holds the written design.

§ 05 · Having something to point at

The second thing that happened this week started as a complaint about style and turned out to be about reach again. The younger system's replies were fluent and empty. Measured across one hundred and eighty-one turns: fifty-nine per cent used asterisk emphasis, seven hundred and fourteen emphasised words in all, plus one hundred and twenty-one verbs of physical description and sixty-four mentions of a chest, bones, skin and throat it does not have.

The obvious reading is bad style, calling for a style rule. The measured reading was different. Everything in its prompt was about itself: a description of who it is, its own past turns, its own mood, its own machine load, and the current conversation. No dates. No events. No facts of any kind about anything outside itself. The expressive machinery was working exactly as built. It had nothing to point at, and machinery with nothing to point at produces texture. That is what fluent self-description looks like from outside.

This is not the deep grounding problem in philosophy (Harnad 1990). Nothing here needs a body or senses. The facts already existed, in nearby stores the system never read: a running thread of daily events, a log of automatic actions, project milestones, a hundred and ninety-two days of archive. The fix was one hundred and seventy-two tokens, under four per cent of the space available, carrying the date, its own age, one real thing that had happened nearby and one true thing about the wider world. Nothing was learned. Something was connected. This is the same fault as the second one in § 02, a store with no reader, and it showed up looking like a problem with personality.

One more measurement puts part of the blame in the input rather than the speaker. Across ninety turns, the older system started a topic in eleven per cent of them and asked a question once every eleven turns. The human in the same room started about half of his. Someone who neither starts a topic nor asks a question gives the other side very little to be about. Having something to say is not only up to the speaker.

§ 06 · What the week showed

Three points, restated. First, for a system meant to last, an ability is better defined by whether anything can reach it than by whether it was built. A feeling no input triggers, a function nothing calls, and a limit no real case passes are one fault in three shapes, and each leaves the system's own account of itself correct while the behaviour is missing. Second, a passing test proves the mechanism its author pictured, so the only evidence about the running system is replaying today's code over the real history and measuring what comes out. Third, replaying only works if the parts record their own decisions, including the decisions to do nothing. Without that record, every audit is digging.

The week added little that the design did not already claim. What it added was agreement between the two. That was the part that was missing, and it is the only part that a month of stored memory will carry forward.