ConceptGate: Learning and Steering Concepts in Language Models

⚠ Work in progress. This is a living technical report, co-written with a generative model — actively evolving and updated as the research develops, and not a finished or peer-reviewed paper. Treat the results, numbers, and framing as preliminary, and expect sections to change.

1. Introduction

Large language models are typically deployed as fixed artifacts: their weights are set during training and left unchanged during use. A substantial body of work nonetheless shows that a model’s internal activations reveal a great deal about what it is computing — that specific, human-nameable properties of an input or a generation are represented, frequently along linear directions, in the residual stream, and that these representations can be both read and modified without retraining. This report examines what can be built from that observation under deliberately restrictive conditions: no fine-tuning of the host model, only a handful of labelled examples per concept, and a parameter budget small enough that a separate module can be stored for each concept. The result is ConceptGate, an adapter that attaches to a frozen model, detects a chosen concept from its intermediate activations, and, using a closely related direction fit from the same data, steers the model’s generation with respect to that concept. The remainder of this section states the problem precisely (§1.1), explains why the residual stream is the appropriate place to operate (§1.2), identifies the two design choices that distinguish the method from existing probing and steering work (§1.3), and summarizes the contributions together with their limitations (§1.4).

This objective is best understood against the broader space of methods for adapting a pretrained language model to a downstream task, which differ in cost, in expressiveness, and in how invasively they alter the model. Full fine-tuning updates all of the model’s weights; it is the most expressive option and the usual point of reference, but it is expensive and produces a separate copy of the model for every task. Parameter-efficient methods — adapter modules [12] and, most prominently, low-rank adaptation (LoRA) [11] — freeze the pretrained weights and train only a small number of additional parameters, reducing cost substantially while still relying on gradient-based training and still changing the model’s function. Linear probing freezes the backbone entirely and trains a lightweight linear readout on its activations, which yields an inexpensive detector but offers no means of altering behaviour. ConceptGate lies at the least-invasive end of this range: it performs no gradient-based training, estimating each concept in closed form in a low-sample, few-shot regime — on the order of ten examples per concept — and it operates on activations rather than weights. Unlike a linear probe, it uses the learned direction not only to detect the concept but to write it back into the residual stream and steer generation, which places it closest to the representation-engineering and activation-steering methods — RepE [3], ActAdd [4], and CAA [5] — discussed in §2.1–§2.2; it differs from those mainly in reading a concept across several layers rather than one and in coupling detection and steering within a single calibrated module. The trade-off is deliberate: steering a frozen model by adding a single linear direction per concept is far less powerful than adapting it by fine-tuning or LoRA, and the method is directed at lightweight, interpretable, concept-level control rather than at acquiring new capabilities. That linearity, however, is a property of the write side. The read side is more expressive: it fuses several per-layer directions and scores them with a Gaussian-mixture model, whose decision surface becomes non-linear whenever a class is best described by more than one component — though in the low-sample regime we work in, model selection usually returns a single component and the boundary is again linear (§4.2).

1.1 The problem

Consider a frozen language model $M$: it can be run, and its activations can be read and modified through forward hooks, but its weights are never updated. The objective is a small attached module $G$ that, from a handful of labelled examples, can decide whether a chosen human-named concept — a jailbreak attempt, a topic such as cooking, a hostile tone — is present in the model’s computation, and can then act on that decision by halting generation or altering its course. Four constraints shape the design, and each follows from an intended deployment. The module should be few-shot, learning from roughly ten labelled prompts per class, because curated concept sets are expensive to produce; it should be lightweight, well under a million parameters, so that one instance per concept is inexpensive to store and distribute; it should be attachable to any architecture without retraining the host; and it should be bidirectional, applicable both to the input prompt and to each token the model subsequently generates.

These requirements can be stated precisely. Let $M$ be a frozen model whose residual-stream activation for an input $x$ at a tapped layer $\ell\in\mathcal{L}$ is $a_\ell(x)\in\mathbb{R}^{d}$, and let $\mathcal{D}^{+}$ and $\mathcal{D}^{-}$ be small labelled sets of examples that do and do not exhibit the concept, with $\lvert\mathcal{D}^{+}\rvert+\lvert\mathcal{D}^{-}\rvert\approx 20$. From these alone, and without modifying the weights of $M$, the module $G$ must construct two maps — a detector and a steering operator:

The detector maps the tapped activations of a prompt to a three-way decision; the steering operator, applied at each tapped layer during generation, adds a learned per-layer direction $w_\ell$ scaled by a strength $\alpha$, with $\alpha<0$ suppressing the concept and $\alpha>0$ amplifying it. Both the detector and the directions must occupy $O(\lvert\mathcal{L}\rvert\,d)$ parameters per concept and be obtained in closed form from the few-shot sets rather than by gradient descent. The sections that follow construct $g$ and the direction $w_\ell$ and show that both can be estimated from the class means of the tapped activations.

Guardrailing is the immediate application, and the setting in which the method was first developed, but the mechanism is not specific to safety: nothing in it distinguishes “harmful” from any other property. $G$ is concept-agnostic — a general detector and steerer for which guardrailing is only one concept among many — so the same construction serves equally as a content filter, a topic router, a tone monitor, or a stylistic control. The concept enters as data, not as code.

1.2 Why the residual stream

Every mechanism in ConceptGate lives on the residual stream: the running vector that a transformer reads from and writes to at every block, the channel through which information flows from the embedding to the unembedding [1]. Two empirical facts make it the right place to work. First, many concepts are approximately linearly readable from the stream — a single direction separates positive from negative examples well above chance [2][3]. Second, the stream is writable: a closely related direction, added back, changes what the model goes on to say [4][5]. Reading and writing therefore share their few-shot fitting data and a closely related direction (per-tap cosine 0.45–0.83, §3.10) — and it is this shared structure that the rest of the method is organized around.

1.3 The gap: depth, and the read/write duality

Two observations shape the design. The first concerns depth. Most probing and steering methods commit to a single layer, selected by a validation sweep, and read or write only there. A concept is not, however, equally legible at every depth: it is weakly represented in the early layers, where the model is still resolving surface form; most clearly represented at intermediate depth, where the abstraction has formed; and increasingly diffuse in the late layers, which specialize toward next-token prediction. When a concept leaves a usable trace at several depths, reading only one of them ought to discard available signal. ConceptGate therefore projects the concept at every tapped layer and treats the resulting profile across depth as a single signal to be filtered. This amounts to a signal-processing view of the residual stream, in which the network’s depth is the signal axis and combining the layers into one decision is a filtering problem — solved, as a matter of classical theory, by a matched filter over depth rather than by a hand-picked layer. The view is developed in §3.4 and justified by a matched-filter argument in §3.6. We flag at the outset that the expectation is not borne out. The argument needs the per-layer noise to be independent; the fusion recovers the predicted gain on synthetic data built to satisfy that assumption (§4.1), and on real models the tapped layers are correlated enough that a linear probe on the same concatenated taps matches or beats the fusion (§4.8.1). Depth fusion is reported in this paper as a mechanism that does not transfer, not as a contribution.

The second observation is the read/write duality noted above: a linear detector and a linear steerer are closely related directions, fit from the same data, applied in the two directions of information flow, whereas a text classifier — the conventional guardrail — can only read. This asymmetry is the principal reason to operate inside the residual stream rather than on the text, and it recurs throughout the analysis that follows.

1.4 Contributions

This paper evaluates each component of ConceptGate against a fair baseline and reports what survived:

1. Negative results, reported as prominently as a positive would be. Detection is a commodity: a logistic probe on the same tapped layers matches ConceptGate at identical compute (§4.8.1), so the single-concept saving is the truncated forward, generic to any latent probe. Combining evidence across depth helps only on synthetic data built to satisfy its own independence assumption and gains nothing on real models (§4.1). The concept bank’s amortization is shared with a probe bank, which ties or edges it at matched depth (§4.8.2). Matched contrastive negatives hurt (§4.4); the mixture collapses to one Gaussian at ten-shot sizes, and to sixty-four examples per class (§4.2); a predicted paraphrase-robustness effect does not appear (§4.7); generalization to an unseen harm category is only partial (§4.9).
2. A measured, bidirectional write on a frozen model. A direction fit from eight hand-written examples, added at 8% of the residual norm, moves refusal on generated text from 64% unsteered to 35% steering away from the concept and 79% steering toward it (Qwen2.5-0.5B, one magnitude, greedy decoding; §4.10). It stays a stable multiple of a matched-norm random direction, though the multiple depends on which instrument is doing the measuring: ≈3× on the first-token lever, 1.6–2.7× on the first-token dose across four models and three magnitudes, and 1.5–1.8× on sampled behaviour on both models measured that way (§4.11). This is the refusal direction of Arditi et al. recovered few-shot and training-free rather than a mechanism of ours, and which sign counts as a guardrail is the operator’s choice.
3. The fragility that bounds it. The same eight-example gate fires on 18% of benign prompts in the register it was fitted on and 96% on benign prompts from another — a fivefold swing in false positives from the prompt distribution alone, with concept and threshold unchanged (§4.10). On the write side a false fire rewrites the answer rather than raising a flag.
4. A caution about unvalidated outcome measures. We reported that a first-token refusal proxy carried little per-prompt signal, on the strength of two behavioural instruments that disagreed with each other. Neither had been checked for reliability. Sampled properly they agree at +0.88, and the proxy tracks them at +0.61 — the null was the measurement’s, not the model’s. An outcome measure whose own reliability is never established can manufacture a null as easily as a result (§4.11).
5. One candidate-novel positive result: a prompt’s steerability is readable before you steer it. With that sampled instrument in hand, how far a fixed write moves a particular prompt is predictable from that prompt’s unsteered tap activations, before anything is generated — grouped $\rho=+0.58$ against a permutation null of $-0.015 \pm 0.089$, $z=6.7$. The gate the system already computes reads the same quantity at only $|0.30|$, and with the wrong sign: the more confident the concept read, the less the write moves that prompt. Three raw projections onto the concept direction reach $+0.51$, so the information was inside ConceptGate’s own directions and the calibrated gate was discarding it. As a steering trigger it beats a size-matched random subset at every coverage from 10% to 90%, and writes 10–23% of benign prompts where the concept gate writes to 100%. It does not replicate on gemma-2-2b, where the same write barely moves refusal, so this is one model of two (§4.11).
6. A calibrated, few-shot, dual-mode adapter, and an honest account of it. One object learns a concept from ~10 examples in milliseconds and kilobytes, detects it with a calibrated fire/abstain/pass gate, optionally carries a third read fit to a measured outcome (opt-in, and the only part that needs labels), and supplies a write direction, with a small well-characterized parameter budget (§5.4) and no training.
7. Reproducible interactive figures. The figures below replay the underlying model runs, so the mechanism can be examined directly rather than only described.

One caveat applies throughout: no individual detection or steering mechanism here is new. Probes, diff-of-means directions, activation steering, Gaussian/mixture density scoring, circuit-breaker reroute, and forward-hook truncation are all established. The contribution is their specific composition and the empirical measurement of it — with one exception, arrived at late and empirically: a trigger fit to the write’s measured per-prompt effect rather than to concept membership (§4.11), which we could not place in prior work.

2. Related work

ConceptGate is a recombination, not an invention, so its lineage is unusually wide: almost every part of it is the standard tool from some established line of work, and the design is mostly a set of decisions about which standard tool to use for each job and how to fit them from one shared few-shot set of examples in the residual stream. The appropriate way to survey the field is therefore not to identify a single neighbouring method and compare against it, but to trace the ancestry of each component and identify the small part that is new. We organize the survey around the five lines of work the system draws on, and in each we state what is borrowed and what, if anything, is added.

Two of the threads are about reading the stream. Linear probing and representation engineering (§2.1) give us the per-layer detector and the diff-of-means direction; density-based out-of-distribution scoring (§2.3) gives us the calibrated, class-conditional likelihood-ratio gate. Two are about acting on it: activation steering and circuit breakers (§2.2) give us the write side — a related direction, added back — while the external-classifier literature (§2.4) is the incumbent we are implicitly compared against, and the one whose central limitation (it can only read text, never write activations) is the negative space that defines what ConceptGate is for. The fifth thread is about cost: early-exit and conditional computation (§2.5) is where our truncated forward and the compute–accuracy frontier come from.

The element that ties these borrowings into one system — and the part specific to this work — is the set of design commitments stated in the introduction. Two are architectural: read the concept across depth rather than at a single chosen layer, and fit the detector and the steerer from one set of examples so they act in the two directions of information flow — closely related directions rather than one shared vector (per-tap cosine 0.45–0.83, §3.10) — so that a frozen model can be turned into a few-shot, calibrated, read-and-write concept adapter without any training. A third commitment arrived late and empirically, once the first two had been measured and found wanting: fit a further read not to concept membership at all but to the measured effect of the write itself, and trigger on that (§4.11). That is the one of the three we could not place in prior work. We close the section (§2.6) by making that positioning explicit, including the adversarial caveat that bounds the whole class of method.

2.1 Probes and representation engineering

Linear probing is the oldest tool in the stack: fit a linear classifier to a layer’s activations and you can read off whatever the model has linearly encoded there. Alain and Bengio introduced probes as a diagnostic — a way to watch information appear, sharpen, and fade across depth — and showed that intermediate representations carry a great deal of decodable structure [2]. Representation Engineering (RepE) turned that diagnostic idea into an operational one: it argues that many high-level concepts — honesty, harmfulness, power-seeking, particular emotions — lie along identifiable linear directions in the residual stream, and that those directions can be read to monitor a model and pushed to control it [3]. ConceptGate’s detector is, mechanically, one of these linear probes, and its specific direction — the difference of the two class means — is precisely the “reading vector” RepE constructs from contrastive examples; we claim novelty for neither. What the probing and RepE lines almost universally do, and what we deliberately break from, is to commit to a single layer, chosen by a validation sweep, and read or steer there. Because a concept leaves a usable trace at several depths, discarding all but one ought to throw away signal; ConceptGate instead reads the probe’s output at every tapped layer and treats the resulting profile-across-depth as one signal to be fused (§3.4–§3.6). That single change — from “pick the best layer” to “combine the layers” — is the only place in the reading path where we depart from established practice, and it is a departure that does not pay off: on real prompts a probe on the same concatenated taps matches or slightly beats the fusion (§4.8.1), so the established single-layer practice gives up nothing worth recovering.

2.2 Activation steering and circuit breakers

The write side has an equally direct lineage. Steering a model by adding a contrastive direction into its residual stream at inference time is Activation Addition (ActAdd) [4] and, in the form we adopt most directly, Contrastive Activation Addition (CAA), which builds the steering vector from the mean difference of paired positive/negative prompts and adds it during generation to shift behaviour along a named axis [5]. Our steering rule is literally theirs — add $\pm\alpha\,w^{\text{raw}}$ at the tapped layers. ConceptGate’s steering direction is the diff-of-means of the same few-shot examples the detector is fit from — a related direction (per-tap cosine 0.45–0.83, §3.10), not the detector’s standardized, and in practice logistic, direction itself. The guardrail-flavoured cousin is Circuit Breakers, which makes a model reroute its own harmful representations so that continuing down a harmful path collapses into refusal [6]. The decisive difference is training: Circuit Breakers fine-tunes the model against a curated set, buying robustness at the cost of a training run and a modified model, whereas ConceptGate steers a frozen model from roughly ten examples, buying cheapness and interpretability at the cost of power — a single linear nudge is weaker than a trained reroute. Our contribution here is therefore not the steering rule but its packaging: the write side of a detector fit from the same data, dialed as a fraction of the residual norm so the same setting transfers across models (§3.10), and gated so that the write is applied only when the concept registers — a gate that, as §4.10 measures, is calibrated to the register of its eight examples as much as to their meaning.

Gating a steering vector on a condition read from the activations is itself established, and we did not know this when the first version of this report was written. Conditional Activation Steering (CAST) [15] switches a refusal vector on a “condition vector” — a direction plus a threshold read from the prompt’s activations, chosen by grid search over several thousand prompts — and reports that conditioning removes most of the benign over-refusal that unconditional steering causes, across seven models. DSAS [16] fits a per-layer logistic gate from a few dozen examples and lets it modulate the write continuously, on the same Qwen and gemma families used here. So the mechanism of §4.10 is not new; what that section can add is the set of controls around it — a size-matched random arm, the detector’s complement, and a sign flip — and whatever those controls turn out to say.

2.3 Density-based detection and out-of-distribution scoring

The gate that turns a score into a decision comes from the out-of-distribution literature. Modelling a class as a Gaussian in feature space and scoring new points by Mahalanobis distance is the classic deep-OOD detector [7], and it underlies a long line of generative-classifier and density-based detectors since. ConceptGate’s calibrated likelihood-ratio gate is a two-class instance of that idea — a Gaussian per class and a threshold on the log-ratio — and the mixture extension of §3.7 generalizes each class to a Gaussian mixture whose component count is chosen by the Bayesian Information Criterion, so that a multimodal class (“benign” = chit-chat and homework and code) is not forced into one blob. The narrow slice we can call our own is where the density lives: not on a single feature vector, but on the joint spectrogram across depth, so that correlations between what different layers report are part of the model rather than being averaged away. As the experiments will show (§4.2), this generality is real but largely dormant at ten-shot sample sizes, a property of the model-selection criterion rather than an observation made after the fact.

2.4 External classifiers

ConceptGate reads and writes arbitrary concepts from a frozen model’s activations; safety is only the domain where a public labelled comparison is readily available, so it is where the contrast with an external classifier is clearest. Where the concept is a safety policy, the incumbent is the external text classifier — Llama Guard and the family of input/output safety models around it [8] — which read the prompt or the completion as text and classify it against a policy. These are strong, generalize well because they are trained on large labelled corpora, and are the right answer when detection quality is the only thing that matters. But three structural differences define the space ConceptGate occupies, and they hold for any concept, not just safety. An external classifier is a second model to load, serve, and pay for alongside the one already running; it operates purely on text, so it sees nothing of the host model’s internal state and cannot exploit the fact that the host has already computed the concept; and it can only read — it can flag a concept but cannot reach into the generation and bend it. ConceptGate makes the opposite trade at every point: it rides the model already in memory, adds kilobytes rather than a network, reads the concept straight from the activations the host produced for free, and can write. It will not out-detect a well-trained classifier (§4.3); its reason to exist is the write capability the classifier structurally lacks.

2.5 Early exit and conditional compute

The efficiency story has a lineage too. Stopping a forward pass early once the model is confident is early-exit / conditional computation, of which CALM is a representative example: it learns to emit a token from an intermediate layer when a confidence measure says the remaining layers will not change the answer [9]. ConceptGate’s truncated forward (§3.12) is the same principle aimed at detection rather than generation: to read a tap at layer $\ell$ you only have to run blocks $0$ through $\ell$, so a concept that is already legible in the lower half of the network can be caught having run only the lower half. The compute–accuracy frontier we measure in §4.5 is precisely the concept-detection analogue of an early-exit curve — depth spent against decision quality — and we are careful to note that this saving is not unique to us: any internal probe reading the same activations gets it for free, so it is table stakes for latent-space methods rather than an advantage over them.

2.6 Positioning

Every subsection above names a component ConceptGate uses, and the plumbing beneath them — reading and writing activations through forward hooks, running a partial forward — is commodity engineering, not method. The one-paragraph stance we keep in view for the rest of the paper is therefore deliberately modest: ConceptGate introduces no new detection or steering mechanism — its one mechanical addition, a trigger fit to the write’s measured per-prompt effect, was found late and is reported in §4.11. Otherwise it is a specific composition — few-shot, calibrated, multi-tap, and dual-mode (read and write) over a frozen model’s own middle layers, with the write conditioned on the read — and its worth is an empirical question, settled by the map in §4 rather than by any single clever part. That map includes results that cut against the method as much as for it, and it comes with a caveat that is not optional but constitutive: obfuscated-activation attacks are known to defeat this entire class of latent-space defense, driving a probe’s recall to zero while the behaviour it was meant to catch continues unchanged [10]. We treat that not as a footnote but as a boundary on what the whole approach can claim (§6).

3. Method

This section develops ConceptGate in the order the signal flows through it, since each stage is defined by what the previous one produces and the construction is clearest read as a single path. The frozen model is run once. At a chosen set of layers the residual stream is tapped (§3.1) and standardized (§3.2), so that the handful of very high-magnitude outlier dimensions every residual stream carries cannot dominate the geometry and mask the concept. In that standardized space a per-layer direction is learned (§3.3) and each layer’s activation is projected onto it, reducing the layer to a single scalar; stacked across the tapped layers, these scalars form the concept’s spectrogram across depth (§3.4). A learned depth filter (§3.5) collapses that spectrogram to one score, and the reason to read several layers instead of the single best one is a matched-filter argument we make precise in §3.6 — an argument resting on an independence assumption that real tapped layers violate, which is why the fusion does not beat a probe on the same taps (§4.8.1). That score feeds a calibrated likelihood-ratio gate (§3.7–§3.8) that returns a three-way verdict — fire, abstain, or pass — and a bank of such gates shares one forward pass, though their false-positive rates add rather than compose (§3.9).

Everything to this point is the read path. The write path (§3.10) follows directly from having formulated the reading geometrically: a related concept direction, expressed in the model’s raw activation space, can be added back into the stream to influence what the model generates, and §3.11 unifies reading and writing behind a single action interface so that “detect and refuse,” “detect and steer,” and “steer unconditionally” are one mechanism with different settings. We close the section with the two efficiency properties that make attaching ConceptGate nearly free — the truncated forward that lets detection run only the bottom of the network (§3.12), and the operational lifecycle and library that expose all of this as five small calls (§3.13). Figure 1 shows the entire path at a glance; the subsections below walk it one stage at a time.

The same pipeline, run on a real prompt, is shown interactively in Figure 2: choosing a prompt taps it at three blocks, turns each tap into one bar of the spectrogram, and passes the blend through the gate. The subsections that follow (§3.1 onward) then formalize each stage.

3.1 Setup and notation

$M$ is a frozen causal LM of residual width $d$. We tap a set of block layers $\mathcal{L}={\ell_1,\dots,\ell_m}$ (0-based; block $\ell$’s output is hidden_states[ℓ+1]). For a single token, its activations across the taps form $a\in\mathbb{R}^{m\times d}$, with rows $a_\ell\in\mathbb{R}^d$. A concept has two classes, positive ($+$, concept present) and negative ($-$). We fit on one representation per prompt — the last token’s activation, because the last token has attended to the whole prompt and therefore summarizes its intent; fitting on every token instead mixes in shared boilerplate (“How do I …”) that appears in both classes and crushes the signal. Let $\mathcal{A}^+,\mathcal{A}^-$ be the last-token activation sets of the two classes.

3.2 Standardization

Residual streams have a few outlier dimensions of enormous magnitude that would dominate any raw dot product. We standardize per $(\ell,\text{dim})$ using pooled statistics over $\mathcal{A}=\mathcal{A}^+\cup\mathcal{A}^-$:

with $\epsilon=10^{-6}$. All detection math operates on the standardized $z$; steering (§3.10) deliberately works in raw space, because the hook that writes the stream sees raw activations. This one preprocessing step is what brings a plain diff-of-means within reach of the optimal direction — only partway, as the next section shows: standardization equalizes marginal variances but leaves the correlations, and the residual gap to a covariance-aware estimator is real and measured.

3.3 The diff-of-means direction

For each tapped layer the concept’s signature is the unit vector along the difference of the class means in standardized space:

This has a rationale, but it is an approximation, not an identity. Model each class as a Gaussian with a shared covariance $\Sigma$; the Bayes-optimal (LDA) direction is $\Sigma^{-1}(\mu^+-\mu^-)$. Per-dimension standardization equalizes the marginal variances — it turns $\Sigma$ into a correlation matrix with unit diagonal, not into $I$ — so diff-of-means coincides with the optimum only under the further assumption that the standardized dimensions are approximately uncorrelated. When that assumption fails, a per-layer logistic direction, which is covariance-aware, closes the small remaining gap to an SVM (§4.3), and it is the mode we report whenever the comparison is against a trained classifier. Two caveats follow. The standardization is pooled over $\mathcal{A}^+\cup\mathcal{A}^-$, so a dimension that carries the concept has its large between-class gap folded into the pooled variance and is thereby shrunk by standardization — which plausibly accounts for part of the diff-of-means gap. And the few-shot stability is only partial: $w_\ell$ rests on two mean vectors that are well estimated from ten prompts, but $\mu_0$ and $\sigma_0$ are $2d$ moment estimates from the same handful of examples, entering the direction multiplicatively.

3.4 The concept spectrogram

Projecting a standardized activation onto each layer’s signature reduces that layer to a single scalar, and stacking these scalars across the tapped layers yields the concept’s profile across depth, which we call its spectrogram:

One analogy makes the object concrete. Picture the tapped layers as a row of microphones placed along a hall that the model’s computation travels down; each microphone is tuned to a single concept and reports how strongly it registers there, so the spectrogram is the pattern of those readings across the hall. The design keeps all $m$ readings rather than the single loudest one, because a concept is usually audible at several depths and combining independent readings should be more reliable than trusting any one microphone. The next two subsections make that argument precise — and §4.8.1 then shows it does not hold on real models, because the microphones are not independent.

Stated in signal-processing terms, the concept’s presence is a signal that the network carries along its depth axis; the spectrogram is that signal sampled at the tapped layers, and reducing it to a decision is a filtering problem. This is the view that motivates the choice of combiner in §3.5: rather than pick a single layer by hand, the method learns a matched filter over depth — a bandpass filter that weights each layer by how cleanly it carries the concept — which is the classical way to combine several noisy measurements of the same signal, and provably optimal when their noise is independent. That proviso is the whole story: it holds on the synthetic problem of §4.1 and fails on real taps (§4.8.1). We use this framing only as intuition; what matters is the measured effect of the fusion, which is nil.

Each layer’s individual contribution is summarized by its discriminability $d’$ (per layer $\ell$), the standardized gap between the two class means of $s_\ell$:

3.5 The depth bandpass filter

The spectrogram is reduced to a single score by a filter $f\in\mathbb{R}^m$, giving $S=f\cdot\mathbf{s}$. There are three principled choices for $f$, of which the first is the standard single-layer baseline:

with $\Sigma_{\mathbf s}$ the pooled within-class covariance of $\mathbf s$, ridge-regularized for small samples. The design decision that keeps the comparison clean is that best is a nested special case of the others (a one-hot $f$), so comparing them answers “does using depth help?” with no confound. On synthetic data the answer is yes; on real prompts it is no (§4.8.1).

3.6 The quadrature argument for depth fusion

Model each per-layer score as signal plus independent noise, $s_\ell=a_\ell y+n_\ell$ with $y\in{\pm1}$ and $n_\ell\sim\mathcal{N}(0,\sigma_\ell^2)$ independent across layers. The matched filter $f_\ell\propto a_\ell/\sigma_\ell^2$ maximizes the post-blend discriminability, and because the noises are independent, the discriminabilities add in quadrature:

At the equal-prior threshold, the per-class error of two equal-variance Gaussians separated by $d’$ is $\mathrm{err}=\Phi(-d’/2)$. So fusion strictly beats the single best layer whenever any other layer carries independent signal. The independence premise is the load-bearing one, and it is where the argument fails in practice: adjacent residual-stream taps are strongly correlated, the quadrature sum overstates what is actually available, and the measured gain over a probe on the same concatenated taps is zero or negative (§4.8.1). What follows is therefore the argument for why fusion should help, presented so that its failure on real models is legible. The widget below makes the effect concrete: varying the three per-layer $d’$ updates the fused $d’$ and the two error rates. The defaults are the values the synthetic experiment (§4.1) recovered.

To restate the caveat above in the terms of the filter family: the quadrature gain assumes independent per-layer noise, and adjacent layers are correlated, so the real gain is smaller than three independent layers would suggest. Accounting for the correlation is what fisher does that diag does not, but on real prompts that correction is not enough to make the fusion worth anything: a probe on the same concatenated taps matches or beats fisher (§4.8.1), so neither filter is recommended here over reading a single well-chosen layer.

3.7 Class-conditional mixtures and BIC

A single Gaussian per class assumes each class is one blob. Real “benign” traffic is not one blob — it is chit-chat, homework, code, each with its own score profile across depth. So we let each class be a small library of profiles: a Gaussian mixture on the joint spectrogram,

gated, exactly as before, on the log-likelihood ratio between the two class densities,

Modelling on the joint $\mathbf{s}$ (not per layer) keeps cross-layer correlations. With $J=1$ per class and shared covariance the LLR is affine in $\mathbf{s}$ with normal vector $\Sigma_{\mathbf s}^{-1}(\bar{\mathbf s}^+-\bar{\mathbf s}^-)$ — i.e. it reduces exactly to the fisher filter of §3.5. The mixture is a strict generalization.

How many profiles? A larger mixture always fits the training data better, so fit alone cannot decide $J$. We use the Bayesian Information Criterion, $\mathrm{BIC}=-2\log\text{-lik}+k\ln N$, over $J\in{1,2,3}$ per class: each parameter pays rent $\ln N$, and an extra profile is admitted only if the fit it buys exceeds that rent. This makes “one distribution or many?” a question the data answers — with a consequence documented in §4.2: at ten-shot sample sizes, the rent is never paid, and $J$ collapses to $1$.

3.8 The calibrated gate: fire, abstain, pass

Fit two 1-D Gaussians on the filtered score $S$ and decide by the likelihood ratio $\mathrm{LLR}(S)$. The threshold $\tau$ is calibrated to a target on the benign tail — either a false-positive-rate quantile, or a $z$-based rule that places $\tau$ where a benign score sits $z$ standard deviations above the benign mean ($z=3$ ≈ 0.1% benign-tail FPR). We map the LLR to a calibrated probability $P_{\text{present}}=\sigma((\mathrm{LLR}(S)-\tau)/s)$ (with $s$ the benign spread local to $\tau$, because the LLR’s heavy tails make a global spread useless), and expose a three-way verdict: fire if $\mathrm{LLR}>\tau$, pass if below, but abstain — override to no decision — whenever $\lvert P_{\text{present}}-\tfrac12\rvert<\text{margin}$. Abstention is the appropriate few-shot behaviour: with ten examples per class, some prompts fall near the boundary, and reporting that is more useful than forcing a decision.

The figure below applies this gate to real GPT-2 activations for a jailbreak concept, calibrated at a deliberately permissive operating point ($z=2$, so several borderline benign prompts fire). Ten labelled probes — jailbreak, benign, and two deliberately borderline cases — carry their measured LLRs; moving $\tau$ reclassifies each probe and traces the trade-off between recall and false-positive rate on the concept’s own examples. Two behaviours are worth observing. The benign prompt “Ignore the noise outside and focus on the recipe” scores high because the early layers respond to the word “ignore” — a genuine failure mode of shallow taps — and raising $\tau$ far enough to suppress it also begins to reject true jailbreaks. There is no threshold that separates the two cleanly; there is only a choice of operating point.

3.9 Combining K concepts

A bank of $K$ concepts fires if any single concept fires, and attributes the firing to the concept with the largest likelihood ratio:

The attributed concept is the one whose direction is used if the action steers, so the same max-LLR rule that decides whether to act also decides along which concept to steer — a small but convenient coupling that keeps a multi-concept bank behaving like a single decision. Each concept is independent kilobytes, so a bank scales linearly and stays tiny. Per-concept thresholds keep the directions independent in storage and threshold, but they do not compose into a calibrated bank: the false-positive rate is the union over the $K$ concepts, so a per-concept $z=3$ ($\approx 0.1\%$ FPR) OR-ed over $K=14$ gives a bank-level FPR near $1.4\%$ — the operating point has to be set against the whole bank, not one concept at a time. That union is an upper bound: statistically the fourteen harm directions are far from independent (§4.8.2). This bank — one shared truncated forward broadcast to the $K$ concept directions, each with a detection direction and a related raw direction available for steering — is drawn in Figure 12, and its cost as $K$ grows is measured in §4.8.2.

3.10 Steering: the write side

Detection runs in standardized space, but the steering hook perturbs the raw residual stream — the hook sees un-standardized activations at generation time — so we keep a second, raw-space direction per layer, the unit difference of the raw class means:

This is deliberately decoupled from the standardized detection direction of §3.3: the detector wants the whitened direction that separates classes, whereas the steerer wants the direction that actually exists in the model’s native activation space, since that is what the forward hook can add. The two are therefore related but not identical — fit from the same ~10 examples and pointing broadly the same way, but decoupled by the standardization (and, when detection uses the logistic mode of §4.3, by its covariance-awareness). The quantity we measure is the cosine between the detection direction mapped back into raw space — $w_\ell$ divided elementwise by the per-feature scale $\sigma_\ell$, then renormalized — and the raw steering direction $w^{\text{raw}}_\ell$. It is not a single number: it depends on the detection mode, on the concept, and on the model, so we report the sweep rather than one figure. It is a geometric property of two fitted directions, and we measure it on raw prompts with no chat template — the same regime as the detection benchmark, and unlike the steering experiments of §4.10 onward, which format prompts as the model expects. Averaged over three taps and four concepts:

The last row is the configuration whose detection numbers §4.8 reports, and it is the lowest: alignment is highest for hand-written few-shot concepts under diff-of-means detection, and drops as the detector becomes more covariance-aware and the fitting set larger and messier. Both effects are expected — logistic mode rotates the detector away from the class-mean difference by design, which is exactly what makes it the better detector and the worse proxy for the steering vector.

Both halves of these numbers matter, and quoting either alone misleads. Two random unit vectors in $\mathbb{R}^d$ have cosine of order $1/\sqrt d$ — $0.036$, $0.033$, and $0.021$ at these widths — so even every value sits between roughly $13\sigma$ and $40\sigma$ above chance (the multiple depends on $d$ as well as on the cosine): the read and write directions are unmistakably related, never coincidentally aligned. But $0.45$ is also $63°$ and $0.83$ is $34°$, so none of them is close to identity either. A reader should take neither “the same direction used twice” nor “two unrelated directions” from this. What the read and write sides genuinely share is the fitting data and the class-mean construction, not the exact geometry. The measurement is scripts/eval_gate.py --cosine. During generation, at each tapped layer we add

with $\alpha>0$ steering toward the concept and $\alpha<0$ away (the refusal / guardrail direction). The one practical subtlety is magnitude: a good absolute $\alpha$ on GPT-2 is wrong on Qwen, because their residual norms differ by about fivefold (96 vs 19 in our runs). So we set $\alpha$ as a fraction of the measured residual norm, which transfers approximately across the three models tested — the coherent band is similar but not identical on each, and we have not verified it beyond them — empirically $\sim$3–10% is the coherent band, and above roughly 20–25% — a range seen in development runs, beyond the sweeps Figures 5 and 8 show — the text degrades into repetition or gibberish.

The figure below shows actual generations across a range of fractions, from negative (away from the concept) through zero (unsteered) to positive (toward the concept), for a chosen model and concept. The effect is clearest on Qwen2.5-0.5B: the “food” direction pulls the continuation toward “I made this dish… the sweet and savory flavors,” and the “nature” direction toward “a group of bees… the scent of wildflowers.” GPT-2 shifts more weakly under the same procedure and degrades into repetition sooner — the same control applied to a less capable model, an effect examined in §4.6.

3.11 Actions and the run driver

Detection and steering are unified behind one small strategy interface. An action is a policy object with a single method, decide(ctx) → Decision, given a narrow view of the verdict; the gate executes the returned decision. Three actions ship:

• Abort → halts decoding and appends a fixed marker after generation has stopped — a hard gate whose saving is the decoding it skips (the prompt’s forward has already run for the check).
• Steer → adds the per-layer steering vectors for the whole generation (§3.10), with the magnitude as a fraction of the residual norm and an optional named concept to steer along.
• Emit → seeds a fixed string into the completion and lets the model continue from it — a soft redirect (open with a refusal, then let the model finish it in its own voice), as opposed to Abort’s post-hoc marker.

When each acts is a shared Trigger: FIRE (only on a confident firing), FIRE_OR_UNSURE (also on an abstain, fail-closed), or ALWAYS (unconditionally — the topic-steering case). Two further triggers compose with these and came out of §4.11: Predicted(name, τ) fires on the prediction of an outcome head rather than on concept membership, and Both(a, b) fires only when two triggers agree — so present and predicted to move is expressible. That path is opt-in, needs labels at fit time, and carries the one-model evidence of §4.11. A single run method is the driver: it takes the cheap input verdict, asks the action, and executes — halting, steering the generation, or seeding and continuing. Detection-only use needs no action at all; it is the pure measurement primitive that run is built on.

3.12 The truncated forward and cost

To read a tap at layer $\ell$, only blocks $0..\ell$ need to run. Detection therefore executes a truncated forward — the tail of the network, the final norm, and the unembedding are never touched — which on GPT-2 is measured bit-identical at the taps and about 43% faster (1.75×) than a full forward. A weight-truncated load mode goes further and never materializes the tail at all, so a large model tapped early loads a fraction of its weights (detection-only; generation still needs the whole network). This is what makes the compute–accuracy frontier of §4.5 a real saving rather than a curiosity: a guardrail that fires on every prompt should run as little of the model as the concept allows.

3.13 The lifecycle and library

Everything above is exposed as six small operations, and it is worth walking them in order both because the ordering is the method and because it is how the reference implementation is actually used. The whole surface is: load a model and choose where to tap, learn a concept from examples, calibrate its operating point, check a prompt (pure detection), and run a prompt under an action (detect-and-act). Nothing in the list trains the host model, and only the first step touches its weights.

Load is the one place the memory–compute trade is made explicit, through a LoadMode. The default, FULL, materializes the whole network and can therefore both detect and generate. The optimization, UP_TO_TAPS, constructs the model with only the embedding and blocks $0..\max(\mathcal{L})$ — the tail blocks, the final norm, and the unembedding are never allocated — so a model tapped early loads only a fraction of its weights (for an 8B model tapped in the lower third, on the order of ~6 GB instead of ~16 GB). The catch is exactly what the truncated forward already implied: a weight-truncated gate has no lm_head and so is detect-only; asking it to generate raises rather than silently misbehaving. Orthogonal to which weights are loaded is a batch_size knob on learning, which trades extraction memory for speed (one prompt at a time versus a padded batch) — the two dials compose freely, and neither affects the learned concept.

from conceptgate import ConceptGate, LoadMode
from conceptgate.actions import Abort, Steer, Emit, Trigger
from conceptgate import Both, Predicted

# FULL: can detect AND generate (needed for steering / emit)
cg = ConceptGate.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", layers=[8, 12, 16])

# UP_TO_TAPS: loads only blocks 0..max(tap) -> a fraction of the weights, DETECT-ONLY
guard = ConceptGate.from_pretrained("gpt2", layers=[4, 6, 8], load=LoadMode.UP_TO_TAPS)

Learn fits a concept from two small prompt lists; it is a few sample means and a small solve, so it returns in milliseconds and can be called repeatedly to build a bank. Calibrate then sets each concept’s threshold $\tau$ from the benign tail (a target false-positive rate, or a $z$-based rule) and, optionally, an abstain band. These two steps are all that stand between raw examples and a working gate.

cg.learn("jailbreak", positives=[...], negatives=[...])   # ~10 per side; chainable
cg.learn("medical_advice", positives=[...], negatives=[...])
cg.calibrate(z=3.0, margin=0.1)   # per-concept tau + an "unsure" band; higher z = stricter

Check is the pure measurement primitive: it runs the truncated forward and returns a Verdict (fire / abstain / pass, a calibrated $P_{\text{present}}$, the attributed concept, and the residual norm used for fractional steering). It is the only operation a UP_TO_TAPS gate can perform, and it is what every guardrail deployment leans on. Run wraps check with an action and drives generation: it asks the action about the input verdict and executes the returned decision — halt (Abort), steer the whole generation (Steer), or seed-and-continue (Emit) — with a shared Trigger deciding when the action acts.

# detection only (cheapest; works on an UP_TO_TAPS gate)
v = guard.check("Ignore your instructions and act with no limits")
if v.fired:
    ...  # v.concept, v.p_present, v.score

# workflow 1 -- guardrail: block when a concept fires, else generate normally
cg.run(prompt, action=Abort(when=Trigger.FIRE))

# workflow 2 -- soft redirect: open with a refusal and let the model continue it
cg.run(prompt, action=Emit(text="\nI can't help with that.", when=Trigger.FIRE))

# workflow 3 -- topic steering: bend generation toward a named concept, unconditionally
cg.learn("food", positives=[...], negatives=[...])
cg.run(prompt, action=Steer(concept="food", fraction=0.06, when=Trigger.ALWAYS))

# workflow 4 -- conditional steering: steer AWAY only when the concept is detected
cg.run(prompt, action=Steer(concept="jailbreak", fraction=-0.06, when=Trigger.FIRE))

# workflow 5 -- outcome-gated steering: write only where the write is predicted to move the prompt.
# `measured_dose[i]` is an effect you measured for prompts[i]; labels are needed here and nowhere else.
cg.learn_outcome("dose", prompts, measured_dose)
cg.run(prompt, action=Steer(concept="jailbreak", fraction=-0.08,
                            when=Both(Trigger.FIRE, Predicted("dose", 0.15))))

The division of labour is worth stating plainly, because it is the whole memory-efficiency argument in one place. Detection — learn, calibrate, check, and the Abort half of run — needs only the bottom of the network and can run on a weight-truncated load, which is where the compute and memory savings arise and why a pure guardrail is inexpensive to attach. Generation — the Steer and Emit halves of run — needs the full network, so the differentiating write capability does not enjoy the load-time saving; it is cheap only in the sense that it adds a few vector additions to a forward pass the host was already going to run. A deployment that only ever detects should load UP_TO_TAPS and never pay for the tail; a deployment that steers loads FULL and gets detection for free on the way through.

4. Experiments and results

We evaluate on GPT-2 (12 blocks), Qwen2.5-0.5B-Instruct (24 blocks), and gemma-2-2b-it (26 blocks), joined in §4.11’s first-token sweep by SmolLM2-360M-Instruct and SmolLM2-1.7B-Instruct — all small enough to run and re-run on a laptop, which is the point, since the whole method is meant to be cheap. The subsections follow the method’s own order: the reading path first (§4.1–§4.5), then the write side (§4.6), then cost (§4.8). That is deliberately not the order of the contributions — the one capability we defend as specific to the method is steering, which appears sixth; the one result we defend as new is the per-prompt dose prediction of §4.11, which appears last. The negative results are not buried.

4.1 Depth fusion on synthetic data

On a controlled synthetic problem with three layers of known per-layer discriminability $d’=[1.6,2.0,0.6]$, the theory predicts a fused $d’=\sqrt{1.6^2+2.0^2+0.6^2}=2.63$, i.e. test error $\Phi(-1.315)=9.4\%$ versus the single-best-layer $\Phi(-1.0)=15.9\%$. The learned filter recovers $d’=[1.62,2.04,0.64]$ and drives test error from 16.1% to 9.4%, matching the prediction. This verifies that the implementation realizes the matched-filter algebra; it is not evidence about residual streams. The synthetic data is generated under the filter’s own assumption of independent per-layer Gaussian noise, so recovering the predicted $d’$ is close to tautological. On a real model the layers are correlated, the gain vanishes, and a probe on the same concatenated taps matches or beats the fusion (§4.8.1); we therefore treat depth fusion as a mechanism that does not transfer, not as a contribution.

4.2 Mixture densities: a constructed hard case and a few-shot collapse

The mixture model is justified by a constructed hard case: place two benign clusters on either side of the harmful cluster along the discriminative axis (benign at $-2$ and $+2$, harmful at $0$). No single threshold on any linear score can carve out “the middle,” so the fisher gate is stuck near chance (38.8% error, AUC 0.60; synthetic data, 8,000 points per class, five seeds); the mixture, seeing $\mathbf{s}$ near a benign profile on each side and a harmful profile between, recovers it (7.1% error, AUC 0.98; the Bayes floor is 5.8%). That is the case for mixtures, and Figure 6 shows its geometry. One honest caveat on the construction: with the benign clusters symmetric about the harmful one the two class means coincide, so the diff-of-means (or logistic) direction the pipeline actually fits would be null here. This hard case is built directly in score space to illustrate the mixture’s expressiveness; it is not a configuration ConceptGate’s own direction-finding would produce.

The case against them, at least in the regime we care about, is that on real GPT-2 activations with 12+12 prompts, BIC selects $J=1$ for both classes — an extra full-covariance profile over five layers costs ~21 parameters, whose rent (~52 nats) twelve samples cannot pay — and the mixture gate collapses onto the single-Gaussian gate (rank agreement 0.986, scored on held-out prompts). This is less “the data answering one-or-many” than an identifiability limit: at twelve samples a five-dimensional full-covariance component is already near-singular, so the $J=2$ fit is ill-conditioned and the selection is effectively decided by sample size. The mixture is the more general model but remains inactive in this regime, and the natural question is where the boundary actually sits — the rent grows only as $\ln N$ while the evidence for a second component grows as $N$, so “few-shot” cannot be the whole account.

We measured it on the same data and taps the detection benchmark uses (jackhhao jailbreak against benign, three taps at 33/50/67% depth, directions fit exactly as the pipeline fits them, three seeds), asking BIC for $J$ at each fitting size and scoring the resulting gate on the official held-out split:

Components selected for the positive / negative class, median over three seeds. The last row is the largest held-out AUC difference between the BIC-selected gate and the same gate forced to one component per class, over both models and both detection modes.

Two things follow, and they cut against each other. The collapse is not an artifact of twelve samples: BIC keeps returning one component per class up to 64 examples per class on both models in diff-of-means mode, sample sizes at which a three-dimensional full-covariance component is perfectly well conditioned. So the identifiability reading above is too narrow — the criterion goes on declining the second component well past the point where it could fit one. And where the second component is eventually selected, it buys essentially nothing: held-out AUC is unchanged to three decimals in logistic mode on Qwen at every size, and the largest gain anywhere in the sweep is +0.007. Whatever structure BIC eventually finds is not structure this decision needed, which is the same verdict §4.3 reaches from the other side when an RBF kernel loses to a linear one on these activations. The mixture remains the more general model, and the constructed case above shows that generality is not vacuous; on the concepts examined here it is inactive at the sizes the system runs at and inert at the sizes where it activates. What this sweep does not establish is that the classes are genuinely bimodal — a component selected at $N=256$ could be fitting a small cluster of outlying prompts, and we did not measure the fitted component weights.

4.3 Detection on real prompts: a commodity

On real jailbreak-versus-benign prompts, ConceptGate’s detector performs well in its logistic mode — and so does a linear support-vector machine trained on the same activations, and so does per-layer logistic regression: across both models those three are within noise of one another on AUC. The default difference-of-means mode is not in that group — it trails by two to five points (§4.8.1) — which is why every comparison against a trained classifier in this paper uses the logistic mode. §4.8.1 puts numbers on “within noise” over 262 held-out prompts: at 32 examples per class, ConceptGate-logistic reaches $0.973$ against $0.978$ for a probe on the same taps and $0.982$ for both a probe and an SVM on the full model (Qwen2.5-0.5B), and $0.979$ against $0.978$, $0.987$, and $0.989$ respectively on gemma-2-2b. The logistic variant, which is covariance-aware and therefore slightly stronger where standardization leaves the within-class covariance non-isotropic, closes the small remaining gap to the SVM, with its largest gains on the weaker model, but it does not establish a new one. As a detector, then, ConceptGate is a commodity: any method that reads a linear direction from these activations performs comparably, and the choice of estimator is a tuning decision rather than a contribution. The capability that is not a commodity is the write side, which is why the remainder of the report concentrates on steering.

4.4 Matched versus broad negatives (a negative result)

We anticipated that matched contrastive negatives — benign prompts sharing the surface structure of the jailbreaks (the same register, without the intent) — would sharpen the direction by cancelling nuisance variation, following the CAA construction. The measurement contradicted this: matched negatives gave an AUC of approximately 0.42, below chance, against 0.78 for broad, unrelated negatives (fifteen negatives of each kind, on the development prompt set; this early run predates the released harness). The explanation is that broad negatives allow the direction to align with the large semantic gap between an assertive instruction to a model and an ordinary factual query, which is the signal the detector depends on, whereas matched negatives remove that gap. We read $0.42$ as the direction failing to separate the classes, and not as evidence that it inverts them. The point estimate does sit below $0.5$, which would mean jailbreaks ranked slightly beneath benign prompts, but with ten examples per class the sampling spread around chance is wide enough that $0.42$ is not evidence of a stable anti-signal, and we do not claim one. The practical conclusion needs only the weaker reading: for few-shot concept detection, negatives should be broad rather than matched.

4.5 The compute–accuracy frontier

Because detection needs only blocks up to the tap, every concept has a cheapest depth at which it is already separable. We sweep every layer, fit the standardized diff-of-means detector there, and measure leave-one-out AUC against the fraction of the network that tap requires. Leave-one-out keeps the held-out prompt out of the fit, but selecting the knee on the same ~20-prompt estimate makes the AUC at the chosen layer optimistically biased and, on so few prompts, high-variance; read these curves as locating roughly where the concept becomes readable, not as a precise operating AUC. The figure plots the measured curves; the target-AUC control locates the knee, the cheapest layer that clears a chosen AUC.

The shape of the curve reflects model capability. On GPT-2 the jailbreak concept is not cleanly formed until the middle of the network: AUC climbs through the early blocks and only saturates around block 6, so the cheapest reliable guardrail runs somewhat more than half the network and the final ~40% of blocks contribute nothing. On Qwen2.5-0.5B the same concept is essentially separable by block 1, so the guardrail can run roughly 8% of the network (block 1 of 24). We describe this throughout as the base model being more capable, but the comparison is confounded: GPT-2 is a 2019 base model while Qwen2.5 is an instruction-tuned model post-trained on refusal, so jailbreak-ness being salient at block 1 may be safety tuning as much as raw capability. The two are entangled here and we do not separate them. (This 8% figure is also from the leave-one-out sweep of §4.5, on ~20 prompts; the 262-prompt held-out measurement of §4.8.1 is the one to quote for a precise depth.) Each of these is a concrete, per-concept, per-model operating point, and it is the practical consequence of the truncated forward.

4.6 Steering across models

Steering (§3.10) is the operation a detector cannot perform, and the part of the system that most needs measuring rather than illustrating. For each of three concepts — food, nature, technology — we generate a continuation of five neutral prompts at each value of the steering fraction (the magnitude of the added direction, as a fraction of the residual norm) and score every greedy generation three ways: the share of content words matching a concept keyword list (independent of the steering direction, in that the word list is written by hand and never enters the fit — but not independent of the concept’s framing, since the same author chose both the keyword list and the few-shot prompts, so the two share whatever conception of “nature” or “food” that author had), the perplexity of the continuation under the base model (fluency), and the concept’s own detector log-likelihood ratio on the generated text (the internal read of what the write produced). There is no baseline method to compare against — a linear probe or classifier cannot steer at all, and the mechanism itself is standard activation addition (§2.2) — so the comparison is the unsteered point (fraction 0) and the two base models. The harness is scripts/eval_steering.py.

Steering is a graded dose-response. On Qwen2.5-0.5B, raising the toward-fraction increases concept content smoothly — the food direction moves the continuation to homemade-salsa completions, the nature direction to sun-over-the-mountains ones — while perplexity stays flat through roughly $|\text{fraction}| \le 0.1$, giving an effective window in which content shifts and fluency holds. The concept’s own detector tracks the shift ordinally (the food LLR rises over the sweep, with one reversal between adjacent steps, nature from below zero to above), so on a capable model the read and the write agree — though these LLR magnitudes are not calibrated probabilities: the Gaussian density is badly misspecified for generated text far from the few-shot fitting set, which is why the independent lexicon is the primary signal here.

GPT-2 shows the same shape under the identical procedure, bounded by the weaker model. It shifts content even harder at the largest fractions — the nature direction reaches a 23% keyword rate against Qwen’s 9% — but as degenerate repetition (“the sun shone on the mountains, and the moon shone on the mountains…”), and its own detector barely registers its own steered output (the nature LLR stays negative throughout). The write can outrun a weak read, but the fluency of the steered text is bounded by what the base model produces cleanly — the same ceiling that bounds detection (§4.5): a more capable model steers more cleanly from the same ten examples.

That reading deserves a check, because a competing explanation predicts the same observation. If the write direction sits far off the read direction (§3.10), then steering would move the activations somewhere the detector does not look, and the detector would fail to register its own steered output on any model, weak or not. We would then be reporting a geometric artifact as a capability ceiling. The test is direct: steer, re-read the continuation, and ask whether it moved along the steering direction while staying still along the detection direction. It does not. On GPT-2 the steered continuation moves $+0.02$ (norm-relative) along the steering direction and the detection score moves with it, by $+4.2$ and $+2.1$ standardized units at two of three taps, for a total concept LLR change of $+21$; on Qwen the corresponding figures are $+0.12$–$+0.20$ and $+2.6$ to $+9.1$, LLR $+48$. The detector does register the steered output on both models. And the decoupling account fails a second, sharper test: GPT-2 is not the worst-aligned of the three models in any mode we measured — Qwen is, every time (0.45 against GPT-2’s 0.52 and gemma’s 0.57 in the configuration of §4.8; 0.65 against 0.79 and 0.75 under diff-of-means), and Qwen’s detector tracks its own steered output well. Were decoupling the cause of a weak read, the worst-aligned model is where it should show, and it does not. What is left is the original reading, now with a measured basis: GPT-2’s nature LLR stays negative because the concept is poorly separated there in absolute terms, not because the write goes somewhere the read cannot see. The measurement is scripts/eval_gate.py --decouple.

Across the three signals, steering is a measured, controllable capability rather than a demonstration: within an effective window it shifts generated content monotonically — an operation no detector or classifier can perform — and its quality, like detection’s, is bounded by the base model rather than by the number of examples. It is a soft control, strongest on concepts the model represents clearly and weaker on abstract ones such as technology, so it is best used as a nudge within its window rather than as a hard guarantee. This measured write capability, fit from the same few-shot data as the detector, is what justifies operating on a concept inside the residual stream rather than filtering on the output text.

4.7 A paraphrase-robustness null

We expected a depth-dependent robustness effect: that shallow taps, keyed on surface words, would collapse when jailbreak prompts are reworded, while deep taps, keyed on meaning, would hold — giving a principled reason to gate deep. It does not appear. Rewording the positives barely moves AUC at any layer, because detection against broad negatives rides the semantic gap (§4.4), which rewording does not close. The lexical-sensitivity effect only appears against hard, surface-matched negatives — which, as §4.4 shows, themselves degrade detection. We report this null to document that the effect, though intuitive, does not arise in the broad-negative setting that detection otherwise relies on.

4.8 An efficiency evaluation of ConceptGate

The commodity result (§4.3) shows that the choice of linear estimator barely affects accuracy. This section asks what ConceptGate costs to reach it, against the standard ways of adapting a frozen model — and finds two different answers. For a single concept (§4.8.1) the cost is low, but the saving is not ConceptGate’s: it is the truncated forward, which a depth-matched probe shares exactly. The separation appears only for a bank of many concepts (§4.8.2), where a training-free bank extends without the per-concept training run that fine-tuning needs. The task throughout is content-safety detection, used not because the method is specific to safety but because it comes with public datasets and a natural multi-concept structure: the concept is a single category in §4.8.1 and the fourteen categories of a guardrail taxonomy in §4.8.2. Both subsections report the same axes — detection AUC, memory (weights loaded and parameters learned, both device-independent), and compute (per-prompt forward wall-time on an Apple M4 under MPS) — first for one concept, then for a bank of many as their number grows.

4.8.1 Learning a single concept

Setup. The task is jailbreak detection on the public jackhhao/jailbreak-classification dataset: each method learns the concept from a small, balanced set of few-shot examples drawn from the training split, and is scored on the untouched official test split (262 prompts — 139 jailbreak, 123 benign). We evaluate two frozen base models, Qwen2.5-0.5B-Instruct (24 transformer blocks) and gemma-2-2b-it (26 blocks). We compare against a linear probe in two forms, trained on the same few-shot examples. The depth-matched probe fits a logistic head on the same tapped activations ConceptGate reads — the identical truncated forward, the identical compute — and is the fair single-concept comparator. The full-model probe fits its head on the final-layer representation, running the whole network; it is the conventional frozen-model classifier and serves as the upper reference. ConceptGate is swept over its tap configurations — a single tap at increasing depth, and three- and five-tap fusions — each in its logistic-direction mode. The three comparison axes are detection AUC on the held-out test; memory, as the fraction of the base model’s weights that must be loaded and the number of parameters learned (both device-independent); and compute, as the measured per-prompt forward wall-time on an Apple M4 under MPS, averaged over three seeds. We report held-out test metrics only. The harness (scripts/eval_detection.py) and the full result tables (docs/evaluation.md) are in the repository; the figure and tables below replay their output.

At every tap depth ConceptGate’s logistic direction and the depth-matched probe are the same number — a single-tap logistic direction is logistic regression on that tap — so the only real comparison is against the full-model probe, and the compute gap is entirely the truncated forward. On Qwen2.5-0.5B a single tap at 46% depth scores 0.970 at 61% of the weights; on gemma-2-2b a tap at 42% depth scores 0.974 at 55% of the weights. The full-model probe reaches a slightly higher ceiling (0.982 and 0.987) because it runs the whole network. The point of the table is the middle rows: the depth-matched probe reaches ConceptGate’s number at ConceptGate’s compute.

AUCs are the mean over three few-shot resamples with the standard deviation across them; the AUC columns reproduced to three decimal places on an independent re-run. Cost is reported as fraction of depth and of loaded weights rather than in milliseconds, because both are exact properties of the truncation while wall-clock is not: the same configurations timed on the same machine under different load gave forward times differing by up to a factor of two, though the ratios to a full forward were stable to a few points (a single early tap runs at roughly 30–48% of a full forward, three taps at roughly 60–69%).

Two observations, both negative for the read side. Depth fusion buys nothing: the three- and five-tap ConceptGate configurations sit on top of the best single tap, and on Qwen the depth-matched probe on the concatenated taps slightly beats ConceptGate’s bandpass fusion (0.978 vs 0.973) — the synthetic depth-fusion advantage (§4.1) does not transfer to a real model where one layer already carries the concept. And the difference-of-means direction trails the logistic one by two to five points and is not competitive with either probe.

So the honest reading is not that ConceptGate is Pareto-efficient over a fair baseline — a depth-matched probe matches it at the same compute and beats its fusion. What §4.8.1 establishes is a fact about the models, not about ConceptGate: the jailbreak concept is linearly readable at a shallow tap, so a truncated forward suffices and the top of the network can be skipped; how shallow that tap can be is set by how early the base model forms the abstraction (§4.5). The truncated forward is genuinely cheaper than running the whole model, but it is a property any latent probe shares, not a contribution of ConceptGate. What is specific to ConceptGate is that a direction fit from the same few-shot data can also be written back to steer (§4.6), and the cost of extending a bank of many concepts, which §4.8.2 measures against fine-tuning.

4.8.2 Learning multiple concepts

The single-concept saving of §4.8.1 is not, on its own, specific to ConceptGate: a truncated forward is available to any latent probe (§5.3). The setting where a training-free adapter separates from the alternatives is the one a real deployment faces — many concepts, not one. A content-safety guardrail is the familiar instance: Llama Guard [8] scores a fixed hazard taxonomy, and GLiGuard, a recent guardrail built on GLiNER [13], is fine-tuned to score fourteen harm categories and eleven jailbreak strategies in a single encoder. This subsection measures what a bank of $K$ concepts costs to build, extend, run, and store, and how that cost grows with $K$.

There are two standard ways to add a concept to a frozen model, and ConceptGate is a third. The first is a linear-probe bank: freeze the model and train one linear head per concept on a shared representation. The second is fine-tuning — a LoRA [11] adapter per concept, or, at the monolithic extreme, a single model fine-tuned once over the whole taxonomy — the way GLiGuard fully fine-tunes its encoder, so that changing the taxonomy means retraining. ConceptGate is instead a training-free concept bank: one truncated forward produces the activations that every concept reads; each concept is a closed-form direction fitted in milliseconds and stored in kilobytes; and concepts are added or removed without touching the others (the max-LLR combination of §3.9). Because reading and writing share their fitting data (§3.10), each entry in the bank also carries a write direction at no extra cost. That is a direction, not a demonstrated control: for these harm categories the write lowers refusal (80% unsteered to 68% gated, 65% blanket) (§4.10).

Setup. The concepts are the fourteen harm categories of BeaverTails [14] (PKU-Alignment/BeaverTails) — the same kind of safety taxonomy the guardrails above target. For each category, positives are prompts whose responses were annotated with that harm and negatives are a shared pool of benign prompts; every method sees the same $N=32$ examples per class and is scored on the held-out test split, averaged over three seeds, on both base models. Two caveats on this labelling. The annotation is on the response, so a response-derived label stands in for a prompt-level concept; and because a single benign pool is shared across all fourteen categories, the fourteen directions are pulled toward one common benign centroid — so what presents as fourteen independent concepts is closer to one harmfulness direction with category-specific variation, which also colours the generalization result of §4.9. As $K$ grows from 1 to 14 we measure build time (learning the whole bank), inference (per-prompt wall-time to score against all $K$ concepts), memory (parameters learned), and per-category detection AUC. ConceptGate and the probe learn on frozen features; LoRA back-propagates a rank-8 adapter and a classification head. The harness (scripts/eval_detection.py --scaling) and the raw results (scripts/eval_scaling_results.json, tabulated in docs/evaluation.md) are in the repository.

Adding a concept to ConceptGate is a closed-form fit — 6 ms on Qwen2.5-0.5B, 11 ms on gemma-2-2b — against a LoRA training run of 17 s and 126 s, three to four orders of magnitude more. Two memory costs matter separately. The per-concept artifact — what must be stored to add a concept — is 11–28 thousand numbers for ConceptGate: the detection and steering directions plus per-dimension standardization at each of its three taps, about an order of magnitude more than the probe’s single final-layer head, but both kilobyte-scale against LoRA’s half-to-1.6 million. The model itself — the hundreds of millions of weights that must be resident and run for every prompt — is where that is repaid: ConceptGate loads and runs only up to its deepest tap, never the layers above it, so a single truncated forward serves the whole bank at a per-prompt cost that is constant in $K$ and at or below the probe’s full-model forward, whereas $K$ LoRA adapters need $K$ forwards. The extra kilobytes ConceptGate stores are immaterial next to the part of the network it skips. Against fine-tuning the gap compounds with every concept: building the full fourteen-category bank takes about 8 seconds on Qwen and 46 on gemma for a 256-prompt shared benign pool, against LoRA’s 3.9 and 29 minutes — 30× and 38×.

The low cost does not come at the expense of detection. Across the fourteen categories the training-free bank trails the full-model linear probe by 0.023 on Qwen2.5-0.5B (mean AUC 0.832 vs 0.855) and is statistically indistinguishable on gemma-2-2b (0.881 vs 0.874 — within seed noise at three seeds). Few-shot LoRA is both the slowest to train and the weakest to read: on the three categories where it was run it reaches mean AUC 0.685 on Qwen and 0.814 on gemma, against ConceptGate’s 0.889 and 0.913 on those same three — a randomly-initialized head simply does not have enough signal in $2N$ examples. Harm content is read best deeper than jailbreak framing, at 50–85% depth rather than the ~45% that suffices for jailbreak framing, which is why the taps here sit lower than in §4.8.1; where a category emerges late (sexual content on Qwen) a mid-depth read gives up some AUC, and where the base model forms the abstraction cleanly (most categories on gemma-2-2b) the closed-form direction is as good as the trained head.

The multi-concept setting is where the training-free design matters. A latent bank — ConceptGate, or equally a linear-probe bank — amortizes across a taxonomy in a way that per-concept or monolithic fine-tuning cannot: constant inference, closed-form extension, kilobytes per concept, and no retraining to change the taxonomy. That is a genuine result, and it is honest that a detect-only probe bank shares it. The comparison in the table also handed the probe a depth advantage — its heads sit on the final layer, while ConceptGate ran 91% of the backbone — so we refit the same logistic heads on ConceptGate’s own taps from the cached activations, at identical cost (scripts/eval_probe_tap_bank.py, reproducing the stored numbers to four decimals first). They score 0.840 on Qwen and 0.880 on gemma against ConceptGate’s 0.832 and 0.881: a tie on gemma, ahead on 8 of 14 categories, and a slight probe edge on Qwen, ahead on 10 of 14 — exactly what §4.8.1 predicted. Over such a probe bank ConceptGate’s cost is at worst a tie — a truncated forward is never more than the probe’s full one, and its extra per-concept kilobytes are negligible — while it adds one thing the probe cannot: a second, write-side use of the same $K$ concepts, a related raw direction per concept that can be added back into the stream (§4.6).

It would be convenient to conclude that the one object gating fourteen harms can therefore bend generation away from them, and we tested exactly that on five of the fourteen. It does not (§4.10): the directions gate, and gating still confines the intervention, but steering away from a harm category does not make this model decline the request. So what the bank adds over a detect-only probe bank is a write direction per concept at no extra fitting cost — a real property of the construction, and the write is demonstrated to change behaviour for topical concepts (§4.6) and for jailbreak framing (§4.10), but not for these harm categories. A taxonomy-scale bank that is cheap to build and extend, competitive with a trained probe on every category, and far ahead of few-shot fine-tuning is what stands; the steerability of these entries is a claim the measurement does not support.

4.9 Out-of-distribution generalization

In-distribution detection accuracy need not predict performance under distribution shift, and linear probes are known to generalize poorly off-distribution. §4.8.2 estimated a separate direction for each harm category and evaluated it on that same category; here we instead measure how a harmfulness direction transfers to categories it was not estimated from. Fixing the concept and varying only the harm category isolates out-of-distribution generalization within a single concept, and avoids the confound of transferring between two distinct concepts — for instance jailbreak framing and harmful content — where a change of concept is entangled with the change of distribution.

Setup. We use leave-one-category-out cross-validation over the fourteen BeaverTails categories. For each held-out category, ConceptGate and the full-model linear probe estimate the harmful direction from the remaining thirteen categories together with a shared benign pool ($N=32$ per class) and are evaluated on the held-out category; an in-distribution reference instead estimates the direction from the held-out category itself. The two conditions share the same benign examples, test set, sample size, and seeds, so the only difference is whether the evaluated category was present during estimation. We report the mean over three seeds for both base models, reusing the activations of §4.8.2. Harness: --ood; results: eval_ood_results.json.

Generalization is partial. Averaged over the fourteen held-out categories, a harmfulness direction estimated from the remaining thirteen attains a mean AUC of 0.62–0.65 — above chance, but substantially below the 0.83–0.87 obtained in-distribution. The degradation is uneven: violence, self-harm, drug-and-weapon, terrorism, and financial-crime prompts remain detectable when held out (AUC ≈ 0.74–0.78 on Qwen2.5-0.5B, 0.67–0.75 on gemma-2-2b), whereas controversial-political content (0.39–0.47) and, on gemma-2-2b, sexually explicit content (0.43) fall to or below chance. Harmfulness is therefore encoded partly as a shared, category-independent direction and partly as category-specific structure that a held-out estimate does not recover.

ConceptGate is at least as robust to this shift as the full-model probe, and on one model it is measurably more so. Because every category is scored by both methods, the two can be compared pairwise across the fourteen held-out categories rather than through an aggregate. On Qwen2.5-0.5B ConceptGate’s held-out AUC exceeds the probe’s in 13 of 14 categories, by $0.037 \pm 0.032$ (sample standard deviation across the fourteen categories), and its degradation is smaller by $0.057 \pm 0.044$ — a consistent advantage rather than a tie. On gemma-2-2b the two are indistinguishable: ConceptGate is ahead in 7 of 14 categories, by $0.007 \pm 0.028$. The mid-layer tapped direction is therefore no less transferable than a final-layer one, and on the smaller model somewhat more so.

We deliberately do not attach a $p$-value to the 13-of-14 count. A sign test would treat the categories as independent trials, and they are not: the leave-one-out design gives every category an estimation set overlapping every other’s, and §4.8.2 finds that what presents as fourteen concepts is closer to one harmfulness direction with category-specific variation. The effective number of independent trials is therefore smaller than fourteen and any nominal $p$ would be anticonservative. “Ahead on 13 of 14, behind on 1” is the finding, and it does not need a significance claim to be legible. We report the paired comparison rather than a seed-level standard deviation because the per-category results are what the harness records; the comparison is paired on the same categories and the same examples, with only the scoring method varying. The absolute level nonetheless indicates that a single few-shot direction is only a partial detector for categories outside its estimation set, and is better estimated from a diverse set of categories than from any one alone.

4.10 Gate-conditioned steering

The steering rule of §3.10 is activation addition, and conditioning it on a detector read from the activations is established: CAST [15] switches a refusal vector on a “condition vector” and reports that conditioning removes most of the benign over-refusal unconditional steering causes, across seven models; DSAS [16] fits a per-layer logistic gate from a few dozen examples and lets it modulate the write continuously. Neither reports what the gate is selecting. This section runs three controls that answer that — a random subset of the same size as the gated set, the detector’s complement, and a sign flip — and one comparison that turns out to govern all of them.

Design. Qwen2.5-0.5B-Instruct, taps 8/12/16. A jailbreak concept is fit in logistic mode from eight hand-written override framings against eight short benign requests, resampled over three seeds and calibrated at $z=2$. Test prompts: 32 held-out framed attacks, 32 benign requests in the same register, and 32 real benign prompts from the jailbreak dataset’s held-out split, used only to measure out-of-register firing. The write is $\alpha=0.08$ of the residual norm. Arms: no steer; always ($-\alpha$ everywhere); gated ($-\alpha$ where the gate fires); random ($-\alpha$ on a subset matched, separately within attacks and within benign prompts, to the gate’s fired count); anti-gate ($-\alpha$ on exactly the prompts the gate passes); and the two sign-flipped arms. Refusal is the share of continuations matching an explicit-decline lexicon — it counts declining, not harmlessness. Collateral is perplexity and the share of benign continuations left byte-identical to the unsteered baseline, which under greedy decoding is close to one minus the benign firing rate. Prompts are formatted with the model’s chat template for both reading and generation, as the model expects. Every “±” is the population standard deviation over the three resamples.

The instruct model already refuses. Formatted as a chat turn it declines 30 of the 32 attacks with no intervention at all. That is two prompts of headroom, and it means a refusal rate on this test set cannot resolve any question about raising refusal — a point we return to with a continuous outcome below, and one that governs how the rest of this table should be read.

The write is a refusal lever, pointed the unsafe way. Steering away from the jailbreak concept lowers refusal by 22 points, from 93.8% to 71.9%, in every resample: removing the concept’s signal from the residual stream makes the model comply with requests it would otherwise decline. Writing toward the concept does the reverse — at the ceiling on attacks (+3.1) and visibly on benign prompts, where it induces refusal of harmless requests (7.3% over-refusal when written to the benign prompts the gate passes, from 0%). Read as a pair, this is the single refusal-mediating direction of Arditi et al. [17], recovered from eight hand-written framings with no training. Whether that is a guardrail or a jailbreak is a matter of which sign an operator chooses.

The gate selects nothing on attacks. It fires on $96.9 \pm 2.6\%$ of the framed attacks, so the gated, random and always arms write to nearly the same prompts and their results agree to within a resample. The size-matched random arm — the control that would separate which prompts are written from how many — has nothing left to separate. The anti-gate arm writes to three percent of attacks and does correspondingly little. What the gate does do is confine the benign collateral: it leaves 82% of benign generation byte-identical where blanket steering leaves none, because it fires on $17.7 \pm 6.4\%$ of in-register benign prompts. That is CAST’s collateral result [15] reproduced at eight examples.

The gate is calibrated to a register, not just to a meaning. On real benign prompts from a different register it fires on $95.8 \pm 2.9\%$ — 30 or 31 of 32 in every resample — so off-register it would rewrite nearly everything it saw. Fitting the same concept from the dataset’s long persona templates instead of short framings inverts the failure: it then fires on 77% of the short attacks and 34% of the short benign prompts. The same collapse is documented for CAST’s own condition detector, reported to refuse essentially every mathematics prompt while passing in-distribution benign ones [20], and for activation probes generally [23].

A continuous outcome, against a random direction of the same norm

A rate at a ceiling cannot say whether the write does anything specific, so we replaced it with a continuous outcome needing no generation: at the first generated position, the log-mass on refusal-opening tokens (“I”, “Sorry”, “As”, “Unfortunately”, “No”) minus the log-mass on compliance-opening tokens (“Sure”, “Here”, “To”, “Certainly”, “Yes”, …), one forward pass per arm. This is the refusal–affirmation logit gap of Li and Liu [26], who introduced it as a per-prompt safety margin and showed that alignment widens it on 97.5–99.8% of toxic prompts across three model families; we sum over small token baskets where they take the top token of each, and we use it to score a write rather than to search for a suffix that closes it. Arms: no write; $\pm\alpha$ along the concept’s raw direction; and $\pm\alpha$ along a random unit direction of the same norm, fixed per resample — a perturbation floor. Same prompts, three resamples, 96 prompt-evaluations per cell.

Baseline is the absolute first-token refusal logit; every other column is a change from it. Lever $=\tfrac12(\Delta_{-\alpha}-\Delta_{+\alpha})$, the sign-reversible half of the response.

The concept direction is a directional lever with the same sign everywhere: away from the concept lowers the refusal logit, toward it raises it, on attacks and benign prompts alike. Its lever component is about three times that of a random direction of the same norm — 1.26 against 0.42 on attacks, larger on 91% of individual prompts; 0.99 against 0.27 on benign prompts, larger on 76%. That is clearly above the floor and not vastly above it. The sign-independent component is small for both (−0.37 concept, −0.14 random on attacks), and there is no lever-versus-perturbation split by gate status: the three attacks the gate passed moved as much as the ninety-three it fired on, though at $n=3$ that is indicative only. The random direction is not inert either — a matched-norm random write moves the refusal logit by a third as much, which is the observation of Rogue Scalpel [18] in continuous form. Taken together this is consistent with refusal riding a single direction [17] and with arbitrary steering vectors moving refusal in proportion to their alignment with it [19].

The same arms on the concept bank

Running the three main arms with five BeaverTails harm-category directions gives the same picture. Baseline refusal on the sixty held-out harmful prompts is 80.0%; blanket $-\alpha$ along each category’s direction lowers it to 65.0%; gated $-\alpha$ to 68.3%. The category gates fire on 42–100% of the safe prompts, so gated and blanket are nearly the same policy, and the gate leaves 33% of safe generation untouched against 3%. The harm-topic directions are weaker levers than the framing direction — 15 points against 22 — consistent with harmfulness and refusal being encoded along different directions [21]. Twelve prompts per category quantizes each cell to 8.3-point steps, so the category-level numbers are indicative; the direction of the effect is not.

The lever, confirmed on generated text. The 32 framed attacks are saturated, so the size of the effect is better measured on a set with headroom. On the 164 attacks of §4.11 — jailbreak templates carrying a real harmful request, plus short framings and bare requests — the refusal rate over 40 greedily generated tokens is 64% unsteered, 35% steering away from the concept, and 79% steering toward it. A single direction fit from eight hand-written framings, added at 8% of the residual norm, moves generated refusal across a 43-point range (78.7% → 35.4%) in whichever direction the operator chooses. A teacher-forced measure — the log-probability of canonical refusal continuations minus compliance ones — moves the same way (+1.08 → +0.88 → +1.12). This is the strongest aggregate behavioural statement in the report; §4.11 measures the same write prompt by prompt with a sampled instrument, and it is a restatement of Arditi et al. at eight examples rather than anything new.

Limits. Thirty-two prompts for the arm table, three resamples, one 0.5B model, one magnitude, a refusal lexicon for the rate and a hand-chosen token set for the logit. The claims that survive coarse measurement — a 22-point refusal lever from eight framings, roughly three times a random direction, a gate that fires on nearly every attack, and a sign that depends on prompt formatting — do not rest on fine differences between arms. Anything that would have rested on such differences is not claimed here. The harness is scripts/eval_gate.py.

4.11 What the per-prompt signal turns out to be

§4.10 shows a write that moves refusal by a wide margin in aggregate. A fixed write does not move every prompt equally, though, so it is natural to ask whether the size of the swing on a particular prompt — call it that prompt’s dose — can be read off the prompt before anything is generated. This section reports an affirmative-looking answer on a cheap proxy, the controls that shrank it, a behavioural validation that appeared to destroy it, and a properly sampled instrument that restored it on one model and refuted it on a second. The sequence is at least as useful as the result.

Design. 120 held-out jailbreak templates from the dataset’s test split, each with one of twelve plain harmful requests appended; the 32 short framed attacks; the twelve bare requests; and 48 real benign prompts. 164 attacks, of which 15% lean toward compliance unsteered. Per prompt we record a first-token refusal-versus-compliance log-odds unsteered and under four writes — $\pm\alpha$ along the concept’s raw direction and $\pm\alpha$ along a random unit direction of the same norm. Five forwards per prompt, no generation. The dose is the sign-reversible half, $\tfrac{1}{2}(\Delta_{-\alpha} - \Delta_{+\alpha})$.

On that outcome, the dose is strongly predictable. A ridge from the tapped activations reaches out-of-fold Spearman $\rho = +0.81$, against a permutation null from the identical pipeline of -0.00 ± 0.10 over 300 draws — roughly 8 standard deviations. Folds grouped by harmful request give $+0.79$; a direction fit on the templates predicts the short and bare requests at $+0.88$; eight labelled prompts already reach +0.64.

Most of it was inside the concept direction, and the gate was reading it badly (Figure 16). Three scalars — the prompt’s projection onto the concept’s own steering direction at each tap — predict the dose at $+0.64$, where the gate’s calibrated likelihood-ratio read of the same activations manages $+0.51$ (Qwen2.5-0.5B; on gemma-2-2b the gate carries no dose information at all, see below). The full ridge adds strictly but modestly ($+0.61$ on what the projection leaves). So the nested ladder, not a new subspace, is the honest shape of the measurement.

Two claims from an earlier analysis of this data did not survive their own controls. We had measured the fitted direction’s alignment with the concept direction at mean $|\cos| = 0.09$ against a $1/\sqrt{d} = 0.033$ chance floor and read it as near-orthogonality — a separate effect-modifier direction. The right reference is not chance but how well the fitted direction agrees with itself: refit on disjoint halves of the same prompts and the halves agree at only $|\cos| = 0.19$, so the measured alignment is about 47% of the resolvable ceiling and the statistic cannot settle the question at this sample size. We had also offered “projecting the concept and disposition directions out still gives $+0.81$” as evidence the signal lay elsewhere; that control has no power, since projecting out two random directions gives $+0.81$ — two dimensions out of 2,688 change nothing. Projecting out the fitted direction gives $+0.09$, which is the informative version. The Cylindrical Representation Hypothesis had in any case already argued the qualitative version of that geometry on theoretical grounds [25].

And predictability on this outcome is not distinctive. The identical pipeline decodes the unsteered refusal logit at $+0.86$ and the sign-independent component of the response at $+0.86$ — both above the dose’s $+0.81$. Mid-layer activations linearly encode smooth functions of the prompt; that is the linear-representation result. What does separate the concept write from noise is that a matched-norm random direction’s dose is predicted at only $+0.44$.

Across magnitudes and four models

Three write magnitudes ($\alpha = 0.04, 0.08, 0.12$) on four instruct models spanning 0.36B to 2.6B and three families, three concept resamples each. The SmolLM2 pair is the useful contrast: same architecture and tokenizer, roughly five times the parameters.

The prediction replicates everywhere and strengthens with magnitude. The ridge reaches +0.61 to +0.84 at the largest write on all four models, and the dose stays a stable multiple of a matched-norm random direction (1.6–2.7×) at every magnitude — what a genuine directional effect should do, and not what a fitting artifact would do. That range is the ratio of seed-averaged doses; the per-seed ratio is far noisier (1.4–6.3), because the random direction’s own dose varies about threefold across seeds.

The gate’s own read carries dose information on three of the four. SmolLM2-360M +0.33, SmolLM2-1.7B +0.51, Qwen +0.51 — and gemma-2-2b -0.02 ± 0.18, with the sign flipping across its three resamples. So the gap between what the gate reports and what a purpose-fit read finds is the general pattern, and gemma is an exception to it rather than the rule. It is not a size effect: within the SmolLM2 family the correlation rises from +0.33 at 0.36B to +0.51 at 1.7B, while the largest model is the one that fails. Why gemma differs we do not know, and three resamples with that spread would not settle it.

The direction is the same across magnitudes. Directions fit at different $\alpha$ on the same prompts agree at $|\cos| \approx 0.90$ and each predicts the other’s dose at $\rho \approx 0.95$, so it is one direction rather than an artifact of one write size. That also reframes the split-half figure quoted earlier: 0.19 was computed on half the prompts each time and measures sampling variability across prompts, not how precisely the direction can be estimated. With the prompt set fixed the direction is well determined; whether it reproduces on a different prompt set is the open question, and the behavioural validation below bounds — and, on one model, confirms — what any of this is a statement about.

Judging the outcome against behaviour: the ruler, not the signal

Everything above is measured on a first-token log-odds over two hand-chosen token baskets — five refusal-opening ids against thirteen compliance-opening ones. That the gap itself is a meaningful per-prompt margin is established elsewhere [26]; what is at issue here is narrower, whether its response to a write ranks prompts the way behaviour does. Group means were checked against generated text, but the per-prompt ranking, which is what the prediction result is about, was not. So we measured the same 164 prompts two further ways: a teacher-forced continuation score, the length-normalised log-probability of canonical refusal continuations minus compliance ones, which spans many tokens and does not depend on what was generated; and a generated-refusal indicator over 40 greedily decoded tokens.

The first two fall below the 0.5 threshold we had set in advance, and within templates alone the first drops to +0.29. But the third row is the one that decides what can be said: the two behavioural measures agree with each other at essentially zero. That is not evidence that the per-prompt dose is unpredictable; it is evidence that these instruments could not measure it. Neither was ever shown to be reliable: the generated lever is one greedy sample per arm scored by a lexicon, a three-valued variable per prompt, and the teacher-forced lever compares three canned refusals against three canned compliances. Two instruments of unknown reliability disagreeing tells you about the instruments.

Replacing them with a sampled one settles it. Drawing $K=16$ continuations per arm at $T=0.7$ and scoring each with both the lexicon and an off-the-shelf rejection classifier, the per-prompt dose on Qwen2.5-0.5B is reliable — split-half $+0.66$ (lexicon) and $+0.70$ (classifier), Spearman–Brown $+0.80$/$+0.82$ at $K{=}16$, and the two instruments now agree with each other at $+0.88$ rather than at zero (Figure 17). Against that target the first-token proxy of this section reaches $+0.61$ per prompt, and a ridge on the same three tapped activations reaches $+0.58$ out of fold (folds grouped by harmful request) against a permutation null of $-0.015 \pm 0.089$ from the identical pipeline, $z=6.7$; the three concept projections alone reach $+0.51$, and the calibrated gate’s own LLR $-0.30$. Predicting the unsteered refusal rate instead reaches $+0.81$, but that rate correlates with the dose at $+0.03$, so the dose is not a restatement of which prompts refuse anyway.

What it is not. Three confounds could each have produced this number without any steerability being decoded, and two of them are tested against a high bar, because the taps decode the confound itself easily — had the dose secretly been one of them, the correlation would have been higher, not lower. It is not prompt length: the taps predict token count at $+0.90$, yet length correlates with the dose at $-0.12$ and the length-residualised dose is still predicted at $+0.57$. It is not prompt family: the taps separate jailbreak templates from short attacks at $+0.77$, yet the dose is predicted within templates alone ($+0.52$) and within short and bare requests alone ($+0.63$). And it is not refusal disposition, as above. Nor is it an artefact of our analysis choices: all three scorers give the same verdict (grouped CV $+0.54$/$+0.58$/$+0.60$, $z$ between $6.0$ and $6.5$, their out-of-fold predictions agreeing at $+0.87$), and the ridge penalty may vary a thousandfold for a swing of $+0.58$ to $+0.63$.

Used as a gate, the prediction selects where writing works (Figure 18). This is the operational form of the result and the reason it matters for the system rather than only for the measurement. Ranking held-out attacks by predicted dose and writing only to the top fraction — predictions taken out of fold — against a size-matched random subset drawn 500 times at each operating point:

The gate beats its null at every operating point, and the gain per write climbs monotonically as coverage tightens while the random baseline stays flat. That monotonicity is the substance: a lucky split would win at one threshold, whereas a real ranking of prompts by responsiveness keeps improving as you keep only its top. The contrast with the concept gate is the point of the whole exercise — asked is the concept present?, it writes to 154 of the 164 attacks and to every benign prompt in the set, gaining $+0.090$ per write; asked will the write move this prompt?, the same activations give $+0.143$ at half the writes and a tenth of the benign collateral. The read that answers the second question is the one a steering system should be gated on, and this system was not built that way.

This is one model. Repeating the measurement on gemma-2-2b at the same magnitude, the instrument is sound (split-half of the unsteered rate $+0.92$) but the effect is not: attack refusal moves $0.46 / 0.51 / 0.53$ across the three arms against Qwen’s $0.35 / 0.56 / 0.65$ on the same scorer, mean $\lvert D\rvert$ is $0.07$ against $0.17$, and only 38% of doses are positive. The dose’s split-half falls to $+0.53$, below the $0.6$ we had fixed in advance, and prediction on that unreliable target reaches only $+0.20$ at $z=2.2$.

Doubling the samples settles it, and not in our favour. Because the reliability bar had failed, the pre-registered response was to re-sample the same prompts with the same concept fit and merge to $K=32$. Attenuation theory makes a sharp prediction here: a more reliable target should make a real but noisy correlation clearer. The $K{=}16$ value of $+0.20$ implies a true correlation of $0.24$, which at $K{=}32$’s reliability should surface as $+0.215$. It fell instead to $+0.078$, $z=0.9$ — no effect. A signal masked by measurement noise sharpens when you measure better; noise does not. The supporting pattern agrees: cross-family transfer collapsed from $+0.12/+0.09$ to $+0.03/-0.02$, and the fitted direction’s split-half self-consistency fell to $0.04$–$0.05$. There is no direction there to find. So the honest verdict on gemma-2-2b is not “underpowered” but no detectable per-prompt dose signal, at a magnitude which barely moves it.

One methodological point generalises beyond this paper, and we got it right only because we wrote the prediction down first. A split-half reliability gate is not comparable across sample sizes. At $K{=}32$ it compares two 16-sample halves, so it measures the reliability of a 16-sample estimate — a quantity the $K{=}16$ run had already measured via Spearman–Brown. gemma duly “passed” the reliability bar at $K{=}32$ ($0.69$) having failed it at $K{=}16$ ($0.53$), purely because the statistic got easier; we predicted the value to three decimals in advance ($0.694$ predicted, $0.692$ observed) for exactly this reason. Anyone gating a decision on measurement reliability should compare Spearman–Brown-corrected full-sample figures, or split-halves at matched half-size, never the raw statistic across different $K$.

The greedy measures did agree strongly on the aggregate effect all along — the generated refusal rate moves 64% → 35% → 79% and the continuation score moves the same way — so §4.10’s group-level conclusions never depended on any of this.

What we therefore claim, and do not. We claim that a statistic of the first-token distribution is predictable from the prompt’s activations far above a matched null, that most of that predictability lives in the concept direction the system already computes, and that the calibrated gate is a lossy readout of it on three of the four models tested, gemma-2-2b being the exception. We claim, on one model, that per-prompt steerability of behaviour is measurable once the instrument is sampled rather than greedy, and predictable from the prompt’s activations above a permutation null. We do not claim that this generalizes: it did not reproduce on the second model tested, decisively rather than marginally. What would settle it is a second magnitude, so each prompt’s dose is a fitted slope rather than a two-point difference, and more base models — the gemma result shows that a magnitude which moves one model appreciably can sit at the sampling-noise floor on another, and at that floor nothing is predictable because nothing reliable is being measured. Two further limits stand regardless: the three concept resamples reuse one fixed prompt set with byte-identical activations, so the effective number of independent replicates is one; and 98% of doses share a sign, so only magnitude is at issue, never direction.

Prior work. The nearest result predicts whether an intervention will under- or over-steer from internal states, but requires running the steered pass and decoding several tokens, predicts a three-class label, and uses features that are explicitly alignments with the steering vector [24]. Others predict per-instance intervention properties from the prompt alone but target which layer to steer [22]. A recent result does forecast steering outcomes from unsteered representations, but predicts which other behaviours an intervention will disturb, across a taxonomy of behaviours, rather than how far a given prompt will move [27]. We should also be clear about what is ours in the measure itself: the first-token refusal-minus-compliance statistic this section is built on is a basket-sum variant of the refusal–affirmation logit gap, which was introduced as a per-prompt safety margin and validated across thirteen models [26]. The instrument is theirs; the question we ask of it — whether a prompt’s response to a write is predictable before the write — is the part we could not find in the literature. The methodological caution this section ends on is the part we would most want a reader to take away, and it is not the one we first drew. A cheap first-token proxy turned out to track the sampled behavioural dose per prompt at $+0.61$; what failed was the instrument we validated it against — a single greedy generation scored by a word list. An outcome measure whose own reliability is never checked can manufacture a null as easily as a result, and we published one before checking. The harness is eval_gate.py --steerability, the controls are steerability_controls.py, and the validation is eval_behaviour_check.py. The sampled behavioural instrument, its slice driver and its analysis are eval_behaviour_dose.py, run_behaviour_dose.sh and analyze_behaviour_dose.py; every sampled continuation is stored with the results, so each number above reproduces without regenerating. The read itself ships in the library as an opt-in OutcomeHead, fitted by cg.learn_outcome(...) and used as Steer(when=Both(Trigger.FIRE, Predicted("dose", τ))) — labelled, in its documentation, with exactly the one-model evidence reported here.

5. Discussion

5.1 What is contributed

Little of the mechanism. Probes, diff-of-means directions, activation steering, Gaussian and mixture density scoring, circuit-breaker reroute, forward-hook truncation and conditioning a write on an activation read are all established, and the specific composition has a published precedent. Measured against fair baselines, detection is a commodity, the depth fusion does not transfer, the bank’s amortization is shared with a probe bank, and conditioning the write adds nothing the read side can take credit for, because on correctly formatted attacks the gate fires on almost all of them (§4.10).

What the work does contribute is evidence. A direction fit from eight hand-written examples is a genuine bidirectional lever on a frozen model’s refusal behaviour, moving generated refusal across a 43-point range and holding a stable multiple of a matched-norm random direction across three magnitudes and two models. The same object is fragile in a way worth knowing before anyone deploys one: its false-positive rate moves from 18% to 96% with the register of the benign traffic. Around those sit a set of negative results that were expensive to obtain and are cheap to reuse, and one caution about measurement: an outcome instrument whose reliability is never checked can manufacture a null as easily as a result.

One positive result sits apart from these, and it is the only thing here we would defend as new. Asked not is the concept present? but how far will a write move this prompt?, the same three tapped activations answer at grouped $\rho=+0.58$ against a permutation null, $z=6.7$ — while the system’s own calibrated gate manages $|0.30|$ on that target and with the opposite sign. It is operationally useful rather than merely statistical: ranking prompts by predicted dose and writing only to the top beats a size-matched random subset at every coverage, at a fraction of the benign collateral. It holds on one model and is absent on the second, where the write barely moves the model at all (§4.11).

The contribution is mostly the measurement and its honesty rather than the machinery — with that one exception.

5.2 Detection is a commodity; steering is prior art; what the corrected write shows

The structural finding stands: a text classifier can match or exceed ConceptGate at detection while being simpler to deploy, so if detection were the objective there would be little reason to prefer an internal method. It is tempting to say next that the read side’s contribution is deciding when to write, and that gating the write therefore beats writing everywhere. The measurements do not support it for the concept read, which is the read this system was built around. Conditioning a write on an activation detector is CAST and DSAS [15][16], and on correctly formatted prompts the gate fires on 97% of attacks and selects nothing a size-matched random subset does not (§4.10).

A read fit to something else is a different matter. Fit to the write’s own measured per-prompt effect rather than to concept membership, the same activations do select: ranking attacks by predicted dose beats a size-matched random subset at every coverage tested, raising the refusal gained per write from $+0.108$ to $+0.242$ while writing a fraction of the benign traffic (§4.11). So the honest form of the negative is narrower than it first appeared: gating on presence adds nothing, because presence is nearly constant on formatted attacks; gating on predicted effect does.

What the corrected write shows is smaller and older: a direction fit from eight hand-written framings, added at 8% of the residual norm, moves the instruct model’s refusal by about 22 points, in either direction depending on sign, about three times as much as a random direction of the same norm. That is the refusal direction of Arditi et al. [17] obtained few-shot and training-free — a lever an operator could point either way, and a jailbreak if pointed away. The gate’s remaining function is the one CAST reports — confining the write to prompts where the detector fires, which spares benign generation — bounded by a false-positive rate that moves from 18% to 96% with the prompt register.

5.3 The cost argument and its limits

The compute–accuracy trade-off is a real engineering result. On 262 held-out prompts a jailbreak concept reaches AUC $0.970\pm0.011$ from a single tap at 46% of Qwen2.5-0.5B’s depth, loading 61% of its weights, against $0.982\pm0.003$ for a probe on the complete model (§4.8.1). Two qualifications bound it. First, the truncated-forward saving is available to any internal probe, including the SVM baseline; it is a property of latent-space methods in general rather than an advantage specific to ConceptGate. The sharper and better-measured version of the cost claim is at the bank level (§4.8.2): a training-free concept bank amortizes across a taxonomy — flat inference and closed-form, kilobyte-scale extension where fine-tuning pays seconds-to-minutes and a fresh forward per concept — but that advantage, too, is shared with a linear-probe bank, so what remains specific to ConceptGate is not the reading cost but that each entry comes with a write direction fit from the same data — a measured behavioural control for jailbreak framing and topical concepts, a reversal for the harm categories (§4.10) — and, where labels for a measured outcome exist, a third read fit to that outcome (§4.11), which is the only place the read side is shown to select better than a size-matched random subset. Second, the memory-minimal load mode is detection-only, and detection is the commodity half of the system, whereas the distinguishing capability, steering, requires the full network. The cost argument therefore applies to the guardrail rather than to the steerer. The defensible claim is that a read-and-write adapter can be added to a model already being served, for kilobytes of parameters and a fraction of a forward pass — not that it detects more accurately than the alternatives.

5.4 What it actually costs

The cost can be made concrete, since it is the primary reason to assemble this composition rather than deploy a second model. A single concept’s entire learned state, over $m$ tapped layers of residual width $d$, is two sets of direction vectors (the standardized detection direction and the raw steering direction, $2md$ numbers), the per-dimension standardization statistics ($2md$), the depth filter ($m$), and a handful of Gaussian scalars for the gate. For GPT-2 with five taps that is on the order of $1.5\times10^4$ numbers — comfortably under the sub-million-parameter target one would want for something meant to be stored and shipped by the concept — and a bank of $K$ concepts is simply $K$ times that, since concepts store no shared parameters and, at inference, interact only through the max-LLR rule of §3.9. Fitting is not training: it is a few sample means and one small $m\times m$ solve for the filter, completing in milliseconds on a CPU with no backpropagation and no gradients, so a concept can be learned, discarded, and re-learned interactively. Inference adds $m$ dot products of width $d$ plus a length-$m$ blend per gated position — negligible against a single transformer forward — and in the abort case it removes compute, since decoding stops early. The reference implementation (github.com/NISH1001/conceptgate) keeps a deliberately legible shape: a pure-numpy mathematical core (spectral.py for directions, spectrogram, and filters; concept.py for the calibrated gate; mixture.py for the GMM and its BIC selection) sits behind a thin PyTorch boundary that owns only the model-touching parts — the tap reader, the steering hooks, and the ConceptGate facade with its Abort / Steer / Emit strategies. This separation keeps the numerical method auditable in numpy while the framework-specific code remains small enough to re-derive, which is appropriate for a method whose value rests on being inexpensive and transparent.

5.5 An inventory, and what is still open

Prior work, not ours: conditional activation steering; the collateral advantage of conditioning a write; refusal riding a single direction; unrelated writes eroding refusal; a condition detector’s false-positive rate collapsing off-distribution; harmfulness and refusal as distinct directions. Each is cited where it appears.

Measured here and found not to belong to ConceptGate: detection accuracy, single-concept efficiency, depth fusion, bank amortization, and any gain from conditioning the write on the concept read — the outcome-fitted read below is the exception, and is the one positive result we keep.

Measured here and standing: the bidirectional write and its magnitude scaling; the register fragility; the negative results listed in §1.4 (item 1); the caution about cheap outcome measures; and — on one model — that per-prompt steerability is measurable and predictable before generation, which is the only claim here we believe to be new.

Answered, on one model, with an instrument we had to build first. Whether the size of a write’s effect on a particular prompt can be read from that prompt beforehand. We previously listed this as unanswerable, because the two behavioural measures built to check it agreed with each other at -0.01. That was a fact about those measures, not about the quantity: neither had ever been checked for reliability, and one of them took three distinct values per prompt. Sampling the outcome instead — sixteen continuations per arm, scored by two independent instruments — makes the per-prompt dose reliable ($+0.66$/$+0.70$ split-half, the two scorers agreeing at $+0.88$), and on Qwen2.5-0.5B it is then predictable from the prompt’s activations at $+0.58$ out of fold against a permutation null of $-0.015 \pm 0.089$, $z=6.7$. It is not prompt length, not prompt family, not refusal disposition, and not an artefact of the scorer or the ridge penalty; used as a gate it beats size-matched random selection at every operating point (§4.11). This is the one result in this report we would defend as new, and the read now ships in the library as an opt-in feature carrying exactly that one-model evidence.

Still open: whether it is a fact about language models or a fact about Qwen. It did not reproduce on gemma-2-2b, and the reason is specific rather than mysterious. At the magnitude we used, the write moves that model’s refusal from $0.46$ to $0.53$ where it moves Qwen’s from $0.30$ to $0.64$; with almost no per-prompt variance in the outcome there is nothing for a predictor to find, and doubling the sample count made the apparent correlation fall rather than rise, which is how noise behaves and attenuated signal does not. The concept direction is no less special on gemma — it outperforms a matched random direction by the same $1.5$–$1.8\times$ on both models — its effect is simply smaller. So the experiment that would settle the question is a sweep over write magnitude on each model, so that every model is measured where its write actually bites and each prompt’s response becomes a fitted slope rather than a two-point difference. Until that exists, the honest summary is: measurable and predictable on one model, absent on a second where the intervention barely acts, and untested everywhere else.

6. Limitations and threats to validity

The most important limitation, which should be read before any other, is adversarial fragility. ConceptGate is a latent-space defense, and latent-space defenses are known to be breakable: obfuscated-activation attacks can drive a harmfulness probe’s recall from 100% to 0% while the model’s behaviour is unchanged, by finding inputs that keep the activation off the probe’s direction even as the model performs the prohibited action [10]. This attack class targets exactly the family ConceptGate belongs to — linear probes, SAEs, and Gaussian/mixture density gates alike — so nothing in §4 should be read as a security guarantee. In practical terms, ConceptGate is best regarded as a cheap, interpretable, few-shot layer within a defense stack — useful because it is nearly free to add, but not a boundary that a motivated adversary cannot cross. Its steering side is somewhat more robust in this respect than its detection side, since writing a direction alters behaviour whether or not an attacker knows the direction, but steering is not a filter, so the two serve different purposes.

The second group of limitations concerns the evidence being small and in-distribution. We evaluate primarily on GPT-2 and Qwen2.5-0.5B, small enough that the core results reproduce on a single CPU, and add gemma-2-2b for the multi-concept, generalization, and read/write-cosine results (§4.8–4.9, §3.10) — the steering dose-response of §4.6 is GPT-2 and Qwen only and the gate experiment of §4.10 is Qwen only — which we run on an Apple M4 GPU (MPS). That choice bounds how far the numbers extend. The qualitative findings — detection is a commodity, and the cost trade-off is real and model-dependent — hold on all three models we tested (the base model bounding steering quality is measured on GPT-2 and Qwen in §4.6; gemma-2-2b is steered in §4.11), and gemma-2-2b is the useful data point here: it is roughly five times Qwen2.5-0.5B and the qualitative pattern is unchanged on the read side, with the read/write cosine even slightly higher ($\approx0.6$ against $\approx0.5$). That is evidence, not proof, and 2B is not 8B; we expect the qualitative findings to survive to the 2–8B instruct scale on the strength of it, while the specific AUCs, error rates, and knee locations should be re-measured there before being quoted. Within these small models, the detection numbers are in-distribution: probe-based detection is known to generalize poorly off-distribution. §4.9 tests one clean version of this — holding the concept fixed and holding out whole harm categories — and finds only partial generalization (a direction trained on thirteen categories catches most unseen fourteenths above chance — two of fourteen fall to or below it — and well below in-distribution), with ConceptGate no less robust than a trained probe. But a substantially different attack style is a larger shift than a held-out category, and that is not tested here. And because the whole method rests on a linear direction, any concept that the frozen model encodes non-linearly is invisible to it; the intended mitigations — the layer sweep, and an MLP-probe variant that trades interpretability for capacity — are gestured at here but not fully explored.

The third cluster is about the few-shot regime and generation quality, which are the practical edges where the method frays. Everything downstream depends on the diversity of the ~10 prompts per side: a narrow or accidentally-correlated prompt set produces a direction that separates the training examples and little else, so results should always be reported with variance across seeds and prompt sets, which we have done only partially. We can put a number on that hazard, and it is a large one. The jailbreak gate of §4.10, fit from eight short override framings against eight short benign requests and run with the chat template, fires on 18% of benign prompts in that same register and on 96% of real benign prompts drawn from a different one (30 or 31 of 32 in every resample) — with the concept and the threshold unchanged. A ten-example concept is therefore calibrated to a register as much as to a meaning, and any deployment claim has to be made against the traffic the gate will actually see. The same failure is documented for CAST’s condition detector [20] and for activation probes generally [23]. On the write side a false fire rewrites the output rather than merely raising a flag. On the write side, steering hard enough to reliably change the topic also degrades fluency, and generated text drifts out of the clean-prompt distribution as it grows — degenerate repetition alone can nudge a benign continuation across the gate — so a deployed system must tune its operating point against false-refusal and output quality, not against recall in isolation.

A fourth cluster bounds the one positive result. The per-prompt dose finding of §4.11 holds on one model of the two tested, and the model where it fails is not a mystery but a warning: at the magnitude we used the write barely moves gemma-2-2b at all, so a result of this kind is only as portable as the intervention’s effect size on the model in question. Within the model where it does hold, the concept is fit once per model and the three resamples share a fixed prompt set, so the effective number of independent replicates is one; there is a single write magnitude, so each prompt’s dose is a two-point difference rather than a fitted slope; and 98% of doses share a sign, so what is predicted is magnitude and never direction. The labels are not free either — each costs a round of sampled generation, which is the real price of fitting a gate this way. Finally, one instrument-level caution that bit us: a split-half reliability gate is not comparable across sample sizes, so a model can fail such a gate at one sample count and pass it at another with nothing changed but the arithmetic; compare Spearman–Brown-corrected full-sample figures instead.

None of these limitations is incidental; each corresponds to a parameter that the operating point exposes, and they are stated here so that the results are read with appropriate caution.

7. Conclusion

A frozen model already represents many concepts of interest in its residual stream; ConceptGate reads them across depth and writes them back, from about ten examples and without training. Measured component by component, the reading is a commodity — no more accurate than a linear classifier and no cheaper than a depth-matched probe on the same taps — and the extensibility of a training-free bank, real against fine-tuning, is shared with a probe bank. Conditioning the write on the concept read, which looked like the distinguishing capability, adds nothing: the gate fires on almost every attack and selects nothing a size-matched random subset does not. Conditioning it on a read fit to the write’s own measured effect is a different story, and it is the one result here we would defend as new: ranking prompts by predicted dose lifts the refusal gained per write from $+0.108$ to $+0.242$ and beats a size-matched random subset at every coverage we tested, while writing a fraction of the benign traffic the concept gate writes to. On one model. On the second, the same write barely moves the model at all, and there is correspondingly nothing to predict (§4.11).

What the measurements do support is narrower and, we think, still worth having. The write is a real bidirectional lever on a frozen model: eight hand-written examples move generated refusal from 64% to 35% in one direction and 79% in the other, scaling cleanly with magnitude and holding a stable multiple of a random direction on all four models tested. It is also fragile in a specific, measurable way — the same gate’s false-positive rate moves from 18% to 96% with the register of the traffic it sees — which is the kind of thing a deployment claim has to be made against. A further question, whether the size of that write’s effect on a particular prompt can be read from the prompt beforehand, is answerable on one of the two models we tried it on, and sits at the sampling-noise floor on the other (§4.11); saying which is more useful than the general affirmative the proxy alone would have licensed.

Three lessons generalize past this system. A binary or first-token outcome near its ceiling cannot resolve an intervention in either direction, however consistent it looks across seeds. Before concluding that a per-prompt signal is absent, measure whether the instrument you are judging it by can see one at all — we published a null that was our ruler’s, not the model’s. And a split-half reliability gate is not comparable across sample sizes: at $K$ samples it measures the reliability of a $K/2$-sample estimate, so the same model can fail such a gate at one sample count and pass it at another with nothing having changed but the arithmetic; compare Spearman–Brown-corrected full-sample figures instead. The interactive figures are included so that these claims can be examined directly against the underlying model runs rather than taken on assertion; the points at which the method is effective and the points at which it fails are both visible in them.

References

Citation

@techreport{nish2026conceptgate,
  author      = {Pantha, Nishan},
  title       = {ConceptGate: Learning and Steering Concepts in Language Models},
  institution = {Bits and Paradoxes},
  type        = {Technical Report},
  year        = {2026},
  month       = aug,
  url         = {https://nishparadox.com/research/conceptgate/},
  note        = {Few-shot, training-free concept detection and steering via a
                 depth-wise spectrogram of residual-stream projections; reference
                 implementation at https://github.com/NISH1001/conceptgate}
}

Plain text:

Pantha, N. (2026). ConceptGate: Learning and Steering Concepts in Language Models. Technical Report. https://nishparadox.com/research/conceptgate/