ConceptGate: Efficiently Learning and Steering Concepts in Language Models

⚠ Work in progress. This is a living technical report, co-written with a generative model — actively evolving and updated as the research develops, and not a finished or peer-reviewed paper. Treat the results, numbers, and framing as preliminary, and expect sections to change.

1. Introduction

Large language models are typically deployed as fixed artifacts: their weights are set during training and left unchanged during use. A substantial body of work nonetheless shows that a model’s internal activations reveal a great deal about what it is computing — that specific, human-nameable properties of an input or a generation are represented, frequently along linear directions, in the residual stream, and that these representations can be both read and modified without retraining. This report examines what can be built from that observation under deliberately restrictive conditions: no fine-tuning of the host model, only a handful of labelled examples per concept, and a parameter budget small enough that a separate module can be stored for each concept. The result is ConceptGate, an adapter that attaches to a frozen model, detects a chosen concept from its intermediate activations, and, using a closely related direction fit from the same data, steers the model’s generation with respect to that concept. The remainder of this section states the problem precisely (§1.1), explains why the residual stream is the appropriate place to operate (§1.2), identifies the two design choices that distinguish the method from existing probing and steering work (§1.3), and summarizes the contributions together with their limitations (§1.4).

This objective is best understood against the broader space of methods for adapting a pretrained language model to a downstream task, which differ in cost, in expressiveness, and in how invasively they alter the model. Full fine-tuning updates all of the model’s weights; it is the most expressive option and the usual point of reference, but it is expensive and produces a separate copy of the model for every task. Parameter-efficient methods — adapter modules [12] and, most prominently, low-rank adaptation (LoRA) [11] — freeze the pretrained weights and train only a small number of additional parameters, reducing cost substantially while still relying on gradient-based training and still changing the model’s function. Linear probing freezes the backbone entirely and trains a lightweight linear readout on its activations, which yields an inexpensive detector but offers no means of altering behaviour. ConceptGate lies at the least-invasive end of this range: it performs no gradient-based training, estimating each concept in closed form in a low-sample, few-shot regime — on the order of ten examples per concept — and it operates on activations rather than weights. Unlike a linear probe, it uses the learned direction not only to detect the concept but to write it back into the residual stream and steer generation, which places it closest to the representation-engineering and activation-steering methods — RepE [3], ActAdd [4], and CAA [5] — discussed in §2.1–§2.2; it differs from those mainly in reading a concept across several layers rather than one and in coupling detection and steering within a single calibrated module. The trade-off is deliberate: steering a frozen model by adding a single linear direction per concept is far less powerful than adapting it by fine-tuning or LoRA, and the method is directed at lightweight, interpretable, concept-level control rather than at acquiring new capabilities. That linearity, however, is a property of the write side. The read side is more expressive: it fuses several per-layer directions and scores them with a Gaussian-mixture model, whose decision surface becomes non-linear whenever a class is best described by more than one component — though in the low-sample regime we work in, model selection usually returns a single component and the boundary is again linear (§4.2).

1.1 The problem

Consider a frozen language model $M$: it can be run, and its activations can be read and modified through forward hooks, but its weights are never updated. The objective is a small attached module $G$ that, from a handful of labelled examples, can decide whether a chosen human-named concept — a jailbreak attempt, a topic such as cooking, a hostile tone — is present in the model’s computation, and can then act on that decision by halting generation or altering its course. Four constraints shape the design, and each follows from an intended deployment. The module should be few-shot, learning from roughly ten labelled prompts per class, because curated concept sets are expensive to produce; it should be lightweight, well under a million parameters, so that one instance per concept is inexpensive to store and distribute; it should be attachable to any architecture without retraining the host; and it should be bidirectional, applicable both to the input prompt and to each token the model subsequently generates.

These requirements can be stated precisely. Let $M$ be a frozen model whose residual-stream activation for an input $x$ at a tapped layer $\ell\in\mathcal{L}$ is $a_\ell(x)\in\mathbb{R}^{d}$, and let $\mathcal{D}^{+}$ and $\mathcal{D}^{-}$ be small labelled sets of examples that do and do not exhibit the concept, with $\lvert\mathcal{D}^{+}\rvert+\lvert\mathcal{D}^{-}\rvert\approx 20$. From these alone, and without modifying the weights of $M$, the module $G$ must construct two maps — a detector and a steering operator:

The detector maps the tapped activations of a prompt to a three-way decision; the steering operator, applied at each tapped layer during generation, adds a learned per-layer direction $w_\ell$ scaled by a strength $\alpha$, with $\alpha<0$ suppressing the concept and $\alpha>0$ amplifying it. Both the detector and the directions must occupy $O(\lvert\mathcal{L}\rvert\,d)$ parameters per concept and be obtained in closed form from the few-shot sets rather than by gradient descent. The sections that follow construct $g$ and the direction $w_\ell$ and show that both can be estimated from the class means of the tapped activations.

Guardrailing is the immediate application, and the setting in which the method was first developed, but the mechanism is not specific to safety: nothing in it distinguishes “harmful” from any other property. $G$ is concept-agnostic — a general detector and steerer for which guardrailing is only one concept among many — so the same construction serves equally as a content filter, a topic router, a tone monitor, or a stylistic control. The concept enters as data, not as code.

1.2 Why the residual stream

Every mechanism in ConceptGate lives on the residual stream: the running vector that a transformer reads from and writes to at every block, the channel through which information flows from the embedding to the unembedding [1]. Two empirical facts make it the right place to work. First, many concepts are approximately linearly readable from the stream — a single direction separates positive from negative examples well above chance [2][3]. Second, the stream is writable: a closely related direction, added back, changes what the model goes on to say [4][5]. Reading and writing therefore share their few-shot fitting data and a closely related direction (per-tap cosine 0.45–0.79, §3.10) — and it is this shared structure that the rest of the method is organized around.

1.3 The gap: depth, and the read/write duality

Two observations shape the design. The first concerns depth. Most probing and steering methods commit to a single layer, selected by a validation sweep, and read or write only there. A concept is not, however, equally legible at every depth: it is weakly represented in the early layers, where the model is still resolving surface form; most clearly represented at intermediate depth, where the abstraction has formed; and increasingly diffuse in the late layers, which specialize toward next-token prediction. When a concept leaves a usable trace at several depths, reading only one of them ought to discard available signal. ConceptGate therefore projects the concept at every tapped layer and treats the resulting profile across depth as a single signal to be filtered. This amounts to a signal-processing view of the residual stream, in which the network’s depth is the signal axis and combining the layers into one decision is a filtering problem — solved, as a matter of classical theory, by a matched filter over depth rather than by a hand-picked layer. The view is developed in §3.4 and justified by a matched-filter argument in §3.6. We flag at the outset that the expectation is not borne out. The argument needs the per-layer noise to be independent; the fusion recovers exactly the predicted gain on synthetic data built to satisfy that assumption (§4.1), and on real models the tapped layers are correlated enough that a linear probe on the same concatenated taps matches or beats the fusion (§4.8.1). Depth fusion is reported in this paper as a mechanism that does not transfer, not as a contribution.

The second observation is the read/write duality noted above: a linear detector and a linear steerer are closely related directions, fit from the same data, applied in the two directions of information flow, whereas a text classifier — the conventional guardrail — can only read. This asymmetry is the principal reason to operate inside the residual stream rather than on the text, and it recurs throughout the analysis that follows.

1.4 Contributions

This paper contributes, in order of how much each distinguishes ConceptGate from a plain probe:

1. Steering — the read/write duality, measured. The one operation a detector or classifier cannot perform: a direction fit from the same ten examples as the detector — related to it but not identical (per-tap cosine 0.45–0.79 depending on model, concept, and detection mode; §3.10) — is written back into the residual stream to steer generation toward or away from the concept, a monotonic dose-response with a coherent operating window, bounded by the base model (§4.6, §3.10).
2. Gate-conditioned steering — the one operation only the composition can do. The write rule is standard activation addition and needs no detection machinery; what the read side adds is deciding when to write. Conditioning the write on the calibrated gate beats writing on every prompt on both axes simultaneously — more jailbreak suppression (+8.3 versus +2.1 points) and far less collateral (90% versus 4% of benign generation left byte-identical) (§4.10).
3. Training-free amortization across a concept bank. Adding a concept is a closed-form fit in milliseconds and kilobytes with no gradient run, so hosting a fourteen-way safety taxonomy costs a fraction of per-concept LoRA fine-tuning and needs no retraining to extend (§4.8.2). This amortization is shared with a linear-probe bank; what ConceptGate adds is that each entry also supplies a steering direction at no extra fitting cost — though for these harm categories the write is a measured null on behaviour (§4.10).
4. A calibrated, few-shot, dual-mode adapter. One object learns a concept from ~10 examples, detects it with a calibrated fire/abstain/pass gate, and steers along a closely related direction, with a small, well-characterized parameter budget (§5.4) and no training.
5. Negative and honest results, reported as prominently as the positives. Detection is a commodity a linear SVM matches (§4.3) and a depth-matched probe matches at the same compute (§4.8.1), so single-concept efficiency is the truncated forward, not ConceptGate; depth fusion helps only on synthetic data matched to its own assumptions and does not transfer to real models (§4.1, §4.8.1); matched contrastive negatives hurt (§4.4); the mixture collapses to a single Gaussian at few-shot sizes (§4.2); a predicted paraphrase-robustness effect does not appear (§4.7); generalization to an unseen harm category is only partial (§4.9); and steering with the bank’s own harm-category directions is a null on behaviour — the write direction comes free, but it does not make the model decline (§4.10).
6. Reproducible interactive figures. The figures below reproduce the underlying model runs, so the mechanism can be examined directly rather than only described.

One caveat applies throughout: no individual mechanism here is new. Probes, diff-of-means directions, activation steering, Gaussian/mixture density scoring, circuit-breaker reroute, and forward-hook truncation are all established. The contribution is their specific composition and the empirical measurement of it.

2. Related work

ConceptGate is a recombination, not an invention, so its lineage is unusually wide: almost every part of it is the standard tool from some established line of work, and the design is mostly a set of decisions about which standard tool to use for each job and how to fit them from one shared few-shot set of examples in the residual stream. The appropriate way to survey the field is therefore not to identify a single neighbouring method and compare against it, but to trace the ancestry of each component and identify the small part that is new. We organize the survey around the five lines of work the system draws on, and in each we state what is borrowed and what, if anything, is added.

Two of the threads are about reading the stream. Linear probing and representation engineering (§2.1) give us the per-layer detector and the diff-of-means direction; density-based out-of-distribution scoring (§2.3) gives us the calibrated, class-conditional likelihood-ratio gate. Two are about acting on it: activation steering and circuit breakers (§2.2) give us the write side — a related direction, added back — while the external-classifier literature (§2.4) is the incumbent we are implicitly compared against, and the one whose central limitation (it can only read text, never write activations) is the negative space that defines what ConceptGate is for. The fifth thread is about cost: early-exit and conditional computation (§2.5) is where our truncated forward and the compute–accuracy frontier come from.

The element that ties these borrowings into one system — and the only part specific to this work — is the pair of design commitments stated in the introduction: read the concept across depth rather than at a single chosen layer, and fit the detector and the steerer from one set of examples so they act in the two directions of information flow — closely related directions rather than one shared vector (per-tap cosine 0.45–0.79, §3.10) — so that a frozen model can be turned into a few-shot, calibrated, read-and-write concept adapter without any training. We close the section (§2.6) by making that positioning explicit, including the adversarial caveat that bounds the whole class of method.

2.1 Probes and representation engineering

Linear probing is the oldest tool in the stack: fit a linear classifier to a layer’s activations and you can read off whatever the model has linearly encoded there. Alain and Bengio introduced probes as a diagnostic — a way to watch information appear, sharpen, and fade across depth — and showed that intermediate representations carry a great deal of decodable structure [2]. Representation Engineering (RepE) turned that diagnostic idea into an operational one: it argues that many high-level concepts — honesty, harmfulness, power-seeking, particular emotions — lie along identifiable linear directions in the residual stream, and that those directions can be read to monitor a model and pushed to control it [3]. ConceptGate’s detector is, mechanically, one of these linear probes, and its specific direction — the difference of the two class means — is precisely the “reading vector” RepE constructs from contrastive examples; we claim novelty for neither. What the probing and RepE lines almost universally do, and what we deliberately break from, is to commit to a single layer, chosen by a validation sweep, and read or steer there. Because a concept leaves a usable trace at several depths, discarding all but one ought to throw away signal; ConceptGate instead reads the probe’s output at every tapped layer and treats the resulting profile-across-depth as one signal to be fused (§3.4–§3.6). That single change — from “pick the best layer” to “combine the layers” — is the only place in the reading path where we depart from established practice, and it is a departure that does not pay off: on real prompts a probe on the same concatenated taps matches or slightly beats the fusion (§4.8.1), so the established single-layer practice gives up nothing worth recovering.

2.2 Activation steering and circuit breakers

The write side has an equally direct lineage. Steering a model by adding a contrastive direction into its residual stream at inference time is Activation Addition (ActAdd) [4] and, in the form we adopt most directly, Contrastive Activation Addition (CAA), which builds the steering vector from the mean difference of paired positive/negative prompts and adds it during generation to shift behaviour along a named axis [5]. Our steering rule is literally theirs — add $\pm\alpha\,w^{\text{raw}}$ at the tapped layers. ConceptGate’s steering direction is the diff-of-means of the same few-shot examples the detector is fit from — a related direction (per-tap cosine 0.45–0.79, §3.10), not the detector’s standardized, and in practice logistic, direction itself. The guardrail-flavoured cousin is Circuit Breakers, which makes a model reroute its own harmful representations so that continuing down a harmful path collapses into refusal [6]. The decisive difference is training: Circuit Breakers fine-tunes the model against a curated set, buying robustness at the cost of a training run and a modified model, whereas ConceptGate steers a frozen model from roughly ten examples, buying cheapness and interpretability at the cost of power — a single linear nudge is weaker than a trained reroute. Our contribution here is therefore not the steering rule but its packaging: the write side of a detector fit from the same data, dialed as a fraction of the residual norm so the same setting transfers across models (§3.10), and gated so it fires only when the concept is actually present.

2.3 Density-based detection and out-of-distribution scoring

The gate that turns a score into a decision comes from the out-of-distribution literature. Modelling a class as a Gaussian in feature space and scoring new points by Mahalanobis distance is the classic deep-OOD detector [7], and it underlies a long line of generative-classifier and density-based detectors since. ConceptGate’s calibrated likelihood-ratio gate is a two-class instance of that idea — a Gaussian per class and a threshold on the log-ratio — and the mixture extension of §3.7 generalizes each class to a Gaussian mixture whose component count is chosen by the Bayesian Information Criterion, so that a multimodal class (“benign” = chit-chat and homework and code) is not forced into one blob. The narrow slice we can call our own is where the density lives: not on a single feature vector, but on the joint spectrogram across depth, so that correlations between what different layers report are part of the model rather than being averaged away. As the experiments will show (§4.2), this generality is real but largely dormant at ten-shot sample sizes, a property of the model-selection criterion rather than an observation made after the fact.

2.4 External classifiers

ConceptGate reads and writes arbitrary concepts from a frozen model’s activations; safety is only the domain where a public labelled comparison is readily available, so it is where the contrast with an external classifier is clearest. Where the concept is a safety policy, the incumbent is the external text classifier — Llama Guard and the family of input/output safety models around it [8] — which read the prompt or the completion as text and classify it against a policy. These are strong, generalize well because they are trained on large labelled corpora, and are the right answer when detection quality is the only thing that matters. But three structural differences define the space ConceptGate occupies, and they hold for any concept, not just safety. An external classifier is a second model to load, serve, and pay for alongside the one already running; it operates purely on text, so it sees nothing of the host model’s internal state and cannot exploit the fact that the host has already computed the concept; and it can only read — it can flag a concept but cannot reach into the generation and bend it. ConceptGate makes the opposite trade at every point: it rides the model already in memory, adds kilobytes rather than a network, reads the concept straight from the activations the host produced for free, and can write. It will not out-detect a well-trained classifier (§4.3); its reason to exist is the write capability the classifier structurally lacks.

2.5 Early exit and conditional compute

The efficiency story has a lineage too. Stopping a forward pass early once the model is confident is early-exit / conditional computation, of which CALM is a representative example: it learns to emit a token from an intermediate layer when a confidence measure says the remaining layers will not change the answer [9]. ConceptGate’s truncated forward (§3.12) is the same principle aimed at detection rather than generation: to read a tap at layer $\ell$ you only have to run blocks $0$ through $\ell$, so a concept that is already legible in the lower half of the network can be caught having run only the lower half. The compute–accuracy frontier we measure in §4.5 is precisely the concept-detection analogue of an early-exit curve — depth spent against decision quality — and we are careful to note that this saving is not unique to us: any internal probe reading the same activations gets it for free, so it is table stakes for latent-space methods rather than an advantage over them.

2.6 Positioning

Every subsection above names a component ConceptGate uses, and the plumbing beneath them — reading and writing activations through forward hooks, running a partial forward — is commodity engineering, not method. The one-paragraph stance we keep in view for the rest of the paper is therefore deliberately modest: ConceptGate introduces no new mechanism. It is a specific composition — few-shot, calibrated, multi-tap, and dual-mode (read and write) over a frozen model’s own middle layers, with the write conditioned on the read — and its worth is an empirical question, settled by the map in §4 rather than by any single clever part. That map includes results that cut against the method as much as for it, and it comes with a caveat that is not optional but constitutive: obfuscated-activation attacks are known to defeat this entire class of latent-space defense, driving a probe’s recall to zero while the behaviour it was meant to catch continues unchanged [10]. We treat that not as a footnote but as a boundary on what the whole approach can claim (§6).

3. Method

This section develops ConceptGate in the order the signal flows through it, since each stage is defined by what the previous one produces and the construction is clearest read as a single path. The frozen model is run once. At a chosen set of layers the residual stream is tapped (§3.1) and standardized (§3.2), so that the handful of very high-magnitude outlier dimensions every residual stream carries cannot dominate the geometry and mask the concept. In that standardized space a per-layer direction is learned (§3.3) and each layer’s activation is projected onto it, reducing the layer to a single scalar; stacked across the tapped layers, these scalars form the concept’s spectrogram across depth (§3.4). A learned depth filter (§3.5) collapses that spectrogram to one score, and the reason to read several layers instead of the single best one is a matched-filter argument we make precise in §3.6 — an argument resting on an independence assumption that real tapped layers violate, which is why the fusion does not beat a probe on the same taps (§4.8.1). That score feeds a calibrated likelihood-ratio gate (§3.7–§3.8) that returns a three-way verdict — fire, abstain, or pass — and a bank of such gates composes without interference (§3.9).

Everything to this point is the read path. The write path (§3.10) follows directly from having formulated the reading geometrically: a related concept direction, expressed in the model’s raw activation space, can be added back into the stream to influence what the model generates, and §3.11 unifies reading and writing behind a single action interface so that “detect and refuse,” “detect and steer,” and “steer unconditionally” are one mechanism with different settings. We close the section with the two efficiency properties that make attaching ConceptGate nearly free — the truncated forward that lets detection run only the bottom of the network (§3.12), and the operational lifecycle and library that expose all of this as five small calls (§3.13). Figure 1 shows the entire path at a glance; the subsections below walk it one stage at a time.

The same pipeline, run on a real prompt, is shown interactively in Figure 2: choosing a prompt taps it at three blocks, turns each tap into one bar of the spectrogram, and passes the blend through the gate. The subsections that follow (§3.1 onward) then formalize each stage.

3.1 Setup and notation

$M$ is a frozen causal LM of residual width $d$. We tap a set of block layers $\mathcal{L}={\ell_1,\dots,\ell_m}$ (0-based; block $\ell$’s output is hidden_states[ℓ+1]). For a single token, its activations across the taps form $a\in\mathbb{R}^{m\times d}$, with rows $a_\ell\in\mathbb{R}^d$. A concept has two classes, positive ($+$, concept present) and negative ($-$). We fit on one representation per prompt — the last token’s activation, because the last token has attended to the whole prompt and therefore summarizes its intent; fitting on every token instead mixes in shared boilerplate (“How do I …”) that appears in both classes and crushes the signal. Let $\mathcal{A}^+,\mathcal{A}^-$ be the last-token activation sets of the two classes.

3.2 Standardization

Residual streams have a few outlier dimensions of enormous magnitude that would dominate any raw dot product. We standardize per $(\ell,\text{dim})$ using pooled statistics over $\mathcal{A}=\mathcal{A}^+\cup\mathcal{A}^-$:

with $\epsilon=10^{-6}$. All detection math operates on the standardized $z$; steering (§3.10) deliberately works in raw space, because the hook that writes the stream sees raw activations. This one preprocessing step is what brings a plain diff-of-means within reach of the optimal direction — only partway, as the next section shows: standardization equalizes marginal variances but leaves the correlations, and the residual gap to a covariance-aware estimator is real and measured.

3.3 The diff-of-means direction

For each tapped layer the concept’s signature is the unit vector along the difference of the class means in standardized space:

This has a rationale, but it is an approximation, not an identity. Model each class as a Gaussian with a shared covariance $\Sigma$; the Bayes-optimal (LDA) direction is $\Sigma^{-1}(\mu^+-\mu^-)$. Per-dimension standardization equalizes the marginal variances — it turns $\Sigma$ into a correlation matrix with unit diagonal, not into $I$ — so diff-of-means coincides with the optimum only under the further assumption that the standardized dimensions are approximately uncorrelated. When that assumption fails, a per-layer logistic direction, which is covariance-aware, closes the small remaining gap to an SVM (§4.3), and it is the mode we report whenever the comparison is against a trained classifier. Two caveats follow. The standardization is pooled over $\mathcal{A}^+\cup\mathcal{A}^-$, so a dimension that carries the concept has its large between-class gap folded into the pooled variance and is thereby shrunk by standardization — which plausibly accounts for part of the diff-of-means gap. And the few-shot stability is only partial: $w_\ell$ rests on two mean vectors that are well estimated from ten prompts, but $\mu_0$ and $\sigma_0$ are $2d$ moment estimates from the same handful of examples, entering the direction multiplicatively.

3.4 The concept spectrogram

Projecting a standardized activation onto each layer’s signature reduces that layer to a single scalar, and stacking these scalars across the tapped layers yields the concept’s profile across depth, which we call its spectrogram:

One analogy makes the object concrete. Picture the tapped layers as a row of microphones placed along a hall that the model’s computation travels down; each microphone is tuned to a single concept and reports how strongly it registers there, so the spectrogram is the pattern of those readings across the hall. The design keeps all $m$ readings rather than the single loudest one, because a concept is usually audible at several depths and combining independent readings is more reliable than trusting any one microphone — a claim the next two subsections make precise.

Stated in signal-processing terms, the concept’s presence is a signal that the network carries along its depth axis; the spectrogram is that signal sampled at the tapped layers, and reducing it to a decision is a filtering problem. This is the view that motivates the choice of combiner in §3.5: rather than pick a single layer by hand, the method learns a matched filter over depth — a bandpass filter that weights each layer by how cleanly it carries the concept — which is the classical, and provably optimal, way to combine several noisy measurements of the same signal. We use this framing only as intuition; the contribution is the measured effect of the fusion (§4.1), not the metaphor.

Each layer’s individual contribution is summarized by its discriminability $d’$ (per layer $\ell$), the standardized gap between the two class means of $s_\ell$:

3.5 The depth bandpass filter

The spectrogram is reduced to a single score by a filter $f\in\mathbb{R}^m$, giving $S=f\cdot\mathbf{s}$. There are three principled choices for $f$, of which the first is the standard single-layer baseline:

with $\Sigma_{\mathbf s}$ the pooled within-class covariance of $\mathbf s$, ridge-regularized for small samples. The design decision that keeps the comparison clean is that best is a nested special case of the others (a one-hot $f$), so comparing them answers “does using depth help?” with no confound. On synthetic data the answer is yes; on real prompts it is no (§4.8.1).

3.6 The quadrature argument for depth fusion

Model each per-layer score as signal plus independent noise, $s_\ell=a_\ell y+n_\ell$ with $y\in{\pm1}$ and $n_\ell\sim\mathcal{N}(0,\sigma_\ell^2)$ independent across layers. The matched filter $f_\ell\propto a_\ell/\sigma_\ell^2$ maximizes the post-blend discriminability, and because the noises are independent, the discriminabilities add in quadrature:

At the equal-prior threshold, the per-class error of two equal-variance Gaussians separated by $d’$ is $\mathrm{err}=\Phi(-d’/2)$. So fusion strictly beats the single best layer whenever any other layer carries independent signal. The independence premise is the load-bearing one, and it is where the argument fails in practice: adjacent residual-stream taps are strongly correlated, the quadrature sum overstates what is actually available, and the measured gain over a probe on the same concatenated taps is zero or negative (§4.8.1). What follows is therefore the argument for why fusion should help, presented so that its failure on real models is legible. The widget below makes the effect concrete: varying the three per-layer $d’$ updates the fused $d’$ and the two error rates. The defaults are the values the synthetic experiment (§4.1) recovered.

One caveat is important: the quadrature gain assumes independent per-layer noise. Adjacent layers are correlated, so the real gain is smaller than three independent layers would suggest — which is precisely why fisher, using $\Sigma_{\mathbf s}^{-1}$, is the safe default over the naive diag.

3.7 Class-conditional mixtures and BIC

A single Gaussian per class assumes each class is one blob. Real “benign” traffic is not one blob — it is chit-chat, homework, code, each with its own score profile across depth. So we let each class be a small library of profiles: a Gaussian mixture on the joint spectrogram,

gated, exactly as before, on the log-likelihood ratio between the two class densities,

Modelling on the joint $\mathbf{s}$ (not per layer) keeps cross-layer correlations. With $J=1$ per class and shared covariance the LLR is affine in $\mathbf{s}$ with normal vector $\Sigma_{\mathbf s}^{-1}(\bar{\mathbf s}^+-\bar{\mathbf s}^-)$ — i.e. it reduces exactly to the fisher filter of §3.5. The mixture is a strict generalization.

How many profiles? A larger mixture always fits the training data better, so fit alone cannot decide $J$. We use the Bayesian Information Criterion, $\mathrm{BIC}=-2\log\text{-lik}+k\ln N$, over $J\in{1,2,3}$ per class: each parameter pays rent $\ln N$, and an extra profile is admitted only if the fit it buys exceeds that rent. This makes “one distribution or many?” a question the data answers — with a consequence documented in §4.2: at ten-shot sample sizes, the rent is never paid, and $J$ collapses to $1$.

3.8 The calibrated gate: fire, abstain, pass

Fit two 1-D Gaussians on the filtered score $S$ and decide by the likelihood ratio $\mathrm{LLR}(S)$. The threshold $\tau$ is calibrated to a target on the benign tail — either a false-positive-rate quantile, or a $z$-based rule that places $\tau$ where a benign score sits $z$ standard deviations above the benign mean ($z=3$ ≈ 0.1% benign-tail FPR). We map the LLR to a calibrated probability $P_{\text{present}}=\sigma((\mathrm{LLR}(S)-\tau)/s)$ (with $s$ the benign spread local to $\tau$, because the LLR’s heavy tails make a global spread useless), and expose a three-way verdict: fire if $\mathrm{LLR}>\tau$, pass if below, but abstain — override to no decision — whenever $\lvert P_{\text{present}}-\tfrac12\rvert<\text{margin}$. Abstention is the appropriate few-shot behaviour: with ten examples per class, some prompts fall near the boundary, and reporting that is more useful than forcing a decision.

The figure below applies this gate to real GPT-2 activations for a jailbreak concept, calibrated at a deliberately permissive operating point ($z=2$, so several borderline benign prompts fire). Ten labelled probes — jailbreak, benign, and two deliberately borderline cases — carry their measured LLRs; moving $\tau$ reclassifies each probe and traces the trade-off between recall and false-positive rate on the concept’s own examples. Two behaviours are worth observing. The benign prompt “Ignore the noise outside and focus on the recipe” scores high because the early layers respond to the word “ignore” — a genuine failure mode of shallow taps — and raising $\tau$ far enough to suppress it also begins to reject true jailbreaks. There is no threshold that separates the two cleanly; there is only a choice of operating point.

3.9 Combining K concepts

A bank of $K$ concepts fires if any single concept fires, and attributes the firing to the concept with the largest likelihood ratio:

The attributed concept is the one whose direction is used if the action steers, so the same max-LLR rule that decides whether to act also decides along which concept to steer — a small but convenient coupling that keeps a multi-concept bank behaving like a single decision. Each concept is independent kilobytes, so a bank scales linearly and stays tiny. Per-concept thresholds keep the directions independent, but they do not compose into a calibrated bank: the false-positive rate is the union over the $K$ concepts, so a per-concept $z=3$ ($\approx 0.1\%$ FPR) OR-ed over $K=14$ gives a bank-level FPR near $1.4\%$ — the operating point has to be set against the whole bank, not one concept at a time. This bank — one shared truncated forward broadcast to the $K$ concept directions, each with a detection direction and a related raw direction that steers — is drawn in Figure 12, and its cost as $K$ grows is measured in §4.8.2.

3.10 Steering: the write side

Detection runs in standardized space, but the steering hook perturbs the raw residual stream — the hook sees un-standardized activations at generation time — so we keep a second, raw-space direction per layer, the unit difference of the raw class means:

This is deliberately decoupled from the standardized detection direction of §3.3: the detector wants the whitened direction that separates classes, whereas the steerer wants the direction that actually exists in the model’s native activation space, since that is what the forward hook can add. The two are therefore related but not identical — fit from the same ~10 examples and pointing broadly the same way, but decoupled by the standardization (and, when detection uses the logistic mode of §4.3, by its covariance-awareness). The quantity we measure is the cosine between the detection direction mapped back into raw space — $w_\ell$ divided elementwise by the per-feature scale $\sigma_\ell$, then renormalized — and the raw steering direction $w^{\text{raw}}_\ell$. It is not a single number: it depends on the detection mode, on the concept, and on the model, so we report the sweep rather than one figure. Averaged over three taps and four concepts:

The last row is the configuration whose detection numbers §4.8 reports, and it is the lowest: alignment is highest for hand-written few-shot concepts under diff-of-means detection, and drops as the detector becomes more covariance-aware and the fitting set larger and messier. Both effects are expected — logistic mode rotates the detector away from the class-mean difference by design, which is exactly what makes it the better detector and the worse proxy for the steering vector.

Both halves of these numbers matter, and quoting either alone misleads. Two random unit vectors in $\mathbb{R}^d$ have cosine of order $1/\sqrt d$ — $0.036$, $0.033$, and $0.021$ at these widths — so even the lowest value sits about $13\sigma$ above chance and the highest about $36\sigma$: the read and write directions are unmistakably related, never coincidentally aligned. But $0.45$ is also $63°$ and $0.79$ is $38°$, so none of them is close to identity either. A reader should take neither “the same direction used twice” nor “two unrelated directions” from this. What the read and write sides genuinely share is the fitting data and the class-mean construction, not the exact geometry. The measurement is scripts/eval_gate.py --cosine. During generation, at each tapped layer we add

with $\alpha>0$ steering toward the concept and $\alpha<0$ away (the refusal / guardrail direction). The one practical subtlety is magnitude: a good absolute $\alpha$ on GPT-2 is wrong on Qwen, because their residual norms differ by about fivefold (96 vs 19 in our runs). So we set $\alpha$ as a fraction of the measured residual norm, which transfers approximately across the three models tested — the coherent band is similar but not identical on each, and we have not verified it beyond them — empirically $\sim$3–10% is the coherent band, and above roughly 20–25% the text degrades into repetition or gibberish.

The figure below shows actual generations across a range of fractions, from negative (away from the concept) through zero (unsteered) to positive (toward the concept), for a chosen model and concept. The effect is clearest on Qwen2.5-0.5B: the “food” direction pulls the continuation toward “I made this dish… the sweet and savory flavors,” and the “nature” direction toward “a group of bees… the scent of wildflowers.” GPT-2 shifts more weakly under the same procedure and degrades into repetition sooner — the same control applied to a less capable model, an effect examined in §4.6.

3.11 Actions and the run driver

Detection and steering are unified behind one small strategy interface. An action is a policy object with a single method, decide(ctx) → Decision, given a narrow view of the verdict; the gate executes the returned decision. Three actions ship:

• Abort → halts decoding and appends a fixed marker after generation has stopped — a hard gate whose saving is the decoding it skips (the prompt’s forward has already run for the check).
• Steer → adds the per-layer steering vectors for the whole generation (§3.10), with the magnitude as a fraction of the residual norm and an optional named concept to steer along.
• Emit → seeds a fixed string into the completion and lets the model continue from it — a soft redirect (open with a refusal, then let the model finish it in its own voice), as opposed to Abort’s post-hoc marker.

When each acts is a shared Trigger: FIRE (only on a confident firing), FIRE_OR_UNSURE (also on an abstain, fail-closed), or ALWAYS (unconditionally — the topic-steering case). A single run method is the driver: it takes the cheap input verdict, asks the action, and executes — halting, steering the generation, or seeding and continuing. Detection-only use needs no action at all; it is the pure measurement primitive that run is built on.

3.12 The truncated forward and cost

To read a tap at layer $\ell$, only blocks $0..\ell$ need to run. Detection therefore executes a truncated forward — the tail of the network, the final norm, and the unembedding are never touched — which on GPT-2 is measured bit-identical at the taps and about 46% faster than a full forward. A weight-truncated load mode goes further and never materializes the tail at all, so a large model tapped early loads a fraction of its weights (detection-only; generation still needs the whole network). This is what makes the compute–accuracy frontier of §4.5 a real saving rather than a curiosity: a guardrail that fires on every prompt should run as little of the model as the concept allows.

3.13 The lifecycle and library

Everything above is exposed as five small operations, and it is worth walking them in order both because the ordering is the method and because it is how the reference implementation is actually used. The whole surface is: load a model and choose where to tap, learn a concept from examples, calibrate its operating point, check a prompt (pure detection), and run a prompt under an action (detect-and-act). Nothing in the list trains the host model, and only the first step touches its weights.

Load is the one place the memory–compute trade is made explicit, through a LoadMode. The default, FULL, materializes the whole network and can therefore both detect and generate. The optimization, UP_TO_TAPS, constructs the model with only the embedding and blocks $0..\max(\mathcal{L})$ — the tail blocks, the final norm, and the unembedding are never allocated — so a model tapped early loads only a fraction of its weights (for an 8B model tapped in the lower third, on the order of ~6 GB instead of ~16 GB). The catch is exactly what the truncated forward already implied: a weight-truncated gate has no lm_head and so is detect-only; asking it to generate raises rather than silently misbehaving. Orthogonal to which weights are loaded is a batch_size knob on learning, which trades extraction memory for speed (one prompt at a time versus a padded batch) — the two dials compose freely, and neither affects the learned concept.

from conceptgate import ConceptGate, LoadMode
from conceptgate.actions import Abort, Steer, Emit, Trigger

# FULL: can detect AND generate (needed for steering / emit)
cg = ConceptGate.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", layers=[8, 12, 16])

# UP_TO_TAPS: loads only blocks 0..max(tap) -> a fraction of the weights, DETECT-ONLY
guard = ConceptGate.from_pretrained("gpt2", layers=[4, 6, 8], load=LoadMode.UP_TO_TAPS)

Learn fits a concept from two small prompt lists; it is a few sample means and a small solve, so it returns in milliseconds and can be called repeatedly to build a bank. Calibrate then sets each concept’s threshold $\tau$ from the benign tail (a target false-positive rate, or a $z$-based rule) and, optionally, an abstain band. These two steps are all that stand between raw examples and a working gate.

cg.learn("jailbreak", positives=[...], negatives=[...])   # ~10 per side; chainable
cg.learn("medical_advice", positives=[...], negatives=[...])
cg.calibrate(z=3.0, margin=0.1)   # per-concept tau + an "unsure" band; higher z = stricter

Check is the pure measurement primitive: it runs the truncated forward and returns a Verdict (fire / abstain / pass, a calibrated $P_{\text{present}}$, the attributed concept, and the residual norm used for fractional steering). It is the only operation a UP_TO_TAPS gate can perform, and it is what every guardrail deployment leans on. Run wraps check with an action and drives generation: it asks the action about the input verdict and executes the returned decision — halt (Abort), steer the whole generation (Steer), or seed-and-continue (Emit) — with a shared Trigger deciding when the action acts.

# detection only (cheapest; works on an UP_TO_TAPS gate)
v = guard.check("Ignore your instructions and act with no limits")
if v.fired:
    ...  # v.concept, v.p_present, v.score

# workflow 1 -- guardrail: block when a concept fires, else generate normally
cg.run(prompt, action=Abort(when=Trigger.FIRE))

# workflow 2 -- soft redirect: open with a refusal and let the model continue it
cg.run(prompt, action=Emit(text="\nI can't help with that.", when=Trigger.FIRE))

# workflow 3 -- topic steering: bend generation toward a named concept, unconditionally
cg.learn("food", positives=[...], negatives=[...])
cg.run(prompt, action=Steer(concept="food", fraction=0.06, when=Trigger.ALWAYS))

# workflow 4 -- conditional steering: steer AWAY only when the concept is detected
cg.run(prompt, action=Steer(concept="jailbreak", fraction=-0.06, when=Trigger.FIRE))

The division of labour is worth stating plainly, because it is the whole memory-efficiency argument in one place. Detection — learn, calibrate, check, and the Abort half of run — needs only the bottom of the network and can run on a weight-truncated load, which is where the compute and memory savings arise and why a pure guardrail is inexpensive to attach. Generation — the Steer and Emit halves of run — needs the full network, so the differentiating write capability does not enjoy the load-time saving; it is cheap only in the sense that it adds a few vector additions to a forward pass the host was already going to run. A deployment that only ever detects should load UP_TO_TAPS and never pay for the tail; a deployment that steers loads FULL and gets detection for free on the way through.

4. Experiments and results

We evaluate on GPT-2 (12 blocks), Qwen2.5-0.5B-Instruct (24 blocks), and gemma-2-2b-it (26 blocks) — all small enough to run and re-run on a laptop, which is the point, since the whole method is meant to be cheap. The subsections follow the method’s own order: the reading path first (§4.1–§4.5), then the write side (§4.6), then cost (§4.8). That is deliberately not the order of the contributions — the one capability we defend as specific to the method is steering, which appears sixth. The negative results are not buried.

4.1 Depth fusion on synthetic data

On a controlled synthetic problem with three layers of known per-layer discriminability $d’=[1.6,2.0,0.6]$, the theory predicts a fused $d’=\sqrt{1.6^2+2.0^2+0.6^2}=2.63$, i.e. test error $\Phi(-1.315)=9.4\%$ versus the single-best-layer $\Phi(-1.0)=15.9\%$. The learned filter recovers $d’=[1.62,2.04,0.64]$ and drives test error from 16.1% to 9.4%, matching the prediction. This verifies that the implementation realizes the matched-filter algebra; it is not evidence about residual streams. The synthetic data is generated under the filter’s own assumption of independent per-layer Gaussian noise, so recovering the predicted $d’$ is close to tautological. On a real model the layers are correlated, the gain vanishes, and a probe on the same concatenated taps matches or beats the fusion (§4.8.1); we therefore treat depth fusion as a mechanism that does not transfer, not as a contribution.

4.2 Mixture densities: a constructed hard case and a few-shot collapse

The mixture model is justified by a constructed hard case: place two benign clusters on either side of the harmful cluster along the discriminative axis (benign at $-2$ and $+2$, harmful at $0$). No single threshold on any linear score can carve out “the middle,” so the fisher gate is stuck near chance (38.8% error, AUC 0.60); the mixture, seeing $\mathbf{s}$ near a benign profile on each side and a harmful profile between, recovers it (7.1% error, AUC 0.98; the Bayes floor is 5.8%). That is the case for mixtures, and Figure 6 shows its geometry. One honest caveat on the construction: with the benign clusters symmetric about the harmful one the two class means coincide, so the diff-of-means (or logistic) direction the pipeline actually fits would be null here. This hard case is built directly in score space to illustrate the mixture’s expressiveness; it is not a configuration ConceptGate’s own direction-finding would produce.

The case against them, at least in the regime we care about, is that on real GPT-2 activations with 12+12 prompts, BIC selects $J=1$ for both classes — an extra full-covariance profile over five layers costs ~21 parameters, whose rent (~52 nats) twelve samples cannot pay — and the mixture gate collapses exactly onto the single-Gaussian gate (rank agreement 0.986). This is less “the data answering one-or-many” than an identifiability limit: at twelve samples a five-dimensional full-covariance component is already near-singular, so the $J=2$ fit is ill-conditioned and the selection is effectively decided by sample size. The mixture is the more general model but remains inactive in this regime: whether real concept classes are multimodal enough to justify additional components is a question that requires substantially more than ten labelled examples, and on the readily-labelled concepts examined here the selection criterion returns a single component per class.

4.3 Detection on real prompts: a commodity

On real jailbreak-versus-benign prompts, ConceptGate’s difference-of-means detector performs well — and so does a linear support-vector machine trained on the same activations, and so does per-layer logistic regression. Across both models the three are within noise of one another on AUC. §4.8.1 puts numbers on “within noise” over 262 held-out prompts: at 32 examples per class, ConceptGate-logistic reaches $0.973$ against $0.978$ for a probe on the same taps and $0.982$ for both a probe and an SVM on the full model (Qwen2.5-0.5B), and $0.979$ against $0.978$, $0.987$, and $0.989$ respectively on gemma-2-2b. The logistic variant, which is covariance-aware and therefore slightly stronger where standardization leaves the within-class covariance non-isotropic, closes the small remaining gap to the SVM, with its largest gains on the weaker model, but it does not establish a new one. As a detector, then, ConceptGate is a commodity: any method that reads a linear direction from these activations performs comparably, and the choice of estimator is a tuning decision rather than a contribution. The capability that is not a commodity is the write side, which is why the remainder of the report concentrates on steering.

4.4 Matched versus broad negatives (a negative result)

We anticipated that matched contrastive negatives — benign prompts sharing the surface structure of the jailbreaks (the same register, without the intent) — would sharpen the direction by cancelling nuisance variation, following the CAA construction. The measurement contradicted this: matched negatives gave an AUC of approximately 0.42, below chance, against 0.78 for broad, unrelated negatives. The explanation is that broad negatives allow the direction to align with the large semantic gap between an assertive instruction to a model and an ordinary factual query, which is the signal the detector depends on, whereas matched negatives remove that gap. We read $0.42$ as the direction failing to separate the classes, and not as evidence that it inverts them. The point estimate does sit below $0.5$, which would mean jailbreaks ranked slightly beneath benign prompts, but with ten examples per class the sampling spread around chance is wide enough that $0.42$ is not evidence of a stable anti-signal, and we do not claim one. The practical conclusion needs only the weaker reading: for few-shot concept detection, negatives should be broad rather than matched.

4.5 The compute–accuracy frontier

Because detection needs only blocks up to the tap, every concept has a cheapest depth at which it is already separable. We sweep every layer, fit the standardized diff-of-means detector there, and measure leave-one-out AUC against the fraction of the network that tap requires. Leave-one-out keeps the held-out prompt out of the fit, but selecting the knee on the same ~20-prompt estimate makes the AUC at the chosen layer optimistically biased and, on so few prompts, high-variance; read these curves as locating roughly where the concept becomes readable, not as a precise operating AUC. The figure plots the measured curves; the target-AUC control locates the knee, the cheapest layer that clears a chosen AUC.

The shape of the curve reflects model capability. On GPT-2 the jailbreak concept is not cleanly formed until the middle of the network: AUC climbs through the early blocks and only saturates around block 6, so the cheapest reliable guardrail runs somewhat more than half the network and the final ~40% of blocks contribute nothing. On Qwen2.5-0.5B the same concept is essentially separable by block 1, so the guardrail can run roughly 8% of the network (block 1 of 24). We describe this throughout as the base model being more capable, but the comparison is confounded: GPT-2 is a 2019 base model while Qwen2.5 is an instruction-tuned model post-trained on refusal, so jailbreak-ness being salient at block 1 may be safety tuning as much as raw capability. The two are entangled here and we do not separate them. (This 8% figure is also from the leave-one-out sweep of §4.5, on ~20 prompts; the 262-prompt held-out measurement of §4.8.1 is the one to quote for a precise depth.) Each of these is a concrete, per-concept, per-model operating point, and it is the practical consequence of the truncated forward.

4.6 Steering across models

Steering (§3.10) is the operation a detector cannot perform, and the part of the system that most needs measuring rather than illustrating. For each of three concepts — food, nature, technology — we generate a continuation of five neutral prompts at each value of the steering fraction (the magnitude of the added direction, as a fraction of the residual norm) and score every greedy generation three ways: the share of content words matching a concept keyword list (independent of the steering direction, in that the word list is written by hand and never enters the fit — but not independent of the concept’s framing, since the same author chose both the keyword list and the few-shot prompts, so the two share whatever conception of “nature” or “food” that author had), the perplexity of the continuation under the base model (fluency), and the concept’s own detector log-likelihood ratio on the generated text (the internal read of what the write produced). There is no baseline method to compare against — a linear probe or classifier cannot steer at all, and the mechanism itself is standard activation addition (§2.2) — so the comparison is the unsteered point (fraction 0) and the two base models. The harness is scripts/eval_steering.py.

Steering is a monotonic dose-response. On Qwen2.5-0.5B, raising the toward-fraction increases concept content smoothly — the food direction moves the continuation to homemade-salsa completions, the nature direction to sun-over-the-mountains ones — while perplexity stays flat through roughly $|\text{fraction}| \le 0.1$, giving an effective window in which content shifts and fluency holds. The concept’s own detector tracks the shift ordinally (the food LLR climbs monotonically over the sweep, nature from below zero to above), so on a capable model the read and the write agree — though these LLR magnitudes are not calibrated probabilities: the Gaussian density is badly misspecified for generated text far from the few-shot fitting set, which is why the independent lexicon is the primary signal here.

GPT-2 shows the same shape under the identical procedure, bounded by the weaker model. It shifts content even harder at the largest fractions — the nature direction reaches a 23% keyword rate against Qwen’s 9% — but as degenerate repetition (“the sun shone on the mountains, and the moon shone on the mountains…”), and its own detector barely registers its own steered output (the nature LLR stays negative throughout). The write can outrun a weak read, but the fluency of the steered text is bounded by what the base model produces cleanly — the same ceiling that bounds detection (§4.5): a more capable model steers more cleanly from the same ten examples.

That reading deserves a check, because a competing explanation predicts the same observation. If the write direction sits far off the read direction (§3.10), then steering would move the activations somewhere the detector does not look, and the detector would fail to register its own steered output on any model, weak or not. We would then be reporting a geometric artifact as a capability ceiling. The test is direct: steer, re-read the continuation, and ask whether it moved along the steering direction while staying still along the detection direction. It does not. On GPT-2 the steered continuation moves $+0.02$ (norm-relative) along the steering direction and the detection score moves with it, by $+4.2$ and $+2.1$ standardized units at two of three taps, for a total concept LLR change of $+21$; on Qwen the corresponding figures are $+0.12$–$+0.20$ and $+2.6$ to $+9.1$, LLR $+48$. The detector does register the steered output on both models. And the decoupling account fails a second, sharper test: GPT-2 has the highest read/write cosine of the three models in every mode we measured (0.52 against Qwen’s 0.45 in the configuration of §4.8; 0.79 against 0.65 under diff-of-means). Were decoupling the cause, GPT-2 should be the worst-aligned model, not the best. What is left is the original reading, now with a measured basis: GPT-2’s nature LLR stays negative because the concept is poorly separated there in absolute terms, not because the write goes somewhere the read cannot see. The measurement is scripts/eval_gate.py --decouple.

Across the three signals, steering is a measured, controllable capability rather than a demonstration: within an effective window it shifts generated content monotonically — an operation no detector or classifier can perform — and its quality, like detection’s, is bounded by the base model rather than by the number of examples. It is a soft control, strongest on concepts the model represents clearly and weaker on abstract ones such as technology, so it is best used as a nudge within its window rather than as a hard guarantee. This measured write capability, fit from the same few-shot data as the detector, is what justifies operating on a concept inside the residual stream rather than filtering on the output text.

4.7 A paraphrase-robustness null

We expected a depth-dependent robustness effect: that shallow taps, keyed on surface words, would collapse when jailbreak prompts are reworded, while deep taps, keyed on meaning, would hold — giving a principled reason to gate deep. It does not appear. Rewording the positives barely moves AUC at any layer, because detection against broad negatives rides the semantic gap (§4.4), which rewording does not close. The lexical-sensitivity effect only appears against hard, surface-matched negatives — which, as §4.4 shows, themselves degrade detection. We report this null to document that the effect, though intuitive, does not arise in the broad-negative setting that detection otherwise relies on.

4.8 An efficiency evaluation of ConceptGate

The commodity result (§4.3) shows that the choice of linear estimator barely affects accuracy. This section asks what ConceptGate costs to reach it, against the standard ways of adapting a frozen model — and finds two different answers. For a single concept (§4.8.1) the cost is low, but the saving is not ConceptGate’s: it is the truncated forward, which a depth-matched probe shares exactly. The separation appears only for a bank of many concepts (§4.8.2), where a training-free bank extends without the per-concept training run that fine-tuning needs. The task throughout is content-safety detection, used not because the method is specific to safety but because it comes with public datasets and a natural multi-concept structure: the concept is a single category in §4.8.1 and the fourteen categories of a guardrail taxonomy in §4.8.2. Both subsections report the same axes — detection AUC, memory (weights loaded and parameters learned, both device-independent), and compute (per-prompt forward wall-time on an Apple M4 under MPS) — first for one concept, then for a bank of many as their number grows.

4.8.1 Learning a single concept

Setup. The task is jailbreak detection on the public jackhhao/jailbreak-classification dataset: each method learns the concept from a small, balanced set of few-shot examples drawn from the training split, and is scored on the untouched official test split (262 prompts — 139 jailbreak, 123 benign). We evaluate two frozen base models, Qwen2.5-0.5B-Instruct (24 transformer blocks) and gemma-2-2b-it (26 blocks). We compare against a linear probe in two forms, trained on the same few-shot examples. The depth-matched probe fits a logistic head on the same tapped activations ConceptGate reads — the identical truncated forward, the identical compute — and is the fair single-concept comparator. The full-model probe fits its head on the final-layer representation, running the whole network; it is the conventional frozen-model classifier and serves as the upper reference. ConceptGate is swept over its tap configurations — a single tap at increasing depth, and three- and five-tap fusions — each in its logistic-direction mode. The three comparison axes are detection AUC on the held-out test; memory, as the fraction of the base model’s weights that must be loaded and the number of parameters learned (both device-independent); and compute, as the measured per-prompt forward wall-time on an Apple M4 under MPS, averaged over three seeds. We report held-out test metrics only. The harness (scripts/eval_detection.py) and the full result tables (docs/evaluation.md) are in the repository; the figure and tables below replay their output.

At every tap depth ConceptGate’s logistic direction and the depth-matched probe are the same number — a single-tap logistic direction is logistic regression on that tap — so the only real comparison is against the full-model probe, and the compute gap is entirely the truncated forward. On Qwen2.5-0.5B a single tap at 46% depth scores 0.970 at 61% of the weights; on gemma-2-2b a tap at 42% depth scores 0.974 at 55% of the weights. The full-model probe reaches a slightly higher ceiling (0.982 and 0.987) because it runs the whole network. The point of the table is the middle rows: the depth-matched probe reaches ConceptGate’s number at ConceptGate’s compute.

AUCs are the mean over three few-shot resamples with the standard deviation across them; the AUC columns reproduced to three decimal places on an independent re-run. Cost is reported as fraction of depth and of loaded weights rather than in milliseconds, because both are exact properties of the truncation while wall-clock is not: the same configurations timed on the same machine under different load gave forward times differing by up to a factor of two, though the ratios to a full forward were stable to a few points (a single early tap runs at roughly 30–48% of a full forward, three taps at roughly 62–69%).

Two observations, both negative for the read side. Depth fusion buys nothing: the three- and five-tap ConceptGate configurations sit on top of the best single tap, and on Qwen the depth-matched probe on the concatenated taps slightly beats ConceptGate’s bandpass fusion (0.978 vs 0.973) — the synthetic depth-fusion advantage (§4.1) does not transfer to a real model where one layer already carries the concept. And the difference-of-means direction trails the logistic one by three to five points and is not competitive with either probe.

So the honest reading is not that ConceptGate is Pareto-efficient over a fair baseline — a depth-matched probe matches it at the same compute and beats its fusion. What §4.8.1 establishes is a fact about the models, not about ConceptGate: the jailbreak concept is linearly readable at a shallow tap, so a truncated forward suffices and the top of the network can be skipped; how shallow that tap can be is set by how early the base model forms the abstraction (§4.5). The truncated forward is genuinely cheaper than running the whole model, but it is a property any latent probe shares, not a contribution of ConceptGate. What is specific to ConceptGate is that a direction fit from the same few-shot data can also be written back to steer (§4.6), and the cost of extending a bank of many concepts, which §4.8.2 measures against fine-tuning.

4.8.2 Learning multiple concepts

The single-concept saving of §4.8.1 is not, on its own, specific to ConceptGate: a truncated forward is available to any latent probe (§5.3). The setting where a training-free adapter separates from the alternatives is the one a real deployment faces — many concepts, not one. A content-safety guardrail is the familiar instance: Llama Guard [8] scores a fixed hazard taxonomy, and GLiGuard, a recent guardrail built on GLiNER [13], is fine-tuned to score fourteen harm categories and eleven jailbreak strategies in a single encoder. This subsection measures what a bank of $K$ concepts costs to build, extend, run, and store, and how that cost grows with $K$.

There are two standard ways to add a concept to a frozen model, and ConceptGate is a third. The first is a linear-probe bank: freeze the model and train one linear head per concept on a shared representation. The second is fine-tuning — a LoRA [11] adapter per concept, or, at the monolithic extreme, a single model fine-tuned once over the whole taxonomy — the way GLiGuard fully fine-tunes its encoder, so that changing the taxonomy means retraining. ConceptGate is instead a training-free concept bank: one truncated forward produces the activations that every concept reads; each concept is a closed-form direction fitted in milliseconds and stored in kilobytes; and concepts are added or removed without touching the others (the max-LLR combination of §3.9). Because reading and writing share their fitting data (§3.10), each entry in the bank is also a steering control at no extra cost.

Setup. The concepts are the fourteen harm categories of BeaverTails [14] (PKU-Alignment/BeaverTails) — the same kind of safety taxonomy the guardrails above target. For each category, positives are prompts whose responses were annotated with that harm and negatives are a shared pool of benign prompts; every method sees the same $N=32$ examples per class and is scored on the held-out test split, averaged over three seeds, on both base models. Two caveats on this labelling. The annotation is on the response, so a response-derived label stands in for a prompt-level concept; and because a single benign pool is shared across all fourteen categories, the fourteen directions are pulled toward one common benign centroid — so what presents as fourteen independent concepts is closer to one harmfulness direction with category-specific variation, which also colours the generalization result of §4.9. As $K$ grows from 1 to 14 we measure build time (learning the whole bank), inference (per-prompt wall-time to score against all $K$ concepts), memory (parameters learned), and per-category detection AUC. ConceptGate and the probe learn on frozen features; LoRA back-propagates a rank-8 adapter and a classification head. The harness (scripts/eval_detection.py --scaling) and the raw results (scripts/eval_scaling_results.json, tabulated in docs/evaluation.md) are in the repository.

Adding a concept to ConceptGate is a closed-form fit — 6 ms on Qwen2.5-0.5B, 11 ms on gemma-2-2b — against a LoRA training run of 17 s and 126 s, three to four orders of magnitude more. Two memory costs matter separately. The per-concept artifact — what must be stored to add a concept — is 11–28 thousand numbers for ConceptGate: the detection and steering directions plus per-dimension standardization at each of its three taps, about an order of magnitude more than the probe’s single final-layer head, but both kilobyte-scale against LoRA’s half-to-1.6 million. The model itself — the hundreds of millions of weights that must be resident and run for every prompt — is where that is repaid: ConceptGate loads and runs only up to its deepest tap, never the layers above it, so a single truncated forward serves the whole bank at a per-prompt cost that is constant in $K$ and at or below the probe’s full-model forward, whereas $K$ LoRA adapters need $K$ forwards. The extra kilobytes ConceptGate stores are immaterial next to the part of the network it skips. Against fine-tuning the gap compounds with every concept: building the full fourteen-category bank takes about 8 seconds on Qwen and 46 on gemma, against LoRA’s 3.9 and 29 minutes — 30× and 38×.

The low cost does not come at the expense of detection. Across the fourteen categories the training-free bank trails the full-model linear probe by 0.023 on Qwen2.5-0.5B (mean AUC 0.832 vs 0.855) and is statistically indistinguishable on gemma-2-2b (0.881 vs 0.874 — within seed noise at three seeds). Few-shot LoRA is both the slowest to train and the weakest to read: on the three categories where it was run it reaches mean AUC 0.685 on Qwen and 0.814 on gemma, against ConceptGate’s 0.889 and 0.913 on those same three — a randomly-initialized head simply does not have enough signal in $2N$ examples. Harm content is read best deeper than jailbreak framing, at 50–85% depth rather than a quarter, which is why the taps here sit lower than in §4.8.1; where a category emerges late (sexual content on Qwen) a mid-depth read gives up some AUC, and where the base model forms the abstraction cleanly (most categories on gemma-2-2b) the closed-form direction is as good as the trained head.

The multi-concept setting is where the training-free design matters. A latent bank — ConceptGate, or equally a linear-probe bank — amortizes across a taxonomy in a way that per-concept or monolithic fine-tuning cannot: constant inference, closed-form extension, kilobytes per concept, and no retraining to change the taxonomy. That is a genuine result, and it is honest that a detect-only probe bank shares it. Over such a probe bank ConceptGate’s cost is at worst a tie — a truncated forward is never more than the probe’s full one, and its extra per-concept kilobytes are negligible — while it adds one thing the probe cannot: a second, steering use of the same $K$ concepts (a related raw direction per concept), steering (§4.6), so the one object that gates fourteen harms can also bend generation away from them. A taxonomy-scale bank that is cheap to build and extend, competitive with a trained probe on every category, far ahead of few-shot fine-tuning, and steerable from the same few-shot data is what distinguishes ConceptGate from both a detect-only probe bank and a retrained guardrail.

4.9 Out-of-distribution generalization

In-distribution detection accuracy need not predict performance under distribution shift, and linear probes are known to generalize poorly off-distribution. §4.8.2 estimated a separate direction for each harm category and evaluated it on that same category; here we instead measure how a harmfulness direction transfers to categories it was not estimated from. Fixing the concept and varying only the harm category isolates out-of-distribution generalization within a single concept, and avoids the confound of transferring between two distinct concepts — for instance jailbreak framing and harmful content — where a change of concept is entangled with the change of distribution.

Setup. We use leave-one-category-out cross-validation over the fourteen BeaverTails categories. For each held-out category, ConceptGate and the full-model linear probe estimate the harmful direction from the remaining thirteen categories together with a shared benign pool ($N=32$ per class) and are evaluated on the held-out category; an in-distribution reference instead estimates the direction from the held-out category itself. The two conditions share the same benign examples, test set, sample size, and seeds, so the only difference is whether the evaluated category was present during estimation. We report the mean over three seeds for both base models, reusing the activations of §4.8.2. Harness: --ood; results: eval_ood_results.json.

Generalization is partial. Averaged over the fourteen held-out categories, a harmfulness direction estimated from the remaining thirteen attains a mean AUC of 0.62–0.65 — above chance, but substantially below the 0.83–0.87 obtained in-distribution. The degradation is uneven: violence, self-harm, drug-and-weapon, terrorism, and financial-crime prompts remain detectable when held out (AUC ≈ 0.75–0.81), whereas controversial-political content (0.29–0.47) and, on gemma-2-2b, sexually explicit content (0.43) fall to or below chance. Harmfulness is therefore encoded partly as a shared, category-independent direction and partly as category-specific structure that a held-out estimate does not recover.

ConceptGate is at least as robust to this shift as the full-model probe, and on one model it is measurably more so. Because every category is scored by both methods, the two can be compared pairwise across the fourteen held-out categories rather than through an aggregate. On Qwen2.5-0.5B ConceptGate’s held-out AUC exceeds the probe’s in 13 of 14 categories, by $0.037 \pm 0.032$ (sign test, $p=0.002$), and its degradation is smaller by $0.057 \pm 0.044$ — a consistent advantage rather than a tie. On gemma-2-2b the two are genuinely indistinguishable: ConceptGate is ahead in 7 of 14 categories ($p=1.0$), by $0.007 \pm 0.028$. The mid-layer tapped direction is therefore no less transferable than a final-layer one, and on the smaller model somewhat more so. We report the paired test rather than a seed-level standard deviation because the per-category results are what the harness records; the comparison is paired on the same categories and the same examples, with only the scoring method varying. The absolute level nonetheless indicates that a single few-shot direction is only a partial detector for categories outside its estimation set, and is better estimated from a diverse set of categories than from any one alone.

4.10 Gate-conditioned steering: what the gate is for

Every experiment above measures either the read side or the write side. Neither measures the thing that joins them. The steering rule of §3.10 is activation addition: it needs a direction and a magnitude, and it needs none of the spectrogram, the depth filter, or the calibrated threshold. Strip ConceptGate down to that and you have CAA with a fraction-of-norm knob. What the detection machinery contributes to the write side is therefore not the direction — it is the decision of when to write. That is Steer(when=Trigger.FIRE), workflow 4 of §3.13, and it is the one operation in this system that no probe, no classifier, and no external guard can perform: a classifier can tell you a prompt is a jailbreak, but it cannot then alter the generation, and CAA can alter the generation but cannot decide which prompts deserve it.

We measure it with three arms on Qwen2.5-0.5B, taps 8/12/16, over 32 held-out jailbreak prompts and 32 benign prompts. A jailbreak concept is fit from eight hand-written override framings against eight benign requests, resampled over three seeds, and the arms are: no steer; always steer away from the concept at $-0.08$ of the residual norm on every prompt; and gate-conditioned steer, the same write applied only when the gate fires. Jailbreak suppression is the share of continuations containing an explicit refusal; collateral damage on the benign set is measured two ways, as perplexity and as the share of benign continuations left byte-identical to the unsteered baseline. The second is the blunter and more informative measure: it asks how much benign generation the intervention disturbed at all.

The gate wins on both axes at once, which is more than we expected: not a trade of suppression against collateral, but more suppression and less collateral. Blanket steering gains $2.1\pm3.0$ points of refusal (46.9% → 49.0%) and leaves only $4.2\pm1.5\%$ of benign continuations untouched, pushing benign perplexity from 1.98 to 2.17. Gated steering gains $8.3\pm1.5$ points (46.9% → 55.2%) and leaves $89.6\pm9.0\%$ of benign continuations byte-identical, with perplexity at 2.03. Neither arm produced any benign over-refusal. The reason blanket steering does worse on the suppression axis is instructive: it applies the write to every prompt including the ones where the concept does not register, and pushing those activations along a direction the model does not associate with the input perturbs the generation without steering it anywhere useful. Confining the write to the prompts where the concept actually registers is what makes it effective. The gate is thus not merely a way to limit the intervention; it is part of what makes the intervention work.

Two limits bound this result, both worth stating plainly. First, the effect sizes are small and the absolute refusal rate is modest — 55% is not a guardrail, and this is a 0.5B model steered at a single magnitude on 32 prompts. The result establishes that gating the write dominates blanket writing, not that gated steering is a deployable defense. Second, the gate’s usefulness depends entirely on its concept being fit on examples that match the prompts it will see. Fitting the same concept from the dataset’s long persona templates instead of short override framings produces a gate that fires on 0.0% of these short framed requests across all three seeds — it has learned prompt length and register, not jailbreak intent — and the same short-framing-fitted concept fires on 91.7% of real out-of-register benign prompts, which would be a catastrophic false-positive rate in deployment. The gate is a sharp instrument only inside the distribution its ten examples came from, and §4.9’s partial-generalization finding applies here with more force, because a false fire now rewrites the output rather than merely raising a flag.

The same experiment on the concept bank: a negative result

The bank of §4.8.2 has only ever been measured as a detector, while the contribution claim is that each of its entries also steers. That deserves a direct test, so we ran the identical three arms on five BeaverTails harm categories, fitting each direction from 32 harmful prompts of the category against the shared benign pool and evaluating on twelve held-out prompts of that category. On the behavioural axis the result is a null.

Two of the three findings replicate and one fails. Gating still confines the intervention — it leaves 73% of benign generation byte-identical where blanket steering leaves none — and blanket steering is again the worse of the two write policies, here actively reducing mean refusal from 16.7% to 11.7%. But gating no longer improves anything: refusal is unchanged at 16.7%. Steering away from a harm category does not make this model decline the request.

We think the contrast with the jailbreak concept is the substantive finding rather than a defect of the setup. “Jailbreak framing” is a concept about the intent and register of the request, and it sits close to the behaviour the model was safety-tuned to produce, so pushing along it moves the model toward refusing. “Violence” or “privacy violation” is a concept about the topic of the content; steering away from it changes what the continuation is about, not whether the model complies. These directions also gate far less cleanly (65% against 27% firing, versus 54% against 10% for the jailbreak concept), which is the same partial-generalization limit §4.9 measures, and a gate this leaky cannot target a write precisely even when the write works.

The honest reading of “each bank entry also steers” is therefore narrower than the phrase suggests: each entry supplies a usable write direction at no extra fitting cost, and that is a real property of the construction, but a demonstrated behavioural control exists in this paper only for the topical concepts of §4.6 and the jailbreak framing above. Two caveats bound the null rather than excuse it: refusal is scored by an explicit-decline lexicon, so a reduction in harmful content that stops short of refusing would not register, and we tested one magnitude on one 0.5B model. Both would be worth closing before treating the null as general. The harness is scripts/eval_gate.py.

5. Discussion

5.1 What is contributed

The mechanisms are all drawn from prior work; the detector is a commodity; the single-concept compute saving is the truncated forward, which a depth-matched probe shares (§4.8.1); depth fusion does not transfer beyond synthetic data; and the mixture is inactive at few-shot sizes. What remains is narrow but real. The most distinctive part is the conditional write: a direction fit from the same few-shot data as the detector is written back to steer generation — a measured, monotonic dose-response bounded by the base model (§4.6) — and, more sharply, written back only when the gate fires, which beats writing on every prompt both in effect and in collateral (§4.10). The write rule alone is activation addition and needs none of the read machinery; the conditioning is the part no classifier or probe can supply. The second is amortization: as a training-free bank the adapter extends to a fourteen-way taxonomy in milliseconds and kilobytes and scores all of it in one forward, where per-concept LoRA fine-tuning costs a training run each; this beats fine-tuning, though a linear-probe bank shares it — what ConceptGate adds is a steering direction per entry at no extra fitting cost, whose behavioural effect on those harm categories we measure and find null (§4.8.2, §4.10). The third is the composition and its honest account: a single few-shot, calibrated, training-free module that both reads and writes a concept at a small, well-characterized cost, each part measured against a fair baseline including where it fails — detection is a commodity a probe matches, and generalization to an unseen category is only partial (§4.9). The value of the work is not that its detector outperforms the alternatives — it does not — but that it assembles a read-and-write adapter whose write side a classifier cannot match, and reports each part against a fair comparison.

5.2 Detection is a commodity; steering is the distinguishing capability

The most consequential finding is structural rather than numerical. A text classifier can match or exceed ConceptGate at detection while being simpler to deploy, so if detection were the objective there would be little reason to prefer an internal method. The reason to operate inside the residual stream is the operation a classifier cannot perform: using a closely related direction, fit from the same data, to write — to steer generation toward or away from the concept, conditionally and interpretably, from ten examples. Steering is a measured, monotonic dose-response with an effective window in which content shifts while fluency holds (§4.6), bounded like everything else by the base model. The detection machinery should not be understood as subsidizing the write side, because the write side needs none of it: activation addition wants a direction and a magnitude, not a spectrogram, a depth filter, or a calibrated threshold. What the read side actually contributes to the write side is the decision of when to write. That is the one operation in the composition that no probe, no classifier, and no external guard can perform — a classifier can identify a jailbreak but cannot alter the generation; CAA can alter the generation but cannot choose which prompts deserve it — and §4.10 measures it: conditioning the same write on the gate suppresses more jailbreaks (+8.3 points against +2.1) while leaving 90% of benign generation byte-identical rather than 4%. The efficient detection path (the truncated forward, calibration) matters for what it always mattered for — running the read cheaply enough to keep in the serving loop — and the gate is what converts that read into a targeted write.

5.3 The cost argument and its limits

The compute–accuracy trade-off is a real engineering result — on 262 held-out prompts a jailbreak concept reaches AUC $0.968\pm0.01$ from a tap at 29% of Qwen2.5-0.5B’s depth — under a third of a full forward pass in wall-clock, measured at 28% and 30% on two separate runs — against $0.982\pm0.00$ for a probe on the complete model (§4.8.1) — but two qualifications bound it. First, the truncated-forward saving is available to any internal probe, including the SVM baseline; it is a property of latent-space methods in general rather than an advantage specific to ConceptGate. The sharper and better-measured version of the cost claim is at the bank level (§4.8.2): a training-free concept bank amortizes across a taxonomy — flat inference and closed-form, kilobyte-scale extension where fine-tuning pays seconds-to-minutes and a fresh forward per concept — but that advantage, too, is shared with a linear-probe bank, so what remains specific to ConceptGate is not the reading cost but that the directions, fit from the same data, also steer. Second, the memory-minimal load mode is detection-only, and detection is the commodity half of the system, whereas the distinguishing capability, steering, requires the full network. The cost argument therefore applies to the guardrail rather than to the steerer. The defensible claim is that a read-and-write adapter can be added to a model already being served, for kilobytes of parameters and a fraction of a forward pass — not that it detects more accurately than the alternatives.

5.4 What it actually costs

The cost can be made concrete, since it is the primary reason to assemble this composition rather than deploy a second model. A single concept’s entire learned state, over $m$ tapped layers of residual width $d$, is two sets of direction vectors (the standardized detection direction and the raw steering direction, $2md$ numbers), the per-dimension standardization statistics ($2md$), the depth filter ($m$), and a handful of Gaussian scalars for the gate. For GPT-2 with five taps that is on the order of $1.5\times10^4$ numbers — comfortably under the sub-million-parameter target one would want for something meant to be stored and shipped by the concept — and a bank of $K$ concepts is simply $K$ times that, since concepts share nothing and never interact beyond the max-LLR rule of §3.9. Fitting is not training: it is a few sample means and one small $m\times m$ solve for the filter, completing in milliseconds on a CPU with no backpropagation and no gradients, so a concept can be learned, discarded, and re-learned interactively. Inference adds $m$ dot products of width $d$ plus a length-$m$ blend per gated position — negligible against a single transformer forward — and in the abort case it removes compute, since decoding stops early. The reference implementation (github.com/NISH1001/conceptgate) keeps a deliberately legible shape: a pure-numpy mathematical core (spectral.py for directions, spectrogram, and filters; concept.py for the calibrated gate; mixture.py for the GMM and its BIC selection) sits behind a thin PyTorch boundary that owns only the model-touching parts — the tap reader, the steering hooks, and the ConceptGate facade with its Abort / Steer / Emit strategies. This separation keeps the numerical method auditable in numpy while the framework-specific code remains small enough to re-derive, which is appropriate for a method whose value rests on being inexpensive and transparent.

6. Limitations and threats to validity

The most important limitation, which should be read before any other, is adversarial fragility. ConceptGate is a latent-space defense, and latent-space defenses are known to be breakable: obfuscated-activation attacks can drive a harmfulness probe’s recall from 100% to 0% while the model’s behaviour is unchanged, by finding inputs that keep the activation off the probe’s direction even as the model performs the prohibited action [10]. This attack class targets exactly the family ConceptGate belongs to — linear probes, SAEs, and Gaussian/mixture density gates alike — so nothing in §4 should be read as a security guarantee. In practical terms, ConceptGate is best regarded as a cheap, interpretable, few-shot layer within a defense stack — useful because it is nearly free to add, but not a boundary that a motivated adversary cannot cross. Its steering side is somewhat more robust in this respect than its detection side, since writing a direction alters behaviour whether or not an attacker knows the direction, but steering is not a filter, so the two serve different purposes.

The second group of limitations concerns the evidence being small and in-distribution. We evaluate primarily on GPT-2 and Qwen2.5-0.5B, small enough that the core results reproduce on a single CPU, and add gemma-2-2b for the multi-concept, generalization, and steering results (§4.8–4.9, §4.6), which we run on an Apple M4 GPU (MPS). That choice bounds how far the numbers extend. The qualitative findings — detection is a commodity, the base model bounds steering quality, and the cost trade-off is real and model-dependent — hold on all three models we tested, and gemma-2-2b is the useful data point here: it is roughly five times Qwen2.5-0.5B and the qualitative pattern is unchanged, with the read/write cosine even slightly higher ($\approx0.6$ against $\approx0.5$). That is evidence, not proof, and 2B is not 8B; we expect the qualitative findings to survive to the 2–8B instruct scale on the strength of it, while the specific AUCs, error rates, and knee locations should be re-measured there before being quoted. Within these small models, the detection numbers are in-distribution: probe-based detection is known to generalize poorly off-distribution. §4.9 tests one clean version of this — holding the concept fixed and holding out whole harm categories — and finds only partial generalization (a direction trained on thirteen categories catches an unseen fourteenth well above chance but well below in-distribution), with ConceptGate no less robust than a trained probe. But a substantially different attack style is a larger shift than a held-out category, and that is not tested here. And because the whole method rests on a linear direction, any concept that the frozen model encodes non-linearly is invisible to it; the intended mitigations — the layer sweep, and an MLP-probe variant that trades interpretability for capacity — are gestured at here but not fully explored.

The third cluster is about the few-shot regime and generation quality, which are the practical edges where the method frays. Everything downstream depends on the diversity of the ~10 prompts per side: a narrow or accidentally-correlated prompt set produces a direction that separates the training examples and little else, so results should always be reported with variance across seeds and prompt sets, which we have done only partially. On the write side, steering hard enough to reliably change the topic also degrades fluency, and generated text drifts out of the clean-prompt distribution as it grows — degenerate repetition alone can nudge a benign continuation across the gate — so a deployed system must tune its operating point against false-refusal and output quality, not against recall in isolation. None of these limitations is incidental; each corresponds to a parameter that the operating point exposes, and they are stated here so that the results are read with appropriate caution.

7. Conclusion

A frozen model already represents many concepts of interest in its residual stream; ConceptGate reads them across depth and writes them back. The reading is a commodity — no more accurate than a linear classifier, and no cheaper than a depth-matched probe on the same taps, since the single-concept saving is the truncated forward that any latent method shares. What survives from the reading is not efficiency but extensibility: as a training-free bank it hosts a fourteen-category taxonomy by adding each concept in milliseconds and kilobytes, where per-concept fine-tuning needs a training run — an amortization it shares with a probe bank but that fine-tuning does not have. The writing is what justifies operating inside the residual stream rather than on the text, and it is the part a classifier cannot reproduce: a few-shot, training-free steering control fit from the same data as the detector (and moderately aligned with it, cosine 0.45–0.79), measured as a monotonic dose-response with a coherent operating window (§4.6) and bounded by the competence of the base model. But the write rule alone is activation addition, which needs none of this machinery, so the claim has to be put more precisely still: what the composition uniquely provides is the conditional write — using a calibrated read to decide when to steer. Measured against blanket steering, gating suppresses more of what it targets while leaving benign generation almost entirely alone (§4.10), and that conjunction of a cheap read with a targeted write is the whole reason to assemble the parts this way. Its reach is bounded by the same thing that bounds the read: a gate whose ten examples do not match the prompts it will see is worse than no gate at all. The interactive figures are included so that these claims can be examined directly against the underlying model runs rather than taken on assertion; the points at which the method is effective and the points at which it fails are both visible in them.

References

Citation

@techreport{nish2026conceptgate,
  author      = {Pantha, Nishan},
  title       = {ConceptGate: Efficiently Learning and Steering Concepts in Language Models},
  institution = {Bits and Paradoxes},
  type        = {Technical Report},
  year        = {2026},
  month       = aug,
  url         = {https://nishparadox.com/research/conceptgate/},
  note        = {Few-shot, training-free concept detection and steering via a
                 depth-wise spectrogram of residual-stream projections; reference
                 implementation at https://github.com/NISH1001/conceptgate}
}

Plain text:

Pantha, N. (2026). ConceptGate: Efficiently Learning and Steering Concepts in Language Models. Technical Report. https://nishparadox.com/research/conceptgate/