Sensory Transduction

The Problem

Every major mechanistic interpretability method — sparse autoencoders (SAEs), probing classifiers, activation classifiers — converts a model's internal state into text before presenting it to a human. The assumption is that human-readable text is a good enough representation of what's happening inside the model.

I think that's limiting. When you convert activations into text features, you don't just lose some information — you lose it in a biased direction. Anything that doesn't map cleanly onto language categories gets dropped, and you never know what you missed.

I call this interpretive displacement: the text label assigned to an activation pattern can actively misdirect human understanding, not just simplify it.

The Approach: Activation Sensory Transduction (AST)

Instead of converting activations into text, AST routes a compressed version of the model's activation signals directly to a human operator's sensory system — bypassing language entirely.

The analogy is a radiologist reading an MRI scan. They don't need to transcribe the scan into words to act on what they see. A trained operator, given enough exposure to activation patterns, may develop the same kind of pattern-recognition intuition — without needing every internal state to be labeled in text first.

• Visual

  Heatmaps, phase portraits, and activation trajectory plots that operators can learn to read over time.

• Auditory (Sonification)

  Mapping activation dimensions to pitch, timbre, and rhythm — letting operators hear when a model's internal state shifts.

• Haptic / Tactile

  Encoding activations as touch, pressure, or vibration — adding another sensory channel for oversight.

• BCI (Brain–Computer Interface)

  The long-term vision: routing compressed activation signals directly via multi-modal BCI, giving operators maximum bandwidth for detecting model anomalies.

The Experiments

Experiment 1 — How much information does text lose?

I gave GPT-2 240 prompts (factual and counterfactual) and measured the model's uncertainty for each using entropy. Then I trained two classifiers to predict whether the model was uncertain:

• Classifier A — trained on the model's raw internal activations → 93.1% accurate

• Classifier B — trained on SAE text features derived from those activations → 84.7% accurate

The 8.3 percentage point gap is behaviorally relevant information that existed in the model's geometry but didn't survive the compression into human-readable text features.

→ Source code (notebook)

Experiment 2 — Can text features actively mislead?

I looked at cases where the two classifiers disagreed. In those disagreement cases, the raw activation classifier matched ground truth more often than the SAE classifier. More strikingly: when I computed entropy over SAE features rather than model outputs, the ordering of factual vs. counterfactual prompts reversed — the opposite direction from the model's actual uncertainty.

This confirms interpretive displacement: the SAE isn't just losing signal, it is sometimes pointing in the wrong direction entirely.

→ Source code (notebook)

Experiment 3 — Demonstrations of AST modalities

To test the core idea in practice, I built three demonstration systems:

• Phase Portrait (Visual)

  Plots the trajectory of activation vectors across prompt tokens in 2D. Factual prompts converge quickly; counterfactual and open-ended prompts show wider, more distinctive paths. Operators could learn to recognise prompt types by trajectory shape alone.

  → Source code

• Sonification Dashboard (Audio)

  A real-time dashboard that maps dimensionality-reduced activations and entropy to pitch, timbre, spatialization, and rhythm. The goal is for trained operators to be able to hear a shift in model state before any text label is produced.

  → Source code

• EDF View via MNE-Python (BCI-style)

  Activation vectors are serialised into EDF format and rendered using MNE-Python — the same tooling used for EEG/brain signals. The result is multi-channel traces, heatmaps, and spectral views of model state, without ever converting to text. Generated from the prompt "France" as a test case.

  → Source code

Why This Matters

AI safety oversight currently runs through a single channel: text. A single channel is a single point of failure. If a model's internal deceptive state is hard to describe in text, it may still be detectable through a trained sensory channel.

AST doesn't replace SAEs or probing classifiers — it adds a parallel oversight channel. If a trained operator flags an anomalous activation pattern before any text-based probe names it, that's a safety gain, regardless of whether the operator can articulate what they perceived.

"The doctor does not need to dictate the MRI to act on what they see."