MLP Transcoders -Bridging Layers

A Mechanistic Layer Perceptron Transcoder is the linear + non-linear block in a transformer layer (usually called the MLP or FFN) understood mechanistically as a feature translator. It detects certain directions in the residual stream → applies a nonlinearity → and writes out transformed features into new directions.

“MLP = a dictionary of detectors + a dictionary of writers.”

On the left, you see the input, like small colored blocks coming in. In the middle is the thinking part, where the machine has little tools that do things like stop repeats, remember patterns, and choose the right path for the information. On the right is the output, where the machine turns the cleaned and organized information back into new blocks. So the whole picture shows a machine that takes information in, fixes and organizes it using simple rules, and then sends out a better version.

Our team

Our strength lies in our individuality. Set up by Esther Bryce, the team strives to bring in the best talent in various fields, from architecture to interior design and sales.