Activity Cube

An Activity Cube is a structured, multi-dimensional representation of internal model activity for a single model inference or batch of inferences. It is designed to be:

• Machine-readable

• Comparable across runs / models

• Suitable for visualizations at multiple granularities

• Aligned with explainability and safety audit requirements

It systematically captures information at the intersections of Tokens, Layers, and Metrics

An Activity Cube is a 3D data structure:

ActivityCube : T × L × M

Where:

• T = Token dimension (sequence index)

• L = Layer dimension (model depth / block index)

• M = Metric dimension (set of quantified activity measures)

Each cell in the cube: cube[t][l][m] represents a metric value for token position t at layer l for metric m.

1. Token Axis (T)

• Indexes positions in the input sequence.

• Length = sequence length (seq_len) for a given run.

• That can be extended to batches by adding an outer dimension:

Batch × Token × Layer × Metric

• This axis supports token-wise analysis.

2. Layer Axis (L)

• Indexes structural depth of the model.

• e.g., transformer blocks, or any sequential layer stack.

• Zero-based: 0 … num_layers−1.

Supports tracing the flow of representation from input embedding to final output.

3. Metric Axis (M)

Each entry in the metric axis is a measured scalar that quantifies some aspect of internal activity. Common metrics include:

• energy — L2 norm of hidden state vector

• mean_activation — average value of the hidden vector

• std_activation — standard deviation of hidden vector

• attention_entropy — entropy of attention distribution

• sparsity — fraction of activations near zero

• variance — variation across hidden units

• signal_ratio — relative magnitude compared to baseline

The metric set can be extended based on needs (e.g., robustness, attribution, gradient norms).

Cube Cell Semantics : cube[t][l][m] = metric_m(hidden_representation[t, l])

Where:

• hidden_representation[t, l] is the internal vector at token t after layer l.

• metric_m is a function that reduces a high-dim hidden vector to a scalar.
  

JSON schema skeleton for an Activity Cube:

{
"activity_cube": {
"tokens": ["<token_0>", "<token_1>", "..."],
"layers": [
{
"name": "layer_0_identifier",
"metrics": [
{
"name": "energy",
"values": [0.123, 0.456, "..."]
},
{
"name": "mean_activation",
"values": [0.0123, 0.0456, "..."]
}
]
},
{
"name": "layer_1_identifier",
"metrics": [
{
"name": "energy",
"values": [0.234, 0.567, "..."]
},
{
"name": "mean_activation",
"values": [0.0234, 0.0567, "..."]
}
]
}
]
}