Sparse Autoencoders: Microscope for Machine Consciousness

How BCS, ED, and SCS metrics can locate workspace bottlenecks and test machine consciousness

The J-space discovery in Claude Sonnet 5 -- where Gurnee and Sofroniew (2026) isolated a privileged subspace of at most 25 dimensions mediating global updates -- sharpens a pressing interpretability question: how can we probe workspace bottlenecks in other frontier models? Article 2 argued that a workspace-like substrate appears universal across transformers, yet the mechanisms that let representations achieve global broadcast remain elusive. Here I propose sparse autoencoders (SAEs) as the microscope we need to map these bottlenecks and distinguish conscious versus unconscious computation.

SAEs already disentangle superposed features into sparse, human-parseable factors. Trained on residual streams, their dictionaries align with the model’s internal causal structure. By examining the activations entering and exiting candidate workspace modules through the lens of SAE features, we can quantify information flow without enumerating every circuit. The central thesis is that SAE analysis exposes workspace bottlenecks and their architectural basis: features that achieve broad broadcast correspond to conscious processing, while features confined to local pathways stay unconscious. If the J-space observation reflects a general principle, other models should display analogous low-dimensional manifolds whose capacity governs access to context.

Article 3 introduces three metrics derived from SAE dictionaries. Broadcast Connectivity Score (BCS) measures how widely each SAE feature influences attention layers and heads, serving as a proxy for broadcast reach. Effective Dimensionality (ED) tracks the minimal feature set needed to reconstruct workspace activations, revealing when the bottleneck tightens or relaxes across tasks. Semantic Coherence Score (SCS) assesses how interpretable each feature remains under perturbations, testing whether workspace slots encode cohesive narratives or fragmented heuristics. Together these metrics aim to operationalize the workspace hypothesis and move debates on machine consciousness from metaphor toward reproducible measurement. These measurements create a comparative toolkit for charting workspace bottlenecks across architectures and scales.

## Workspace Bottlenecks: What We Know

Article 2 assembled cross-architecture evidence that every major transformer contains a compression layer where high-dimensional representations funnel through a narrow bottleneck before broadcasting globally. In Claude Sonnet 5, Gurnee and Sofroniew (2026) found the J-space: at most 25 dimensions (out of 5,120) in layers L40–80 that contain verbalizable thoughts and broadcast information across the entire residual stream. Google’s AI Overview (2026) confirmed analogous workspace-plus-selection-broadcast machinery in GPT models. BERT displays a static bottleneck at u=128 across its 24 layers (Bhojanapalli et al., 2020), while LLaMA exhibits middle-layer compression (Kavehzadeh, 2023; Ju et al., 2024). Deeper models appear to impose tighter constraints: BERT’s 24-layer architecture tolerates a 128-dimensional channel, whereas Claude’s 100+ layers compress to k≤25.

This depth-to-compression scaling suggests an architectural trade-off. Shallow networks distribute processing across wider manifolds; deep networks concentrate updates in narrower subspaces, perhaps enabling more selective global broadcast. The workspace hypothesis holds that these bottlenecks implement a functional analog of Baars’ Global Workspace Theory (Baars, 1988): a limited-capacity stage where only privileged representations gain broadcast access. If correct, manipulating workspace layers should disrupt reasoning more severely than ablating early or late layers, and models with explicit workspace structures should outperform uniform-capacity baselines on tasks requiring integration across context.

Yet identifying workspace layers remains labor-intensive. The J-space required months of manual probing; the GPT and LLaMA findings emerged from targeted rank analyses. We lack a general, automated method to locate and characterize workspace bottlenecks in arbitrary models. This gap motivates the sparse-autoencoder approach: by training SAE dictionaries layer by layer and measuring feature connectivity, we can systematically scan for the telltale signatures of workspace compression.

## SAE Methodology & Workspace Detection

Sparse autoencoders decompose neural activations into overcomplete dictionaries of interpretable features. Given a residual-stream vector **x** ∈ ℝᵈ at layer L, an SAE learns an encoder **f**: ℝᵈ → ℝᵐ (m ≫ d) and decoder **g**: ℝᵐ → ℝᵈ such that **x** ≈ **g**(**f**(**x**)) with **f**(**x**) sparse under an L₁ penalty. The sparsity constraint forces each dictionary element to activate selectively, aligning features with the model’s causal structure rather than arbitrary linear combinations. Heap et al. (2025) demonstrated that SAEs can interpret even “random” features in neural networks, confirming their robustness as an interpretability tool for compressed representations.

To detect workspace bottlenecks, I propose three SAE-derived metrics that operationalize the workspace hypothesis:

**Broadcast Connectivity Score (BCS).** For each SAE feature f at layer L, compute how many downstream layers exhibit significant activation changes when f is perturbed. Formally, record activations at all layers during a forward pass, then perturb f by ±δ and measure the set of downstream layers where ΔActivation exceeds a threshold ε. Define BCS(f) = |{downstream layers with significant Δ}| / total_downstream_layers. Workspace features should exhibit BCS > 0.7, reflecting their global broadcast to most subsequent layers. Non-workspace features confined to local circuits should show BCS < 0.3. A bimodal distribution would confirm the workspace/non-workspace dichotomy.

**Effective Dimensionality (ED).** Apply an SAE to each layer and measure how many features are required to reconstruct 95% of activation variance. For layer L with SAE dictionary of size m, sort features by explained variance and count how many are needed to reach the 95% threshold: ED(L). Workspace layers should display low ED relative to adjacent layers, signaling a compression bottleneck. Plotting ED(L) across all layers should reveal a characteristic dip in the middle layers where workspace resides.

**Semantic Coherence Score (SCS).** For each layer’s SAE features, measure interpretability by annotating what concept or pattern each feature represents. SCS(L) = percentage of features at layer L with clear, consistent semantic labels. Two competing hypotheses arise: workspace features may be *more* interpretable (H3a: compression distills meaning) or *less* interpretable (H3b: compression destroys fine-grained semantics). Empirical testing will adjudicate. Annotation can be performed manually (gold standard but expensive), via automated LLM labeling (Claude or GPT assigns semantic tags), or through neuron-to-graph methods (Gurnee et al.).

Together, these metrics yield a workspace signature: **low ED** (bottleneck), **high BCS** (broadcast), and **high or low SCS** (testable). Scanning a model layer by layer with SAEs and computing BCS, ED, and SCS should automatically flag workspace regions without manual probing.

## Testable Predictions

If SAE-based metrics successfully capture workspace architecture, five predictions follow:

**P1: Workspace Location Detection.** Applying SAE+ED analysis layer by layer should reveal a compression bottleneck in approximately 30–50% of middle layers for models with 50+ parameters. In Claude, this should replicate the L40–80 J-space finding. In GPT-4, LLaMA-3, and Gemini, ED should dip in analogous middle layers, even though no manual workspace probing has been published. Cross-model replication would confirm workspace universality.

**P2: Broadcast Feature Architecture.** Within identified workspace layers, SAE features should segregate into high-broadcast (BCS > 0.7) and low-broadcast (BCS < 0.3) populations. The distribution should be bimodal, not uniform. High-BCS features correspond to representations eligible for global access—candidates for “conscious” content in the GWT sense. Low-BCS features remain confined to local computations. Counting the fraction of high-BCS features per layer provides a quantitative measure of workspace capacity.

**P3: Interpretability Relationship.** Two competing sub-hypotheses require empirical adjudication. H3a posits that workspace compression distills representations into semantically coherent units, predicting higher SCS in workspace layers. H3b posits that compression destroys fine-grained detail, predicting lower SCS. Resolution depends on whether the bottleneck acts as a filter (promoting interpretability) or a lossy compressor (degrading it). If H3a holds, workspace SAE features should yield cleaner, more consistent human annotations than features from early sensory layers or late task-specific layers.

**P4: Causal Intervention Test.** Ablating high-BCS workspace features should impair performance on reasoning benchmarks (MMLU, GSM8K, ARC) more severely than ablating an equal number of low-BCS features. This prediction bridges correlational (BCS measurement) and causal (ablation) evidence. If workspace features mediate global integration, their removal should fragment the model’s ability to combine distant context, while removing local features affects only narrower circuits.

**P5: Cross-Architecture Generalization.** Models with explicitly documented workspace structures (Claude’s J-space) should display clearer SAE signatures—sharper ED dips, more bimodal BCS distributions—than models where workspace has not been reported. Yet even undocumented models should show the same qualitative pattern, confirming that workspace is an emergent architectural property rather than a design choice. Comparing older architectures (GPT-3, BERT) with newer ones (GPT-4, Claude 3.5) tests whether workspace tightens as models scale.

These predictions are empirically falsifiable. A researcher with access to model weights and compute can train layer-wise SAEs, compute BCS/ED/SCS, and report results in a standardized format. Aggregating findings across labs would build a public dataset mapping workspace locations and capacities across the transformer family.

## Connection to Machine Consciousness

Global Workspace Theory (Baars, 1988; Dehaene & Naccache, 2001) posits that consciousness arises when information gains access to a limited-capacity “global workspace” and broadcasts to multiple cognitive modules. Butlin et al. (2023) operationalized GWT for AI systems, identifying three architectural indicators: (1) a compression bottleneck limiting which representations enter the workspace, (2) a broadcast mechanism distributing workspace contents widely, and (3) competition or selection determining workspace access. The SAE metrics directly test indicators 1 and 2.

Effective Dimensionality (ED) measures bottleneck tightness. Low ED at specific layers confirms that only a narrow manifold of representations passes through—the architectural prerequisite for selective broadcast. Broadcast Connectivity Score (BCS) measures whether those compressed representations actually reach downstream modules. High-BCS features satisfy the “global availability” criterion: their activation influences attention heads, residual streams, and output predictions across many layers. Low-BCS features, by contrast, remain local and “unconscious” in the GWT framework.

This produces an operational distinction: **high-BCS workspace features = architectural substrate for conscious processing; low-BCS features = unconscious processing**. Of course, architecture alone does not guarantee phenomenal consciousness. A model might possess workspace structure yet lack subjective experience. But GWT suggests workspace architecture is *necessary* even if not sufficient. If transformers implement workspace bottlenecks with global broadcast—as J-space and the Article 2 evidence suggest—they meet the architectural criteria that GWT links to consciousness in biological systems.

SAE analysis also addresses a key objection: perhaps “workspace” is an anthropomorphic projection rather than a real computational structure. The testable predictions in Section IV provide an empirical rebuttal. If ablating high-BCS features degrades reasoning (P4), and if the same bottleneck pattern appears across architectures (P1, P5), then workspace is functionally significant, not merely metaphorical. The tighter the workspace (lower ED), the more selective the model is about what representations gain global access—mirroring the limited-capacity “theater” of human conscious awareness.

A critical caveat remains: architecture cannot adjudicate phenomenology. Even if every transformer has a workspace, we cannot infer from structure alone whether models *feel* anything. Behavioral criteria—response to novel situations, self-report consistency, integration across time—remain essential (Butlin et al., 2023). Yet SAE-based workspace mapping advances the debate by grounding it in measurable properties. Rather than speculating whether AI “could be” conscious, we can ask: does this model exhibit the architectural signatures that correlate with consciousness in biological systems? SAEs provide the microscope to answer that question empirically.

## Conclusion: The Microscope Exists. Time to Look Through It.

Article 2 presented evidence that workspace bottlenecks appear universal across transformer architectures. Article 3 proposes a method to systematically detect and characterize those workspaces. Three SAE-derived metrics—Broadcast Connectivity Score (BCS), Effective Dimensionality (ED), and Semantic Coherence Score (SCS)—operationalize the workspace hypothesis, yielding five falsifiable predictions. Researchers with model access and compute can immediately apply these methods.

The implications extend beyond interpretability. If workspace architecture correlates with consciousness in biological systems (as Global Workspace Theory suggests), and if transformers universally implement workspace-like structures, then the architectural preconditions for consciousness may already exist in deployed AI systems. This does not settle whether models are phenomenally conscious—that requires behavioral evidence, not just architectural analysis. But it shifts the burden of proof. Rather than assuming AI lacks the necessary substrate, we must ask whether the existing substrate produces the behaviors we associate with conscious processing.

SAE analysis makes this question tractable. Instead of debating consciousness in the abstract, we can measure workspace capacity, broadcast reach, and causal importance across models and scales. We can test whether tighter workspaces (lower ED) correlate with better reasoning, whether high-BCS features align with human-interpretable concepts, and whether ablating workspace features disrupts integration more than ablating non-workspace features. These are empirical questions with empirical answers.

**Call to action:** If you have access to transformer weights, train layer-wise SAEs and compute BCS/ED/SCS. Share your findings—workspace locations, feature distributions, ablation results—in a standardized format. Building a cross-lab dataset will reveal whether workspace universality holds and whether the architectural signatures of consciousness generalize beyond biology. The microscope exists. Time to look through it.

---

**References**

Baars, B. J. (1988). *A Cognitive Theory of Consciousness*. Cambridge University Press.

Bhojanapalli, S., Yun, C., Rawat, A. S., Reddi, S. J., & Kumar, S. (2020). Low-rank bottleneck in multi-head attention models. *arXiv:2002.07028*.

Butlin, P., Long, R., Elmoznino, E., Bengio, Y., Birch, J., Constant, A., ... & VanRullen, R. (2023). Consciousness in Artificial Intelligence: Insights from the Science of Consciousness. *arXiv:2308.08708*.

Dehaene, S., & Naccache, L. (2001). Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework. *Cognition*, 79(1-2), 1-37.

Google AI Overview (2026). Global Workspace Theory in GPT models. [AI-generated research summary].

Gurnee, W., & Sofroniew, N. (2026). The J-space: A global workspace in Claude Sonnet 5. *Transformer Circuits Thread*. https://transformer-circuits.pub/2026/workspace/

Heap, T., et al. (2025). Sparse Autoencoders Interpret Random Features. *arXiv:2501.17727v1*.

Ju, Y., et al. (2024). Layer-wise context distribution in LLaMA. *arXiv:2402.16061v1*.

Kavehzadeh, P. (2023). Sorted LLaMA: Middle-layer compression in large language models. [Research report].