Sparse autoencoders provide a promising unsupervised approach for extracting interpretable features from a language model by reconstructing activations from a sparse bottleneck layer. Since language models learn many concepts, autoencoders need to be very large to recover all relevant features. However, studying the properties of autoencoder scaling is difficult due to the need to balance reconstruction and sparsity objectives and the presence of dead latents. We propose using k-sparse autoencoders [Makhzani and Frey, 2013] to directly control sparsity, simplifying tuning and improving the reconstruction-sparsity frontier. Additionally, we find modifications that result in few dead latents, even at the largest scales we tried. Using these techniques, we find clean scaling laws with respect to autoencoder size and sparsity. We also introduce several new metrics for evaluating feature quality based on the recovery of hypothesized features, the explainability of activation patterns, and the sparsity of downstream effects. These metrics all generally improve with autoencoder size. To demonstrate the scalability of our approach, we train a 16 million latent autoencoder on GPT-4 activations for 40 billion tokens. We release training code and autoencoders for open-source models, as well as a visualizer.
核心贡献 · Key contributions
提出使用 TopK 激活函数的 k 稀疏自编码器直接控制稀疏性，简化调参并改善重建-稀疏性前沿。 Propose k-sparse autoencoders with TopK activation to directly control sparsity, simplifying tuning and improving reconstruction-sparsity frontier.
发现修改方法（编码器初始化为解码器转置、辅助损失）即使在最大规模下也能减少死亡潜变量。 Identify modifications (encoder initialized to decoder transpose, auxiliary loss) that result in few dead latents even at largest scales.
展示自编码器规模和稀疏性的清晰缩放定律，并在 GPT-4 激活上训练 1600 万潜变量的自编码器。 Demonstrate clean scaling laws for autoencoder size and sparsity, and train a 16 million latent autoencoder on GPT-4 activations.
引入新的特征质量指标：探针损失、可解释性（N2G）、消融稀疏性和下游损失。 Introduce new metrics for feature quality: probe loss, explainability (N2G), ablation sparsity, and downstream loss.
表明更大的自编码器通常在所有提出的质量指标上表现更好。 Show that larger autoencoders generally improve on all proposed quality metrics.
发布训练代码、开源模型的自编码器以及特征可视化工具。 Release training code, autoencoders for open-source models, and a feature visualizer.
局限 · Limitations
TopK 强制每个词元恰好 k 个活跃潜变量，可能次优；期望 L0 约束可能更好。 TopK forces exactly k active latents per token, which may be suboptimal; expected L0 constraint might be better.
探针损失和可解释性等指标依赖于对自然特征的假设，可能无法捕捉所有方面。 Metrics like probe loss and explainability rely on assumptions about natural features and may not capture all aspects.
N2G 解释仅限于简单的 n-gram 模式，无法捕捉复杂的特征交互。 N2G explanations are limited to simple n-gram patterns and cannot capture complex feature interactions.
训练仅使用 64 词元上下文，可能不足以捕捉 GPT-4 中的长程行为。 Training uses only 64-token context, potentially insufficient for capturing long-range behaviors in GPT-4.
缩放定律在极端规模下可能因每个潜变量的词元预算次线性而失效。 Scaling laws may break at extreme scales due to sublinear token budget per latent.
论文章节 · Sections（共 21）
摘要Abstract
1 引言1 Introduction
2.1 设置2.1 Setup
2.2 基线：ReLU 自编码器2.2 Baseline: ReLU autoencoders
2.3 TopK 激活函数2.3 TopK activation function
2.4 防止死神经元2.4 Preventing dead latents
3 缩放定律3 Scaling laws
3.1 神经元数量3.1 Number of latents
3.2 主体模型大小 Ls(N)3.2 Subject model size Ls(N)subscript𝐿𝑠𝑁L_{s}(N)
4 评估4 Evaluation
4.1 下游损失4.1 Downstream loss
4.2 用一维探针恢复已知特征4.2 Recovering known features with 1d probes
4.3 为特征寻找简单解释4.3 Finding simple explanations for features