What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes

Victor Lecomte, Kushal Thaman, Rylan Schaeffer, Naomi Bashkansky, Trevor Chow, Sanmi Koyejo

arXiv preprint Under Review

December 2024

Mechanistic Interpretability Polysemanticity Neural Networks Representation Learning

arXiv

Summary

Polysemanticity may arise from incidental causes rather than superposition.