Engineering Monosemanticity in Toy Models

• Main Authors: Jermyn, Adam S, Schiefer, Nicholas, Hubinger, Evan
• Format: Journal Article
• Language: English
• Published: 16-11-2022
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Learning
• DOI: 10.48550/arxiv.2211.09169

In some neural networks, individual neurons correspond to natural
``features'' in the input. Such \emph{monosemantic} neurons are of great help
in interpretability studies, as they can be cleanly understood. In this work we
report preliminary attempts to engineer monosemanticity in toy models. We find
that models can be made more monosemantic without increasing the loss by just
changing which local minimum the training process finds. More monosemantic loss
minima have moderate negative biases, and we are able to use this fact to
engineer highly monosemantic models. We are able to mechanistically interpret
these models, including the residual polysemantic neurons, and uncover a simple
yet surprising algorithm. Finally, we find that providing models with more
neurons per layer makes the models more monosemantic, albeit at increased
computational cost. These findings point to a number of new questions and
avenues for engineering monosemanticity, which we intend to study these in
future work.