Abstract
We release a dataset of text snippets that strongly activate MLP neurons in the Llama-3-8B model. Using this dataset, we demonstrate examples of meaningful features that can be discovered. We open-source our work, enabling others to easily create similar datasets for other transformer models and uncover important features. We also create a simple web interface to facilitate the exploration of MLP neurons.