I’m working on a novel (toy scale) kind of LM that is explicitly interpretable and programmable. In that it learns to predict words from text and you can directly see what it learned and teach it new things without retraining.