MoE Microscope

MoE Microscope is an educational visualization tool that lets you step through recorded training runs of tiny mixture-of-experts (MoE) models. It displays how expert load balances, routing decisions, and loss metrics evolve across training steps. The interface shows scalars as sparklines, vectors as labeled bars, and matrices with highlighted active gates, making the internal mechanics of sparse models visible and understandable.

Designed for ML researchers and students, it includes lessons covering dense baselines, top-k routing, recurrent blocks, state-space models, and hybrid architectures. All computations are pre-recorded—no code runs in your browser. The creator built this as part of an ML study group exploring model optimization techniques using an array-programming DSL written in Rust.