This work studies the mathematical structure of learned optimization updates and uses that structure to design optimizers.