Distillation — AI Dictionary

Training a smaller 'student' model to reproduce a larger 'teacher' model's outputs, transferring much of its behavior into far fewer parameters.