KL penalty — AI Dictionary

A term that punishes the policy for straying too far, in probability, from the frozen reference model.