Reward model — AI Dictionary

A model trained on human preference comparisons to predict which of two responses people would prefer; a learned stand-in for a human judge.