Arbitrage: a speculative-decoding method for cheaper long-chain-of-thought reasoning

Apple researchers propose "Arbitrage," a speculative-decoding technique that cuts wasted rejections to make long chain-of-thought reasoning cheaper to run.