ConfidenceBench: Testing Whether LLMs Actually Know When They're Wrong

ConfidenceBench scores 15 frontier LLMs on how well their stated confidence matches reality, and finds accuracy and calibration often don't track together.