Stability vs. Manipulability: Evaluating Robustness Under Post-Decision Interaction in LLM Judges

본문 미리보기

arXiv:2606.05384v1 Announce Type: new Abstract: LLM-as-judge evaluation is widely used in benchmarking pipelines, where model outputs are compared and ranked using automated evaluators. These pipelines typically assume that judgments are stable properties of fixed inputs. We show that this assumption does not hold under interaction. We study post-decision manipulability: the extent to which an evaluation outcome can be altered through subsequent conversation with the judge after an initial deci

Stability vs. Manipulability: Evaluating Robustness Under Post-Decision Interaction in LLM Judges

본문 미리보기

관련 글

Thousand Token Wood: shipping a multi-agent economy on a 3B model

An interpretable and trustworthy AI framework for large-scale longitudinal structure-pain association studies using data from the Osteoarthritis Initiative (OAI)

SentinelBench: A Benchmark for Long-Running Monitoring Agents

LeanMarathon: Toward Reliable AI Co-Mathematicians through Long-Horizon Lean Autoformalization