Human Eval, Arenas, and LLM Judges

When there is no answer key: human evaluation, preference arenas and their ratings, and models judging models — with the biases of each.

Content last verified 2026-09.

Lessons

1. Human Evaluation
2. Arenas and Ratings
3. LLM as Judge

Sources

• Zheng et al. (2023) — Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Chiang et al. (2024) — Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference