About Leaderboard
eiboard is a static leaderboard for large language models, built fromevalscope evaluation reports. Every number on this site is computed at build time — no runtime fetching, no client-side data loading.
How benchmarks work
Each evaluation run is executed by evalscope against one model and one or more benchmark datasets. A run produces a per-benchmark report containing the metric score and the sample count (n). eiboard loads those reports, dedupes by(model, dataset)keeping the most recent run, and renders one row per model with one column per benchmark metric. Missing evaluations are shown as an em-dash — absent is never treated as zero.
How scores are computed
Scores are taken directly from evalscope's per-dataset aggregation (typically mean). Each cell shows the score and the sample count (score · n=). Cells with fewer than 30 samples are dimmed, and cells whose metric differs from the column's primary metric are marked with a dashed border and a tooltip so cross-metric comparison is never silent.
Data provenance
Run metadata — model id, datasets, sample limits, generation parameters — is read from each run'sconfigs/task_config.yaml. Sensitive fields (API keys, URLs, sandbox settings) are never parsed or exposed. Currently tracking3 models,2 benchmarks, and 5 evaluationsacross 45 runs.
evalscope version: 1.10.0, 1.9.1