Jev Ordinal-Scale Bias
A third-party preprint examining how ordered answer scales affect option use and errors in JEV-like direct-decision models.
The authors evaluate JEV 1.13 and three KEV variants across 40 datasets, compare ordinal and nominal scales, and report a BA-LoRA mitigation experiment. All figures shown here are author-reported.
- Publisher
- arXiv
- Published or submitted
- September 30, 2026
- Last verified
- October 8, 2026
Type Safe catalogs sources and evidence boundaries. Provider datasets and third-party preprints do not become an independent leaderboard.
Key facts
- Evidence type
- Third-party preprint
- Datasets
- 40
- Models studied
- JEV 1.13 and three KEV variants
Task and scope
JEV 1.13 and three KEV variants evaluated across 40 datasets with ordinal and nominal output scales.
Paper authors
Tianxiang Gao, Jinzhe Li, Zhiyuan Li, Yi Chang, Yuan Wu
Paper-reported findings
- On ANLI, the paper reports 38.8% of predictions and 51.3% of errors concentrated on Neutral at 74.95% accuracy.
- Reported ordinal-scale utilization is 67–76%, compared with 87–102% for nominal scales; at K=14 the reported range falls to 26–75%.
- The BA-LoRA experiment is reported to improve utilization from roughly 47% to 86% in the evaluated setting.
Code and data availability · Code
The authors link a public GitHub repository for the study code.
Code and data availability · Data
The paper names the evaluated datasets; no separate consolidated dataset artifact was verified.
Method and usage
- The study compares ordered ordinal scales with nominal answer sets across 40 datasets.
- It analyzes output utilization and error concentration for JEV 1.13 and three KEV variants.
- The authors test BA-LoRA as a mitigation for low ordinal-scale utilization.
Limitations and evidence boundaries
- The taxonomy covers three scale sources and may not represent every ordered decision task.
- The study examines one hosted system and one open model family, limiting provider and architecture coverage.
- Potential training exposure cannot be excluded for all evaluated datasets.