Researchers Optimise Evidence Selection to Overcome Majority Errors in Legal Reasoning

Researchers at Singapore Management University, led by Laura Wynter, have developed a novel methodology to systematically evaluate reasoning fragments and aggregate them using quality-derived weights, offering a principled approach to combining reasoning processes within large language models. These models, while demonstrating proficiency in various expert-level examinations, often exhibit brittleness when applied to specialised, evidence-intensive domains such as law. Errors in these domains stem not only from deficiencies in general world knowledge but also from the models’ inability to discern subtle distinctions between pieces of evidence and their inconsistent application of supporting evidence during reasoning. The team’s work addresses these limitations, aiming to improve the reliability of…