analiza aiqbench

Amazon researchers have demonstrated that relying on LLM judge panels as independent validators is flawed because shared biases and prompting can create correlated errors that artificially inflate consensus. By applying Ising models to map these dependencies within a network of judges, the team developed an aggregation method that outperforms traditional weighted voting by up to 14% across classification and summarization tasks. This approach effectively identifies redundant evaluators and corrects for systematic blind spots, offering a more robust framework for auditing large language model outputs.