aiqbench takeAI Products

Vals aims to address the critical gap between rapidly advancing AI models and outdated academic benchmarks that fail to accurately measure current capabilities. By developing more rigorous and neutral evaluation metrics, the startup seeks to restore trust in how model performance is verified for real-world deployment. This initiative directly impacts the reliability of claims made by major AI developers regarding their system's intelligence and utility.

Source coveringTechCrunch

What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Vals, backed by Andreessen Horowitz, is looking to become the gold standard for AI benchmarking

TechCrunchSeptember 19, 20264 min read

Vals AI is hoping to make AI benchmarking a more neutral and trustworthy resource in a world increasingly inundated by AI models.

Benchmarking has become the industry norm for how AI companies validate their models’ capabilities and, when the metrics swing in their favor, stand out from competitors and advertise their superiority. In other words, good benchmarks pretty much always mean good PR. Unfortunately, companies have also figured out how to outwit legacy benchmarking systems — many of which are older, and not built to measure the capabilities of modern models. Vals, a startup formed in 2024, says that it is on a mission to fix this very…

Excerpt shown under the publisher's RSS terms · 86 words · full text and figures are on the publisher's site

Read at TechCrunch ↗Ask about this article