Researchers have proposed novel unsupervised metrics to evaluate the accuracy of Large Language Models (LLMs) in adhering to assigned tasks. The metrics leverage information theory and thermodynamics to measure how faithfully a model transforms context into answers, helping to control hallucination. The proposed framework was tested on financial report summarization models.
In a significant development to address the challenges of hallucination in Large Language Models (LLMs), researchers have presented a new scientific paper on arXiv proposing an innovative framework for evaluating semantic fidelity and model faithfulness to the assigned task. The new metrics are based on concepts from information theory and thermodynamics, offering objective tools to measure the extent to which a model adheres to the provided context without fabrication or distortion.
The proposed framework treats a Large Language Model as a binary information engine, where the hidden layers act as a "Maxwell's demon" controlling the transformation of context (C) into an answer (A) via a prompt (Q). Question-Context-Answer triples (QCA) are modeled as probability distributions over shared topics. The topic transitions from context to question and answer are represented by two transition matrices (Q and A) encoding the query intent and the actual outcome, respectively.
The Semantic Fidelity (SF) metric measures the faithfulness of any QCA triple through the Kullback-Leibler (KL) divergence between these two matrices. The two matrices are inferred simultaneously via convex optimization of this divergence, and the final metric value is obtained by mapping the minimum divergence to the unit interval [0,1], where higher scores indicate greater faithfulness.
Additionally, the researchers propose a secondary, thermodynamics-based metric called Semantic Entropy Production (SEP) in answer generation, showing that high faithfulness generally implies low entropy production. The SF and SEP metrics can be used together or separately to evaluate LLMs and control hallucination.
The effectiveness of the proposed framework was demonstrated by applying it to the task of summarizing corporate financial reports (SEC 10-K filings), showcasing its ability to distinguish accurate responses from those suffering from hallucination or deviation from context. These metrics open the door to developing more reliable and transparent language models, especially in sensitive applications requiring high accuracy and strict adherence to source information.
Source: arXiv AI Papers | Exclusive coverage from AI Tools Oasis
Bringing you the latest news and analysis in the world of Artificial Intelligence with accuracy and credibility. Follow us for all updates.
OpenAI is advancing its ambitious super app project, aiming to integrate advanced AI capabilities into a single, multifunctional platform. This development is part of the company's strategy to expand services and deliver a unified user experience. Discover the full details and expected impact of this move.
Notion has restored access to its Anthropic AI integration after a 4-hour outage disrupted users relying on Claude-powered features. The incident highlights the growing dependency on AI productivity tools and raises questions about infrastructure stability. All user data remained secure during the disruption.
A new report from TechCrunch AI warns of a potential 'Tokenpocalypse'—a massive collapse of digital tokens due to oversupply. With over 80% of new tokens losing 90% of their value, the market faces a crisis reminiscent of the dot-com bubble. This analysis explores the risks, impacts, and how investors can protect themselves.
