论文标题
Hycedis：深层文档情报系统的混合信心引擎
HYCEDIS: HYbrid Confidence Engine for Deep Document Intelligence System
论文作者
论文摘要
测量AI模型的信心对于在现实世界中安全部署AI至关重要。置信度测量的一种重要应用是从扫描文档中提取信息。但是，没有解决方案可以为当前最新的基于学习的信息提取器提供可靠的置信度评分。在本文中，我们提出了一个完整而新颖的体系结构，以衡量文档信息提取任务中当前深度学习模型的信心。我们的体系结构由多模式的保形预测指标和一个面向群集的异常检测器组成，经过训练，可以忠实地估算其对输出的信心，而无需修改主机模型。我们在现实折叠数据集上评估了我们的体系结构，不仅可以通过巨大的利润来优于相互竞争的置信度估计器，而且还表明了分布数据外的概括能力。
Measuring the confidence of AI models is critical for safely deploying AI in real-world industrial systems. One important application of confidence measurement is information extraction from scanned documents. However, there exists no solution to provide reliable confidence score for current state-of-the-art deep-learning-based information extractors. In this paper, we propose a complete and novel architecture to measure confidence of current deep learning models in document information extraction task. Our architecture consists of a Multi-modal Conformal Predictor and a Variational Cluster-oriented Anomaly Detector, trained to faithfully estimate its confidence on its outputs without the need of host models modification. We evaluate our architecture on real-wold datasets, not only outperforming competing confidence estimators by a huge margin but also demonstrating generalization ability to out-of-distribution data.
