Cocoon: Semantic Table Profiling Using Large Language Models

• Main Authors: Huang, Zezhou, Wu, Eugene
• Format: Journal Article
• Language: English
• Published: 18-04-2024
• Subjects: Computer Science - Databases
• DOI: 10.48550/arxiv.2404.12552

Data profilers play a crucial role in the preprocessing phase of data
analysis by identifying quality issues such as missing, extreme, or erroneous
values. Traditionally, profilers have relied solely on statistical methods,
which lead to high false positives and false negatives. For example, they may
incorrectly flag missing values where such absences are expected and normal
based on the data's semantic context. To address these, we introduce Cocoon, a
data profiling system that integrates LLMs to imbue statistical profiling with
semantics. Cocoon enhances traditional profiling methods by adding a three-step
process: Semantic Context, Semantic Profile, and Semantic Review. Our user
studies show that Cocoon is highly effective at accurately discerning whether
anomalies are genuine errors requiring correction or acceptable variations
based on the semantics for real-world datasets.