TL;DR

Validate both confidence and quality a rule has on the facts of a knowledge graph with:

• A flexible model that use the power of the crowd
• Only few interactions with a user that validates the knowledge
• A method that finds the true confidence of a rule
• The largest dataset of manually annotated rules ever created - Link

In this paper:

• We propose Colt, a framework to assess quality and confidence of a rule by using expert-validated facts
• We enable the conditional application of rules and compute their confidence
• We establish a connection between our problem, the weighted-coverage problem, and quality-preserving Gaussian processes
• We show how our interactive learning approach only 20 user interactions, halves the error in confidence obtained with rule learning systems
• We publish our dataset consisting of 26 rules with more than 23,000 annotated facts.
• We provide code and data to be used for further research