Few-Shot Knowledge Validations using Rules

Michael Loster, Davide Mottin, Paolo Papotti, Jan Ehmueller, Benjamin Feldmann, Felix Naumann

TheWebConf (WWW) 2021

How can be sure of a confidence of a rule in a knowledge graph?

How can be sure of a confidence of a rule in a knowledge graph?

TL;DR

Validate both confidence and quality a rule has on the facts of a knowledge graph with:

  • A flexible model that use the power of the crowd
  • Only few interactions with a user that validates the knowledge
  • A method that finds the true confidence of a rule
  • The largest dataset of manually annotated rules ever created - Link

In this paper:

  • We propose Colt, a framework to assess quality and confidence of a rule by using expert-validated facts
  • We enable the conditional application of rules and compute their confidence
  • We establish a connection between our problem, the weighted-coverage problem, and quality-preserving Gaussian processes
  • We show how our interactive learning approach only 20 user interactions, halves the error in confidence obtained with rule learning systems
  • We publish our dataset consisting of 26 rules with more than 23,000 annotated facts.
  • We provide code and data to be used for further research