Reason to Trust: Certifying Conclusions in Data Analysis

Mentored by Markus Krötzsch, Carsten Lutz
at TU Dresden, Chair of Knowledge-Based Systems

Current advances in AI are deepening the crisis of trust in digital information, with vast amounts of misinformation being created and shared either deliberately or accidentally. Trust is often attached to individual sources, but the complex processing pipelines of data analysis and AI obtain their output by combining and modifying data from many sources. The goal of this PhD project therefore is to explore concepts for certifying the validity of such outputs in a way that can be validated independently (e.g., by a data journalist).

Our starting point are existing ideas from several related fields. In databases, provenance is the art of enriching results with information that explains their origins. In knowledge representation and formal methods, proofs are instructions on how to confirm a result. And in trusted computing, cryptographic certificates are used to ensure the validity of information. All of these approaches are declarative: they can be validated by following trusted principles, independently of specific implementations.

As a declarative computation platform, the PhD project can build upon our system Nemo, which supports data analysis and transformation on large knowledge graphs. This rule-based approach to computation can be extended with further data analysis tools as required. As an application scenario, the Wikidata knowledge graph can be a first target, since it combines data from many sources and requires reference information to certify their trust (in a rather informal way that is hard to validate).

The topic allows for several specialisations, depending on own interests. Important research challenges in this context include:

• Conceptual: What kind of certification is there (deductive, statistical, cryptographic, etc.)?How can they be represented and shared? Can background knowledge be used to simplify certificates? How can existing AI explanation approaches be integrated in this framework?
• Technical: How can certificates be computed and validated? How can this scale to larger data sets? How should one share certificates across applications? How can these functionalities be integrated into existing systems of computation?
• Human and socio-technical: How can invalid certificates allow humans to understand what went wrong? How can certificates be integrated in data curation workflows? Can certification be a basis for trustworthy shared information ecosystems and community work?

Work Environment

You will be working at the Chair of Knowledge-Based Systems (KBS) at the Faculty of Computer Science of TU Dresden. At KBS, you will be part of a successful team of experienced researchers of many nationalities and backgrounds. You will work in the vibrant scientific environment of the International Center for Computational Logic (ICCL). TU Dresden is one of the leading German research universities in a highly livable city with a rich cultural life and beautiful nature. Your workplace is located at the main campus of TU Dresden.

Your second supervisor in SECAI will be Carsten Lutz of the Chair of Foundations of Knowledge Representation at Leipzig University. It is planned to have a frequent exchange and regular meetings across the groups.

Prerequisites

• Master’s Degree (or equivalent) in computer science, mathematics, or related fields
• Excellent skills and practical experience in one or more of the following areas:
  • Mathematical logic and universal algebra
  • Knowledge representation and graph databases
  • Data analysis and machine learning
  • Compilers and formal languages
• Expertise in any area optionally related to the topic can be beneficial (e.g., dependable systems, user and web community studies, mathematical statistics, etc.)
• Software development experience is an advantage, especially in a system programming language (e.g. Rust, C++) or in combination with databases or Web data
• Ability to collaborate well in an interdisciplinary environment

Further details on the requirements and application process can be found in SECAI's announcement for open PhD positions in 2024.