Probabilistic modelling and uncertainty (RT 1)
Understanding and quantifying uncertainty in outputs of Machine Learning (ML) methods is an active research area, because of a lack of proper and theoretically solid approaches to this quite general problem. With the purpose to strengthen predictive precision and to better understand the uncertainties connected to predictions, Integreat will inject knowledge into ML by developing further the probabilistic, Bayesian and post-Bayesian perspectives on ML, through new forms of informative priors, penalties, complex latent structures, model ensembles and network designs.
Structure and Logic (RT 2)
Most real-life data are structured; for example, they are often relational and stored in multiple interlinked database tables, hierarchical as in XML, or in the form of knowledge graphs in formats such as RDF and Property Graphs. Logical languages can be used to systematise and formalise complex domain knowledge, for example, in the form of ontologies, and logical inference can be used to enrich explicit structured data with implicit information that logically follows from the explicit data and formalised domain knowledge. Most common Machine Learning methods today can neither ingest nor produce structured data directly, and they are unable to take into account existing domain knowledge formalised in a logical language.
Scalable Approximation of Models and Algorithms (RT 3)
In Bayesian inference, Markov Chain Monte Carlo (MCMC) algorithms are considered the gold standard but are far too slow for big data and complex models. Recent developments (from superefficient subsampling and quasi-stationary Monte Carlo (MC), to non-reversible processes) are computationally unfeasible in high dimensions. A different approach is to approximate the original model with one which can be efficiently sampled from.
Integrative learning (RT 4)
Integreat will integrate different forms of knowledge into joint analyses, as well as knowledge with a large variety of data types. Knowledge can be expressed by means of stochastic models, differential equations, semantic technologies, graphical representations, and logic-based ontologies and so on: the gap between these approaches is considerable today, but Integreat will study how they can be used in combination.
Transfer learning (RT 5)
Transfer learning has led to ground-breaking progress in natural language processing (NLP), computer vision and many application domains, with deep models pre-trained on massive amounts of unlabelled data. The representation of data learned by unsupervised AI models provides a powerful basis for downstream machine learning tasks. Moreover, pre-training has become the default practice in deep learning to substantially save compute costs and data. Pre-trained encoders also called foundation models compress raw data into compact latent embeddings that preserve rich semantic information. Understanding distribution and structure of these embeddings is an important problem. Meanwhile, sharing these embeddings without adequate protection exposes them to unauthorized use in a wide range of malicious activities. Transfer learning is therefore a central topic in AI.
Causal learning (RT 6)
When humans reason, they typically think in terms of cause and effect, an ability that current machine learning (ML) models generally lack. This may prove to be a fundamental limitation; in fact, the lack of robustness and generalisation in current ML models has recently been linked to their incapacity to identify and exploit causal structures. One of the key characteristics of causal relationships is that they are stable (or even invariant) across domains. Hence, when an ML model is guided to capture causal relationships, it is more likely to generalise well to different contexts and scenarios outside its specific training domain. Moreover, causality-aware models are also thought to be more parsimonious than pure correlation-based models, enjoying an improved interpretability and sample efficiency.
Natural language (RT 7)
Recent breakthroughs in Natural Language Processing (NLP), powered by pre-trained large language models (LLMs), have transformed the landscape of AI. These models have demonstrated impressive capabilities across a wide range of applications by leveraging vast amounts of data and sophisticated neural architectures, showing great promise in enabling natural language as an interface between humans and machines. However, despite their successes, current LLMs exhibit critical limitations: they often lack robustness, produce biased or factually incorrect outputs, struggle with reasoning, and operate as "black boxes" with limited explainability. Furthermore, their development raises concerns about environmental sustainability, due to massive computational demands.
Explainability and fairness (RT 8)
This RT aims to advance methods for explainability and fairness in complex AI systems. Explainability in AI helps us understand how AI makes decisions, ensuring it is transparent and trustworthy. Further, it is crucial to assess the fairness of such decisions where humans are involved and guarantee non-discriminative, equal treatment of all.
Ethics (RT 9)
Integreat has a dedicated strand of research activity analysing ethical aspects of knowledge-driven machine learning (ML). We employ the methods of analytical philosophy, in conjunction with experimental philosophy and conceptual engineering, to examine three interrelated themes which pertain to all other research themes and define the ethical ground of our four objectives.