Upcoming Meeting in autumn 2026

TimeProgram
14:30-15:15Otto Nyberg (UvA, Amsterdam UMC) - 2-Step Agent: How a Bayesian Decision Maker Learns from AI-Decision Support.

Prognostic predictions from ML models support human decision making in several fields, including high-stakes ones such as healthcare and the judiciary. Yet, we still lack a clear understanding of how decision makers learn from ML-based decision support (ML-DS). In this paper, we introduce a general computational framework, the 2-Step Agent, to capture this process. As a prediction from an ML model contains information about the training data and the causal mechanism that generated it, a prediction can also be used for inference. Our framework models (i) how a prediction for a new observation affects the beliefs of a rational Bayesian agent, and (ii) how this change in beliefs affects the estimation of causal effect, the downstream decision, and the subsequent outcome. In addition to the framework itself, we make three contributions. First, for the multivariate linear-Gaussian setting, we derive a tractable solution for the Bayesian inference problem introduced, i.e. one in which the agent infers from an ML prediction. Second, we experimentally identify conditions under which ML-DS is beneficial. Third, we show that a single misaligned prior belief can be sufficient for ML-DS to lead to worse downstream outcomes compared to no decision support even when the prognostic model is well-specified and the agent is perfectly rational. Hence, even under ideal conditions, ML-DS can do more harm than good.
15:15-15:30Break
15:30-16:15Nadja Rutsch (VU). - Causal Inference in Finite Samples: The Potential of Invalid Adjustments.

The standard advice for estimating causal effects from observational data is to adjust for a valid adjustment set, ideally the asymptotically optimal one. In practice, however, researchers often work with limited sample sizes. There, an invalid adjustment set can lead to a better estimator: accepting a small bias may buy a substantial reduction in variance. We look for the adjustment set that minimizes the mean squared error of the causal effect estimator, taking into account the joint distribution of the variables and the sample size. We present examples in which this MSE-optimal set differs from the asymptotically optimal set, including cases where it is invalid. For linear Gaussian models, we introduce a sample size criterion for comparing adjustment sets, and graphical criteria that use the causal graph to prune the search space. Finally, we propose a data-based procedure that estimates the mean squared error of each adjustment set that remains after graphical pruning, and selects the smallest. In simulations, this procedure outperforms adjustment for the asymptotically optimal set at small sample sizes, making causal inference more practical in such scenarios.
16:30-17:30Drinks at Miss Scarlett