Cardiac electrophysiology (EP) time series are frequently degraded by various sources of overlapping artifacts that obscure physiologically relevant morphologies. This is particularly limiting for intracardiac electrograms, an opaque yet potentially informative modality, and remains relevant for large-scale surface electrocardiograms (ECGs) denoising. We cast denoising as conditional generation using conditional denoising diffusion probabilistic models (cDDPMs). We further introduce an antithetic-variable (AV) sampling approach that couples reverse diffusion trajectories via additive-inverse Gaussian transition noise at each step. This variance-reduction scheme improves reconstruction and stabilizes uncertainty estimates without increasing the number of reverse-chain evaluations. We evaluate on a ventricular electrogram cohort of over 6000 curated samples from over 50 patients, as well as the QTDB + Noise Stress Test ECG benchmark. The proposed AV-cDDPM suppresses baseline wander, powerline contamination, spikes, and EP-derived residual artifacts while preserving depolarization–repolarization morphology in monophasic action potentials (MAPs) and ECG waveforms. Additionally, AV-cDDPM achieves state-of-the-art reconstruction on MAPs (root mean square error 3.32 x 10−3, Pearson correlation coefficient 0.978) and improves ECG denoising (cosine similarity 0.926) versus crude Monte Carlo sampling and competitive baselines. Importantly, MAP denoising translates to improved recovery of repolarization markers and time-resolved uncertainty maps aligned with physiological transitions with direct potential of clinical deployment. Overall, variance-reduced conditional diffusion enables uncertainty-aware, compute-efficient denoising both for therapeutically-relevant intracardiac signals and ubiquitous ECGs supporting clinical reliability with reduced manual curation burden.

Samuel Ruipérez-Campillo, Pablo Blasco-Fernández, Moritz Rau, Prasanth Ganesan, Sabyasachi Bandyopadhyay, Charles Sillett, Lukas P Arts, Esteban Peralta, Albert J Rogers, Fleur VY Tjong, Sanjiv M Narayan, Julia E Vogt

IEEE Journal of Biomedical and Health Informatics

22.06.2026

LinkDOI

Objective:Accurate pre-procedural identification of atrial flutter (AFL) mechanisms can streamline mapping and indirectly inform ablation strategy, yet surface-electrocardiogram (ECG) criteria remain unreliable and circuit definition is typically confirmed invasively.Methods and procedures:We analyzed 97 consecutive patients undergoing electrophysiological (EP) study with simultaneous 12-lead ECG and EP-verified AFL subtype; adenosine-induced atrioventricular AV block enabled extraction of clean atrial segments. We reconstructed atrial vectorcardiograms (VCGs) and engineered interpretable descriptors of loop morphology and kinematics, including archetype cosine correlation, geometric complexity, and velocity-based slow-occupancy indices, then fused these with clinical variables in an explainable tree-ensemble model evaluated with nested cross-validation.Results:VCG loops exhibited subtype-specific archetypes (within-class correlation: 0.832±0.129 CCCW, 0.874±0.154 CCW, 0.647±0.127 PMCCW, 0.667±0.159 PMCW; C: common; PM: perimitral; CW: clockwise; CCW: counter-CW). On the test set, the multimodal Random-Forest improved discrimination over VCG-only and clinical-only baselines, achieving AUROC of 0.870 (CCCW), 0.900 (CCW), 0.840 (PMCCW), and 0.790 (PMCW), with high sensitivity for common AFL (0.833 and 0.929) and very high specificity for PMCW (0.988).Conclusion:This interpretable framework provides a practical route to non-invasive, mechanism-oriented AFL stratification to support targeted mapping and more efficient ablation planning. Future work will focus on multicenter prospective validation and robust atrial-signal extraction without adenosine to broaden routine applicability.

Samuel Ruipérez-Campillo, David Hernando, Elisa Ramírez, Sergio Castrejón, Cecilia Zapata, Carlos Rodríguez-Carneiro, Julia E. Vogt, José L. Merino, Francisco Castells, José Millet

IEEE Journal of Translational Engineering in Health and Medicine

19.05.2026

LinkDOI

Abstract Foundation models achieve state-of-the-art performance across different tasks, but their size and computational demands raise concerns about accessibility and sustainability. Existing efficiency methods often require additional retraining or fine-tuning, limiting their practicality. Recent findings suggest that deep neural networks exhibit internal representation similarities. While such similarities across different models have been exploited for enabling techniques such as model stitching and merging, intra-network redundancy remains underexplored as a source for efficiency gains. In this paper, we introduce Transformer Optimization using Adaptive and Simple Transformations (TOAST), a framework that exploits these redundancies to approximate entire transformer blocks with lightweight closed-form mappings, such as linear transformations or even the identity function, without any additional training. Across state-of-the-art pretrained vision models (e.g., ViT, DINOv2, DeiT) and datasets ranging from MNIST to ImageNet-1k, TOAST reduces parameters and computation while preserving, and in some cases improving, downstream performance. These results show that large portions of transformer depth can be replaced by trivial functions, opening a new perspective on efficient foundation models.

Irene Cannistraci, Simone Antonelli, Emanuele Palumbo, Thomas M. Sutter, Emanuele Rodolà, Bastian Rieck^†, Julia E. Vogt, ^†
† denotes shared last authorship

Journal Transactions on Machine Learning Research (TMLR)

13.05.2026

LinkCode

Concept Bottleneck Models (CBMs) are interpretable models that predict the target variable through high-level human-understandable concepts, allowing users to intervene on mispredicted concepts to adjust the final output. While recent work has shown that modeling dependencies between concepts can improve CBM performance, especially under interventions, such approaches typically require retraining the entire model, which may be infeasible when access to the original data or compute is limited. In this paper, we introduce Post-hoc Stochastic Concept Bottleneck Models (PSCBMs), a lightweight method that augments any pre-trained CBM with a multivariate normal distribution over concepts by adding only a small covariance-prediction module, without retraining the backbone model. We propose two training strategies and show on real-world data that PSCBMs consistently match or improve both concept and target accuracy over standard CBMs at test time. Furthermore, we show that due to the modeling of concept dependencies, PSCBMs perform much better than CBMs under interventions, while remaining far more efficient than retraining a similar stochastic model from scratch.

Wiktor Jan Hoffmann, Sonia Laguna, Moritz Vandenhirtz, Emanuele Palumbo, Julia E Vogt

International Conference on Learning Representations (ICLR) Workshop on Trustworthy AI

27.04.2026

Link

Machine unlearning aims to remove the influence of specific data from trained models while preserving general utility. Existing approximate unlearning methods often rely on performance-degradation heuristics, such as loss maximization or random labeling. However, these signals can be poorly conditioned, leading to unstable optimization and harming the model's generalization. We argue that unlearning should instead prioritize distributional indistinguishability, aligning the model's behavior on forget data with its behavior on truly unseen data. Motivated by this, we propose Reference-Guided Unlearning (ReGUn), a framework that leverages a disjoint held-out dataset to provide a principled, class-conditioned reference for distillation. We demonstrate across various model architectures, natural image datasets, and varying forget fractions that ReGUn consistently outperforms standard approximate baselines, achieving a superior forgetting-utility trade-off.

Jonas Mirlach, Sonia Laguna, Julia E Vogt

International Conference on Learning Representations (ICLR) Workshop on Agents in the Wild

27.04.2026

Link

Background Gait impairment is a hallmark motor deficit of Parkinson’s disease (PD) and represents an important, yet insufficiently understood, target of subthalamic deep brain stimulation (DBS). Although DBS can improve several motor symptoms, identifying robust and physiologically meaningful gait biomarkers that capture both disease-related deficits and stimulation-induced improvements remains a major challenge. In particular, conventional mean-based gait metrics often fail to fully characterize pathological gait or treatment responsiveness. Methods We analyzed 35 spatiotemporal gait parameters obtained during continuous walking from individuals with PD assessed before and after subthalamic DBS, alongside age-matched healthy controls. Multiple machine learning classifiers were evaluated to discriminate between groups, with extreme gradient boosting (XGBoost) achieving the best performance. To enhance interpretability and reduce redundancy among correlated parameters, grouped SHapley Additive exPlanations (SHAP) were applied to rank feature importance and guide feature selection. Results Feature selection consistently highlighted step width variability, step width asymmetry, bilateral interlimb coordination, and the anteroposterior margin of stability as the most discriminative parameters. A compact set of five overlapping features after selection not only reliably distinguished PD gait from healthy controls but also demonstrated a shift toward healthy ranges following DBS. Importantly, these selected features outperformed conventional mean-based metrics in capturing both pathological gait characteristics and treatment-related changes. Discussion Our findings demonstrate that explainable artificial intelligence approaches can identify physiologically grounded gait features that may serve as candidate markers of both PD severity and DBS responsiveness. By emphasizing variability,

Zhongke Mei, Alain Ryser, Gianluca Amprimo, Jinhao Wang, Julia Vogt, Deepak K Ravi

Journal of NeuroEngineering and Rehabilitation

27.04.2026

LinkDOI

Machine unlearning is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training samples. Despite this, most existing methods tackle the problem purely from a post-hoc perspective. They attempt to erase the influence of targeted training samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We propose unlearning by design, a novel paradigm in which models are directly trained to support forgetting as an inherent capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling direct zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.

Sonia Laguna, Jorge da Silva Gonçalves, Moritz Vandenhirtz, Alain Ryser, Irene Cannistraci^†, Julia E Vogt^†
† denotes shared last authorship

Oral at ICLR Workshop TTU

26.04.2026

Link

Selecting compact and informative gene subsets from single-cell transcriptomic data is essential for biomarker discovery, improving interpretability, and cost-effective profiling. However, most existing feature selection approaches either operate as multi-stage pipelines or rely on post hoc feature attribution, making selection and prediction weakly coupled. However, most existing feature selection approaches either operate as multi-stage pipelines or rely on post hoc feature attribution, making selection and prediction weakly coupled. In this work, we present YOTO (you only train once), an end-to-end framework that jointly identifies discrete gene subsets and performs prediction within a single differentiable architecture. In our model, the prediction task directly guides which genes are selected, while the learned subsets, in turn, shape the predictive representation. This closed feedback loop enables the model to iteratively refine both what it selects and how it predicts during training. Unlike existing approaches, YOTO enforces sparsity so that only the selected genes contribute to infer- ence, eliminating the need to train additional downstream classifiers. Through a multi-task learning design, the model learns shared representations across related objectives, allowing different tasks to inform one another, and discovering gene subsets that generalize across tasks without additional training steps. We evaluate YOTO on two representative single-cell RNA-seq datasets, showing that it consistently outperforms state-of-the-art baselines. These results demonstrate that sparse, end-to-end, multi-task gene subset selection improves predictive performance and yields compact and meaningful gene subsets, advancing biomarker discovery and single-cell analysis.

Daphné Chopard, Jorge Gonçalves, Irene Cannistraci, Thomas M. Sutter^†, Julia E Vogt^†
† denotes shared last authorship

Transactions on Machine Learning Research

24.04.2026

LinkCode

EasyRead pictograms are simple, visually clear images that represent specific concepts and support comprehension for people with intellectual disabilities, low literacy, or language barriers. The large-scale production of EasyRead content has traditionally been constrained by the cost and expertise required to manually design pictograms. In contrast, automatic generation of such images could significantly reduce production time and cost, enabling broader accessibility across digital and printed materials. However, modern diffusion-based image generation models tend to produce outputs that exhibit excessive visual detail and lack stylistic stability across random seeds, limiting their suitability for clear and consistent pictogram generation. This challenge highlights the need for methods specifically tailored to accessibility-oriented visual content. In this work, we present a unified pipeline for generating EasyRead pictograms by fine-tuning a Stable Diffusion model using LoRA adapters on a curated corpus that combines augmented samples from multiple pictogram datasets. Since EasyRead pictograms lack a unified formal definition, we introduce an EasyRead score to benchmark pictogram quality and consistency. Our results demonstrate that diffusion models can be effectively steered toward producing coherent EasyRead-style images, indicating that generative models can serve as practical tools for scalable and accessible pictogram production.

Nicolas Dickenmann^*, Yanis Merzouki^*, Sonia Laguna, Thy Nowak-Tran, Emanuele Palumbo, Julia E Vogt, Gerda Binder
^* denotes shared first authorship

ACM Conference on Human Factors in Computing Systems (CHI)

13.04.2026

Link

Building robust medical machine learning systems requires pretraining strategies that exploit the intrinsic structure present in clinical data. We introduce Multiview Masked Autoencoder (MVMAE), a self-supervised framework that leverages the natural multi-view organization of radiology studies to learn view-invariant and disease-relevant representations. MVMAE combines masked image reconstruction with cross-view alignment, transforming clinical redundancy across projections into a powerful self-supervisory signal. We further extend this approach with MVMAE-V2T, which incorporates radiology reports as an auxiliary text-based learning signal to enhance semantic grounding while preserving fully vision-based inference. Evaluated on a downstream disease classification task on three large-scale public datasets, MIMIC-CXR, CheXpert, and PadChest, MVMAE consistently outperforms supervised and vision-language baselines. Furthermore, MVMAE-V2T provides additional gains, particularly in low-label regimes where structured textual supervision is most beneficial. Together, these results establish the importance of structural and textual supervision as complementary paths toward scalable, clinically grounded medical foundation models.

Sonia Laguna^*, Andrea Agostini^*, Alain Ryser, Samuel Ruiperez-Campillo, Irene Cannistraci, Moritz Vandenhirtz, Stephan Mandt, Nicolas Deperrois, Farhad Nooralahzadeh, Michael Krauthammer, Thomas M Sutter^†, Julia E Vogt^†
* denotes shared first authorship, ^† denotes shared last authorship

Transactions on Machine Learning Research (TMLR)

22.03.2026

Link

Reducing electrophysiological (EP) signal noise is essential for diagnosis, mapping, and ablation procedures in patients with arrhythmias or conditions such as cardiomyopathies. However, traditional approaches have been suboptimal due to the varied sources of noise. We hypothesized that variational autoencoders (VAEs) can learn key components of ’clean’ electrophysiological signals by creating robust internal representations, thereby enabling automatic denoising of diverse noise in clinical recordings. We set out to apply a β-VAE model to a dataset of 5706 intra-ventricular monophasic action potential (MAP) signals, selected because their morphology is verifiable and measurable against a reference, from 42 patients with ischemic cardiomyopathy at risk for sudden death. We designed a noise library, and implemented baselines based on state-of-the-art clinical filtering techniques. The proposed β-VAE model was assessed for various noise types, including challenging non-stationary real EP noise. Comprehensive evaluation using general metrics and clinical action potential duration labels by domain experts revealed that our β-VAE outperformed current state-of-the-art filters in denoising efficacy, with key physiological information encoded in the reconstruction. We performed a sensitivity analysis that confirmed the robustness of the β-VAE model to increasing noise levels. These results demonstrate the ability of our model to denoise various sources, including those of time-varying nature. The application to well-studied MAPs verifies that clinically meaningful features were reconstructed in the EP context. This work enhances traditional signal processing approaches to ensure ’clean’ electrical signals, and may have promising applications for diagnosis, tracking therapy and prognostication in patients with EP disorders in real-world clinical environments.

Samuel Ruipérez-Campillo, Alain Ryser, Thomas M Sutter, Brototo Deb, Ruibin Feng, Prasanth Ganesan, Kelly A Brennan, Albert J Rogers, Maarten ZH Kolk, Fleur VY Tjong, Sanjiv M Narayan^†, Julia E Vogt^†
† denotes shared last authorship

Expert Systems with Applications

01.03.2026

LinkDOI

Recent advances in vision–language models (VLMs) have improved Chest X-ray (CXR) interpretation in multiple aspects. However, many medical VLMs rely solely on supervised fine-tuning (SFT), which optimizes next-token prediction without evaluating answer quality. In contrast, reinforcement learning (RL) can incorporate task-specific feedback, and its combination with explicit intermediate reasoning (``thinking'') has demonstrated substantial gains on verifiable math and coding tasks. To investigate the effects of RL and thinking in a CXR VLM, we perform large-scale SFT on CXR data to build an updated RadVLM based on Qwen3-VL, followed by a cold-start SFT stage that equips the model with basic thinking ability. We then apply Group Relative Policy Optimization (GRPO) with clinically grounded, task-specific rewards for report generation and visual grounding, and run matched RL experiments on both domain-specific and general-domain Qwen3-VL variants, with and without thinking. Across these settings, we find that while strong SFT remains crucial for high base performance, RL provides additional gains on both tasks, whereas explicit thinking does not appear to further improve results. Under a unified evaluation pipeline, the RL-optimized RadVLM models outperform their baseline counterparts and reach state-of-the-art performance on both report generation and grounding, highlighting clinically aligned RL as a powerful complement to SFT for medical VLMs.

Benjamin Gundersen, Nicolas Deperrois, Samuel Ruipérez-Campillo, Thomas M. Sutter, Julia E. Vogt, Michael Moor, Farhad Nooralahzadeh, Michael Krauthammer

MIDL 2026

14.02.2026

Link

Intracardiac electrophysiological (EP) signals are frequently contaminated by diverse noise sources, posing a major obstacle to accurate arrhythmia diagnosis. We hypothesized that a physics-inspired conditional denoising diffusion probabilistic model (cDDPM) could outperform both classical filters and variational autoencoders by preserving subtle morphological features. Using 5706 monophasic action potentials from 42 patients, we introduced a range of simulated and real EP noise, then trained the cDDPM in an iterative process analogous to Brownian motion. The proposed model achieved superior performance across RMSE, PCC, and PSNR metrics, confirming its robustness against complex noise while maintaining essential signal fidelity. These findings suggest that diffusion-based methods can significantly enhance the clinical utility of EP signals for arrhythmia management and intervention. Clinical Relevance— We propose a denoising diffusion probabilistic model to reconstruct intracardiac signals in the presence of complex noise, which holds the potential to enhance diagnostic accuracy in EP procedures and inform more targeted treatment strategies.

Samuel Ruipérez-Campillo, Moritz Rau, Prasanth Ganesan, Kelly A Brennan, Ruibin Feng, Sabyasachi Bandyopadhyay, Albert J Rogers, Sanjiv M Narayan, Julia E Vogt

IEEE Engineering in Medicine & Biology Society (47th EMBC, 2025)

03.12.2025

LinkDOI

Background: Timely and accurate detection of arrhythmias from electrocardiograms (ECGs) is crucial for improving patient outcomes. While artificial intelligence (AI)-based ECG classification has shown promising results, limited transparency and interpretability often impede clinical adoption. Methods: We present ECG-XPLAIM, a novel deep learning model dedicated to ECG classification that employs a one-dimensional inception-style convolutional architecture to capture local waveform features (e.g., waves and intervals) and global rhythm patterns. To enhance interpretability, we integrate Grad-CAM visualization, highlighting key waveform segments that drive the model's predictions. ECG-XPLAIM was trained on the MIMIC-IV dataset and externally validated on PTB-XL for multiple arrhythmias, including atrial fibrillation (AFib), sinus tachycardia (STach), conduction disturbances (RBBB, LBBB, LAFB), long QT (LQT), Wolff-Parkinson-White (WPW) pattern, and paced rhythm detection. We evaluated performance using sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC), and benchmarked against a simplified convolutional neural network, a two-layer gated recurrent unit (GRU), and an external, pre-trained, ResNet-based model. Results: Internally (MIMIC-IV), ECG-XPLAIM achieved high diagnostic performance (sensitivity, specificity, AUROC > 0.9) across most tasks. External evaluation (PTB-XL) confirmed generalizability, with metric values exceeding 0.95 for AFib and STach. For conduction disturbances, macro-averaged sensitivity reached 0.90, specificity 0.95, and AUROC 0.98. Performance for LQT, WPW, and pacing rhythm detection was 0.691/0.864/0.878, 0.773/0.973/0.895, and 0.96/0.988/0.993 (sensitivity/specificity/AUROC), respectively. Compared to baseline models, ECG-XPLAIM offered superior performance across most tests, and improved sensitivity over the external ResNet-based model, albeit at the cost of specificity. Grad-CAM revealed physiologically relevant ECG segments influencing predictions and highlighted patterns of potential misclassification. Conclusion: ECG-XPLAIM combines high diagnostic performance with interpretability, addressing a key limitation in AI-driven ECG analysis. The open-source release of ECG-XPLAIM's architecture and pre-trained weights encourages broader adoption, external validation, and further refinement for diverse clinical applications.

Panteleimon Pantelidis^*, Samuel Ruipérez-Campillo^*, Julia E Vogt, Alexios Antonopoulos, Ioannis Gialamas, George E Zakynthinos, Michael Spartalis, Polychronis Dilaveris, Jose Millet, Panagiotis Papapetrou, Theodore G Papaioannou, Evangelos Oikonomou, Gerasimos Siasos
^* denotes shared first authorship

Frontiers in Cardiovascular Medicine

16.10.2025

LinkDOI

We present RadVLM, a compact (7B) multitask conversational foundation model designed for CXR interpretation. Its development relies on the curation of a large-scale instruction dataset comprising over 1 million image-instruction pairs containing both single-turn tasks - such as report generation, abnormality classification, and visual grounding - and multi-turn, multi-task conversational interactions. Our experiments show that RadVLM, fine-tuned on this instruction dataset, achieves state-of-the-art performance in conversational capabilities and visual grounding while remaining competitive in other radiology tasks (report generation, classification). Ablation studies further highlight the benefit of joint training across multiple tasks, particularly for scenarios with limited annotated data. Together, these findings highlight the potential of the RadVLM model as a clinically relevant AI assistant, providing structured CXR interpretation and conversational capabilities to support more effective and accessible diagnostic workflows.

Nicolas Deperrois, Hidetoshi Matsuo, Samuel Ruipérez-Campillo, Moritz Vandenhirtz, Sonia Laguna, Alain Ryser, Koji Fujimoto, Mizuho Nishio, Thomas Sutter, Julia Vogt, Jonas Kluckert, Thomas Frauenfelder, Christian Bluethgen, Farhad Nooralahzadeh, Michael Krauthammer

Physionet

08.10.2025

LinkDOI

Generative modeling and clustering are conventionally distinct tasks in machine learning. Variational Autoencoders (VAEs) have been widely explored for their ability to integrate both, providing a framework for generative clustering. However, while VAEs can learn meaningful cluster representations in latent space, they often struggle to generate high-quality samples. This paper addresses this problem by introducing TreeDiffusion, a deep generative model that conditions diffusion models on learned latent hierarchical cluster representations from a VAE to obtain high-quality, cluster-specific generations. Our approach consists of two steps: first, a VAE-based clustering model learns a hierarchical latent representation of the data. Second, a cluster-aware diffusion model generates realistic images conditioned on the learned hierarchical structure. We systematically compare the generative capabilities of our approach with those of alternative conditioning strategies. Empirically, we demonstrate that conditioning diffusion models on hierarchical cluster representations improves the generative performance on real-world datasets compared to other approaches. Moreover, a key strength of our method lies in its ability to generate images that are both representative and specific to each cluster, enabling more detailed visualization of the learned latent structure. Our approach addresses the generative limitations of VAE-based clustering approaches by leveraging their learned structure, thereby advancing the field of generative clustering.

Jorge Silva Gonçalves, Laura Manduchi, Moritz Vandenhirtz, Julia E Vogt

Joint European Conference on Machine Learning and Knowledge Discovery in Databases

04.10.2025

LinkDOICode

We release the RadVLM instruction dataset, a large-scale resource used to train the RadVLM model on diverse radiology tasks. The dataset contains 1,115,021 image–instruction pairs spanning five task families: (i) report generation from frontal CXRs using filtered Findings/Impression text; (ii) abnormality classification for the standard 14 CheXpert labels; (iii) anatomy grounding; (iv) abnormality detection and grounding; and (v) phrase grounding from report sentences. To support interactive use, we include ~89k LLM-generated multi-turn, multi-task conversations (~3k with spatial grounding) derived from image-linked attributes (reports, labels, boxes). Creation involved curating datasets from public sources, excluding lateral views, removing prior-study references and other non-image context from reports, fusing multi-reader annotations, and harmonizing label and coordinate formats. The resource is intended for training CXR assistants across diverse radiology tasks and within a conversational format.

Nicolas Deperrois, Hidetoshi Matsuo, Samuel Ruipérez-Campillo, Moritz Vandenhirtz, Sonia Laguna, Alain Ryser, Koji Fujimoto, Mizuho Nishio, Thomas Sutter, Julia Vogt, Jonas Kluckert, Thomas Frauenfelder, Christian Bluethgen, Farhad Nooralahzadeh, Michael Krauthammer

Physionet

25.09.2025

LinkDOI

Background In infants, pulmonary hypertension (PH) increases morbidity and mortality. Echocardiography, though standard, is time- and expertise-demanding. We propose a deep learning approach for automated PH detection using standard echocardiography videos, validated by the systolic eccentricity index (EIs). Methods The training and validation set comprised 975 videos and the held-out set 378 videos, including five echocardiographic standard views from infants aged 3–90 days, taken between 2018–2021 and 2021–2022, respectively. Echocardiograms were labeled as PH (EIs < 0.82) and healthy (EIs ≥ 0.87). After preprocessing and random segmentation of all videos into 13.530 frames, spatial and spatio-temporal convolutional neural network architectures were used for training of a PH prediction model and gradient-weighted class activation mapping for explainability. Results The best single-view performance was achieved using parasternal short axis view (AUROC spatial and spatio-temporal: 0.91 and 0.94 in validation set, 0.93 and 0.88 in held-out set, respectively). Combination of three standard views improved accuracy with AUROC 0.96 and 0.90 in validation (spatio-temporal) and held-out set (spatial), respectively. Saliency maps revealed model focus on clinically relevant regions, including interventricular septum and left atrial filling. Conclusions The presented deep learning model for automated detection of PH in neonates shows high accuracy, explainability, and reproducibility.

Holger Michel, Ece Ozkan, Kieran Chin-Cheong, Anna Badura, Verena Lehnerer, Stephan Gerling, Julia E. Vogt, Sven Wellmann

Pediatric Research

24.09.2025

LinkDOI

Vision foundation models (FMs) are accelerating the development of digital pathology algorithms and transforming biomedical research. These models learn, in a self-supervised manner, to represent histological features in highly heterogeneous tiles extracted from whole-slide images (WSIs) of real-world patient samples. The performance of these FMs is significantly influenced by the size, diversity, and balance of the pre-training data. Yet, data selection has been primarily guided by expert knowledge at the WSI level, focusing on factors such as disease classification and tissue types, while largely overlooking the granular details available at the tile level. In this paper, we investigate the potential of unsupervised automatic data curation at the tile-level, taking into account 350 million tiles. Specifically, we apply hierarchical clustering trees to pre-extracted tile embeddings, allowing us to sample balanced datasets uniformly across the embedding space of the pretrained FM. We further show that these datasets are subject to a trade-off between size and balance, potentially compromising the quality of representations learned by FMs. We propose tailored batch sampling strategies to mitigate this effect. We demonstrate the effectiveness of our method through improved performance on a diverse range of clinically relevant downstream tasks.

Boqi Chen, Cédric Vincent-Cuaz, Lydia A Schoenpflug, Manuel Madeira, Lisa Fournier, Vaishnavi Subramanian, Sonali Andani, Samuel Ruipérez-Campillo, Julia E Vogt, Raphaëlle Luisier, Dorina Thanou, Viktor H Koelzer, Pascal Frossard, Gabriele Campanella, Gunnar Rätsch

MICCAI 2025

19.09.2025

LinkDOI

Anomaly detection focuses on identifying samples that deviate from the norm. Discovering informative representations of normal samples is crucial to detecting anomalies effectively. Recent self-supervised methods have successfully learned such representations by employing prior knowledge about anomalies to create synthetic outliers during training. However, we often do not know what to expect from unseen data in specialized real-world applications. In this work, we address this limitation with our new approach, Con2, which leverages prior knowledge about symmetries in normal samples to observe the data in different contexts. Con2 consists of two parts: Context Contrasting clusters representations according to their context, while Content Alignment encourages the model to capture semantic information by aligning the positions of normal samples across clusters. The resulting representation space allows us to detect anomalies as outliers of the learned context clusters. We demonstrate the benefit of this approach in extensive experiments on specialized medical datasets, outperforming competitive baselines based on self-supervised learning and pretrained models and presenting competitive performance on natural imaging benchmarks.

Alain Ryser, Thomas M. Sutter, Alexander Marx, Julia E. Vogt

Transactions on Machine Learning Research

16.09.2025

Link

General movements (GMs) are spontaneous, coordinated body movements in infants that offer valuable insights into the developing nervous system. Assessed through the Prechtl GM Assessment (GMA), GMs are reliable predictors for neurodevelopmental disorders. However, GMA requires specifically trained clinicians, who are limited in number. To scale up newborn screening, there is a need for an algorithm that can automatically classify GMs from infant video recordings. This data poses challenges, including variability in recording length, device type, and setting, with each video coarsely annotated for overall movement quality. In this work, we introduce a tool for extracting features from these recordings and explore various machine learning techniques for automated GM classification.

Daphne Chopard^*, Sonia Laguna^*, Kieran Chin-Cheong^*, Annika Dietz, Anna Badura, Sven Wellmann, Julia E Vogt
^* denotes shared first authorship

Proceedings of Machine Learning Research - Machine Learning for Healthcare 2025, previous version in ICLR 2025 (Best Paper Award - Oral) Workshop AI4CHL

15.08.2025

LinkCode

Accurate extraction of phenotypic information from clinical narratives is essential in diagnostic medicine, yet mapping free-text reports to structured Human Phenotype Ontology (HPO) terms remains challenging. While encoder-only transformer models and small decoder-only generative models are attractive for clinical deployment due to their efficiency and low resource requirements, the former often fail to capture the rich context of clinical texts, and the latter struggle to process lengthy reports effectively. In contrast, larger language models excel at contextual understanding but are impractical for clinical use due to their size, propensity to hallucinate, and privacy concerns associated with non-local inference. To overcome these challenges, we introduce PhenoRAG, a novel retrieval-augmented generation framework that leverages a synthetic database of contextually enriched sentences to augment a lightweight decoder-only model for accurate zero-shot phenotype identification. We demonstrate the capacity of PhenoRAG to capture nuanced contextual clues by 1) evaluating its ability to perform two clinically relevant tasks-guide rare disease diagnosis and facilitate urinary tract infection detection-and 2) validating its performance on a synthetic dataset designed to mimic the challenges of real clinical narratives. Experimental results demonstrate that our lightweight PhenoRAG framework achieves a higher F1-score than both encoder-only transformers and standalone small language models, driven primarily by its high recall. These findings underscore the potential of PhenoRAG as a ready-to-use clinical tool for phenotype identification.

Marc Berndt, Andrea Agostini, Beatrice Stocker, Maria Padrutt, Silvio Daniel Brugger, D Sean Froese, Daphné Chopard^†, Julia E Vogt^†
† denotes shared last authorship

Proceedings of the 10th Machine Learning for Healthcare Conference

14.08.2025

Link

Building generalizable medical AI systems requires pretraining strategies that are data-efficient and domain-aware. Unlike internet-scale corpora, clinical datasets such as MIMIC-CXR offer limited image counts and scarce annotations, but exhibit rich internal structure through multi-view imaging. We propose a self-supervised framework that leverages the inherent structure of medical datasets. Specifically, we treat paired chest X-rays (i.e., frontal and lateral views) as natural positive pairs, learning to reconstruct each view from sparse patches while aligning their latent embeddings. Our method requires no textual supervision and produces informative representations. Evaluated on MIMIC-CXR, we show strong performance compared to supervised objectives and baselines being trained without leveraging structure. This work provides a lightweight, modality-agnostic blueprint for domain-specific pretraining where data is structured but scarce.

Andrea Agostini^*, Sonia Laguna^*, Alain Ryser^*, Samuel Ruipérez-Campillo^*, Moritz Vandenhirtz, Nicolas Deperrois, Farhad Nooralahzadeh, Michael Krauthammer, Thomas M Sutter^†, Julia E Vogt^†
* denotes shared first authorship, ^† denotes shared last authorship

International Conference of Machine Learning (ICML) 2025 Workshop on FM4LS

15.07.2025

Link

We introduce Concept Bottleneck Reward Models (CB-RM), a reward modeling framework that enables interpretable preference learning through selective concept annotation. Unlike standard RLHF methods that rely on opaque reward functions, CB-RM decomposes reward prediction into human-interpretable concepts. To make this framework efficient in low-supervision settings, we formalize an active learning strategy that dynamically acquires the most informative concept labels. We propose an acquisition function based on Expected Information Gain and show that it significantly accelerates concept learning without compromising preference accuracy. Evaluated on the UltraFeedback dataset, our method outperforms baselines in interpretability and sample efficiency, marking a step towards more transparent, auditable, and human-aligned reward models.

Sonia Laguna, Katarzyna Kobalczyk, Julia E Vogt, Mihaela Van der Schaar

International Conference on Machine Learning (ICML) 2025 Workshop on PRAL

12.07.2025

LinkCode

Understanding the decision-making process of machine learning models provides valuable insights into the task, the data, and the reasons behind a model’s failures. In this work, we propose a method that performs inherently interpretable predictions through the instance-wise sparsification of input images. To align the sparsification with human perception, we learn the masking in the space of semantically meaningful pixel regions rather than on pixel-level. Additionally, we introduce an explicit way to dynamically determine the required level of sparsity for each instance. We show empirically on semi-synthetic and natural image datasets that our inherently interpretable classifier produces more meaningful, human-understandable predictions than state-ofthe-art benchmarks.

Moritz Vandenhirtz, Julia E. Vogt

09.07.2025

Link

Concept Bottleneck Models (CBMs) aim to enhance interpretability by structuring predictions around human-understandable concepts. However, unintended information leakage, where predictive signals bypass the concept bottleneck, compromises their transparency. This paper introduces an information-theoretic measure to quantify leakage in CBMs, capturing the extent to which concept embeddings encode additional, unintended information beyond the specified concepts. We validate the measure through controlled synthetic experiments, demonstrating its effectiveness in detecting leakage trends across various configurations. Our findings highlight that feature and concept dimensionality significantly influence leakage, and that classifier choice impacts measurement stability, with XGBoost emerging as the most reliable estimator. Additionally, preliminary investigations indicate that the measure exhibits the anticipated behavior when applied to soft joint CBMs, suggesting its reliability in leakage quantification beyond fully synthetic settings. While this study rigorously evaluates the measure in controlled synthetic experiments, future work can extend its application to real-world datasets.

Mikael Makonnen, Moritz Vandenhirtz, Sonia Laguna, Julia E Vogt

International Conference on Learning Representations (ICLR) 2025 Workshop: XAI4Science

15.04.2025

Link

The dead-in-bed syndrome describes the sudden and unexplained death of young individuals with Type 1 Diabetes (T1D) without prior long-term complications. One leading hypothesis attributes this phenomenon to nocturnal hypoglycemia (NH), a dangerous drop in blood glucose during sleep. This study aims to improve NH prediction in children with T1D by leveraging physiological data and machine learning (ML) techniques. We analyze an in-house dataset collected from 16 children with T1D, integrating physiological metrics from wearable sensors. We explore predictive performance through feature engineering, model selection, architectures, and oversampling. To address data limitations, we apply transfer learning from a publicly available adult dataset. Our results achieve an AUROC of 0.75 +- 0.21 on the in-house dataset, further improving to 0.78 +- 0.05 with transfer learning. This research moves beyond glucose-only predictions by incorporating physiological parameters, showcasing the potential of ML to enhance NH detection and improve clinical decision-making for pediatric diabetes management.

Marco Voegeli^*, Sonia Laguna^*, Heike Leutheuser, Marc Pfister, Marie-Anne Burckhardt, Julia E Vogt
^* denotes shared first authorship

International Conference on Learning Representations (ICLR) 2025 Workshop AI4CHL

14.04.2025

Link

Pulmonary hypertension (PH) in newborns is a critical condition characterized by elevated pressure in the pulmonary arteries, leading to right ventricular strain and heart failure. While right heart catheterization (RHC) is the diagnostic gold standard, echocardiography is preferred due to its non-invasive nature, safety, and accessibility. However, its accuracy highly depends on the operator, making PH assessment subjective. While automated detection methods have been explored, most models focus on adults and rely on single-view echocardiographic frames, limiting their performance in diagnosing PH in newborns. While multi-view echocardiography has shown promise in improving PH assessment, existing models struggle with generalizability. In this work, we employ a multi-view variational autoencoder (VAE) for PH prediction using echocardiographic videos. By leveraging the VAE framework, our model captures complex latent representations, improving feature extraction and robustness. We compare its performance against single-view and supervised learning approaches. Our results show improved generalization and classification accuracy, highlighting the effectiveness of multi-view learning for robust PH assessment in newborns.

Lucas Erlacher^*, Samuel Ruipérez-Campillo^*, Holger Michel, Sven Wellmann, Thomas M. Sutter, Ece Özkan Elsen^†, Julia E. Vogt^†
* denotes shared first authorship, ^† denotes shared last authorship

ICLR 2025 - Workshop on AI for Children

06.03.2025

LinkDOI

Effective blood glucose forecasting is crucial for detecting events such as hypo-or hyperglycemia in people with diabetes, yet remains challenging in domains with only small, heterogeneous datasets, such as in the pediatric field. We present GluTFT, a novel transfer learning approach that allows leveraging models pretrained on publicly available adult diabetes datasets for pediatric glucose forecasting. We systematically evaluate multiple transfer learning strategies, including zero-shot prediction and finetuning across the entire dataset as well as specific subgroups of participants. Our extensive experiments reveal that GluTFT excels on the pretraining datasets and significantly outperforms baseline methods when fine-tuned. To validate the clinical relevance of our approach, we evaluate Parkes Error Grids, demonstrating the quality of GluTFT's blood glucose forecasts and its potential for enhancing clinical decision-making for pediatric diabetes.

Alain Ryser^*, Chuhao Feng^*, Tobias Scheithauer, Marc Pfister, Marie-Anne Burckhardt, Sara Bachmann, Alexander Marx^†, Julia E. Vogt^†
* denotes shared first authorship, ^† denotes shared last authorship

Proceedings of the 4th Machine Learning for Health Symposium, PMLR 259:839-860

15.12.2024

Link

Concept Bottleneck Models (CBMs) have emerged as a promising interpretable method whose final prediction is based on intermediate, human-understandable concepts rather than the raw input. Through time-consuming manual interventions, a user can correct wrongly predicted concept values to enhance the model's downstream performance. We propose Stochastic Concept Bottleneck Models (SCBMs), a novel approach that models concept dependencies. In SCBMs, a single-concept intervention affects all correlated concepts, thereby improving intervention effectiveness. Unlike previous approaches that model the concept relations via an autoregressive structure, we introduce an explicit, distributional parameterization that allows SCBMs to retain the CBMs' efficient training and inference procedure. Additionally, we leverage the parameterization to derive an effective intervention strategy based on the confidence region. We show empirically on synthetic tabular and natural image datasets that our approach improves intervention effectiveness significantly. Notably, we showcase the versatility and usability of SCBMs by examining a setting with CLIP-inferred concepts, alleviating the need for manual concept annotations.

Moritz Vandenhirtz^*, Sonia Laguna^*, Ričards Marcinkevičs, Julia E Vogt
^* denotes shared first authorship

NeurIPS: Thirty-Eighth Annual Conference on Neural Information Processing Systems

14.12.2024

LinkCode

General movements (GMs) are spontaneous, coordinated body movements in infants that offer valuable insights into the developing nervous system. Assessed through the Prechtl GM Assessment (GMA), GMs are reliable predictors for neurodevelopmental disorders. However, GMA requires specifically trained clinicians, who are limited in number. To scale up newborn screening, there is a need for an algorithm that can automatically classify GMs from infant video recordings. This data poses challenges, including variability in recording length, device type, and setting, with each video coarsely annotated for overall movement quality. In this work, we introduce a tool for extracting features from these recordings and explore various machine learning techniques for automated GM classification.

Daphné Chopard^*, Sonia Laguna^*, Kieran Chin-Cheong^*, Annika Dietz, Anna Badura, Sven Wellmann, Julia E Vogt
^* denotes shared first authorship

Findings Machine Learning for Health (ML4H) Symposium colocated with NeurIPS

13.12.2024

Link

Concept-based machine learning methods have increasingly gained importance due to the growing interest in making neural networks interpretable. However, concept annotations are generally challenging to obtain, making it crucial to leverage all their prior knowledge. By creating concept-enriched models that incorporate concept information into existing architectures, we exploit their interpretable capabilities to the fullest extent. In particular, we propose Concept-Guided Conditional Diffusion, which can generate visual representations of concepts, and Concept-Guided Prototype Networks, which can create a concept prototype dataset and leverage it to perform interpretable concept prediction. These results open up new lines of research by exploiting pre-existing information in the quest for rendering machine learning more human-understandable.

Alba Carballo-Castro, Sonia Laguna, Moritz Vandenhirtz, Julia E Vogt

NeurIPS 2024 Workshop Interpretable AI

12.12.2024

LinkCode

Recently, interpretable machine learning has re-explored concept bottleneck models (CBM), comprising step-by-step prediction of the high-level concepts from the raw features and the target variable from the predicted concepts. A compelling advantage of this model class is the user's ability to intervene on the predicted concept values, affecting the model's downstream output. In this work, we introduce a method to perform such concept-based interventions on already-trained neural networks, which are not interpretable by design, given an annotated validation set. Furthermore, we formalise the model's intervenability as a measure of the effectiveness of concept-based interventions and leverage this definition to fine-tune black-box models. Empirically, we explore the intervenability of black-box classifiers on synthetic tabular and natural image benchmarks. We demonstrate that fine-tuning improves intervention effectiveness and often yields better-calibrated predictions. To showcase the practical utility of the proposed techniques, we apply them to deep chest X-ray classifiers and show that fine-tuned black boxes can be as intervenable and more performant than CBMs.

Sonia Laguna^*, Ricards Marcinkevics^*, Moritz Vandenhirtz, Julia E. Vogt
^* denotes shared first authorship

NeurIPS: Thirty-Eighth Annual Conference on Neural Information Processing Systems

11.12.2024

LinkCode

Self-supervised learning (SSL) has emerged as a powerful approach for learning biologically meaningful representations of single-cell data. To establish best practices in this domain, we present a comprehensive benchmark evaluating eight SSL methods across three downstream tasks and eight datasets, with various data augmentation strategies. Our results demonstrate that SimCLR and VICReg consistently outperform other methods across different tasks. Furthermore, we identify random masking as the most effective augmentation technique. This benchmark provides valuable insights into the application of SSL to single-cell data analysis, bridging the gap between SSL and single-cell biology.

Philip Toma^*, Olga Ovcharenko^*, Imant Daunhawer, Julia Vogt, Florian Barkmann^†, Valentina Boeva^†
* denotes shared first authorship, ^† denotes shared last authorship

Preprint

06.11.2024

DOICode

Supervised learning has become a cornerstone of modern machine learning, yet a comprehensive theory explaining its effectiveness remains elusive. Empirical phenomena, such as neural analogy-making and the linear representation hypothesis, suggest that supervised models can learn interpretable factors of variation in a linear fashion. Recent advances in self-supervised learning, particularly nonlinear Independent Component Analysis, have shown that these methods can recover latent structures by inverting the data generating process. We extend these identifiability results to parametric instance discrimination, then show how insights transfer to the ubiquitous setting of supervised learning with cross-entropy minimization. We prove that even in standard classification tasks, models learn representations of ground-truth factors of variation up to a linear transformation. We corroborate our theoretical contribution with a series of empirical studies. First, using simulated data matching our theoretical assumptions, we demonstrate successful disentanglement of latent factors. Second, we show that on DisLib, a widely-used disentanglement benchmark, simple classification tasks recover latent structures up to linear transformations. Finally, we reveal that models trained on ImageNet encode representations that permit linear decoding of proxy factors of variation. Together, our theoretical findings and experiments offer a compelling explanation for recent observations of linear representations, such as superposition in neural networks. This work takes a significant step toward a cohesive theory that accounts for the unreasonable effectiveness of supervised deep learning.

Patrik Reizinger^*, Alice Bizeul^*, Attila Juhos^*, Julia E. Vogt, Randall Balestriero, Wieland Brendel, David Klindt
^* denotes shared first authorship

The Fifteenth International Conference on Learning Representations, ICLR 2025 (Oral)

04.11.2024

Link

Ričards Marcinkevičs^*, Kacper Sokol^*, Akhil Paulraj, Melinda A. Hilbert, Vivien Rimili, Sven Wellmann, Christian Knorr, Bertram Reingruber, Julia E. Vogt^†, Patricia Reis Wolfertstetter^†
* denotes shared first authorship, ^† denotes shared last authorship

medRxiv

29.10.2024

LinkDOI

Finding clusters of data points with similar characteristics and generating new cluster-specific samples can significantly enhance our understanding of complex data distributions. While clustering has been widely explored using Variational Autoencoders, these models often lack generation quality in real-world datasets. This paper addresses this gap by introducing TreeDiffusion, a deep generative model that conditions Diffusion Models on hierarchical clusters to obtain high-quality, cluster-specific generations. The proposed pipeline consists of two steps: a VAE-based clustering model that learns the hierarchical structure of the data, and a conditional diffusion model that generates realistic images for each cluster. We propose this two-stage process to ensure that the generated samples remain representative of their respective clusters and enhance image fidelity to the level of diffusion models. A key strength of our method is its ability to create images for each cluster, providing better visualization of the learned representations by the clustering model, as demonstrated through qualitative results. This method effectively addresses the generative limitations of VAE-based approaches while preserving their clustering performance. Empirically, we demonstrate that conditioning diffusion models on hierarchical clusters significantly enhances generative performance, thereby advancing the state of generative clustering models.

Jorge da Silva Goncalves, Laura Manduchi, Moritz Vandenhirtz, Julia E. Vogt

arXiv

22.10.2024

LinkDOI

Self-Supervised Learning (SSL) methods often consist of elaborate pipelines with hand-crafted data augmentations and computational tricks. However, it is unclear what is the provably minimal set of building blocks that ensures good downstream performance. The recently proposed instance discrimination method, coined DIET, stripped down the SSL pipeline and demonstrated how a simple SSL algorithm can work by predicting the sample index. Our work proves that DIET recovers cluster-based latent representations, while successfully identifying the correct cluster centroids in its classification head. We demonstrate the identifiability of DIET on synthetic data adhering to and violating our assumptions, revealing that the recovery of the cluster centroids is even more robust than the feature recovery.

Attila Juhos^*, Alice Bizeul^*, Patrik Reizinger^*, David Klindt, Randall Balestriero, Mark Ibrahim, Julia E Vogt, Wieland Brendel
^* denotes shared first authorship

NeurIPS 2024 Workshop: Self-Supervised Learning-Theory and Practice

10.10.2024

Link

The structure of many real-world datasets is intrinsically hierarchical, making the modeling of such hierarchies a critical objective in both unsupervised and supervised machine learning. Recently, novel approaches for hierarchical clustering with deep architectures have been proposed. In this work, we take a critical perspective on this line of research and demonstrate that many approaches exhibit major limitations when applied to realistic datasets, partly due to their high computational complexity. In particular, we show that a lightweight procedure implemented on top of pre-trained non-hierarchical clustering models outperforms models designed specifically for hierarchical clustering. Our proposed approach is computationally efficient and applicable to any pre-trained clustering model that outputs logits, without requiring any fine-tuning. To highlight the generality of our findings, we illustrate how our method can also be applied in a supervised setup, recovering meaningful hierarchies from a pre-trained ImageNet classifier.

Emanuele Palumbo, Moritz Vandenhirtz, Alain Ryser, Imant Daunhawer^†, Julia E. Vogt^†
† denotes shared last authorship

Preprint

10.10.2024

DOI

This paper introduces Diffuse-TreeVAE, a deep generative model that integrates hierarchical clustering into the framework of Denoising Diffusion Probabilistic Models (DDPMs). The proposed approach generates new images by sampling from a root embedding of a learned latent tree VAE-based structure, it then propagates through hierarchical paths, and utilizes a second-stage DDPM to refine and generate distinct, high-quality images for each data cluster. The result is a model that not only improves image clarity but also ensures that the generated samples are representative of their respective clusters, addressing the limitations of previous VAE-based methods and advancing the state of clustering-based generative modeling.

Jorge da Silva Gonçalves, Laura Manduchi, Moritz Vandenhirtz, Julia E. Vogt

ICML 2024 Workshop on Structured Probabilistic Inference & Generative Modeling

27.07.2024

Link

Concept Bottleneck Models (CBMs) have emerged as a promising interpretable method whose final prediction is based on intermediate, human-understandable concepts rather than the raw input. Through time-consuming manual interventions, a user can correct wrongly predicted concept values to enhance the model's downstream performance. We propose Stochastic Concept Bottleneck Models (SCBMs), a novel approach that models concept dependencies. In SCBMs, a single-concept intervention affects all correlated concepts. Leveraging the parameterization, we derive an effective intervention strategy based on the confidence region. We show empirically on synthetic tabular and natural image datasets that our approach improves intervention effectiveness significantly. Notably, we showcase the versatility and usability of SCBMs by examining a setting with CLIP-inferred concepts, alleviating the need for manual concept annotations.

Moritz Vandenhirtz^*, Sonia Laguna^*, Ricards Marcinkevics, Julia E. Vogt
^* denotes shared first authorship

ICML 2024 Workshop on Structured Probabilistic Inference & Generative Modeling, Workshop on Models of Human Feedback for AI Alignment, and Workshop on Humans, Algorithmic Decision-Making and Society

26.07.2024

Link

Patent Ductus Arteriosus (PDA) is a heart condition that affects newborn infants. One way to estimate the severity of PDA is by measuring the Cardiac Output (CO) of the infants. Current techniques for estimating CO include echocardiography or invasive methods. While invasive methods generally provide a more accurate CO estimate, they are difficult procedures for premature infants due to the infants’ size and potential complications. On the other hand, echocardiography CO estimates do not allow for continuous CO monitoring. In this work, we examine the relationship between Pulse Wave Transition Time (PWTT) and stroke volume (SV-VTI), a clinical measure from which it is possible to derive CO. In addition, we build a machine-learning model to predict SV-VTI from PWTT, which provides a novel way of estimating CO inexpensively and non-invasively.

Doriela Grabocka, Alain Ryser, Sven Wellman, Holger Michel, Julia E. Vogt

11th ACM Celebration of Women in Computing: womENcourage

26.06.2024

Link

Recent years have witnessed a rapid development of deep generative models for creating synthetic media, such as images and videos. While the practical applications of these models in everyday tasks are enticing, it is crucial to assess the inherent risks regarding their fairness. In this work, we introduce a comprehensive framework for benchmarking the performance and fairness of conditional generative models. We develop a set of metrics—inspired by their supervised fairness counterparts—to evaluate the models on their fairness and diversity. Focusing on the specific application of image upsampling, we create a benchmark covering a wide variety of modern upsampling methods. As part of the benchmark, we introduce UnfairFace, a subset of FairFace that replicates the racial distribution of common large-scale face datasets. Our empirical study highlights the importance of using an unbiased training set and reveals variations in how the algorithms respond to dataset imbalances. Alarmingly, we find that none of the considered methods produces statistically fair and diverse results. All experiments can be reproduced using our provided repository.

Mike Laszkiewicz, Imant Daunhawer, Julia E. Vogt^†, Asja Fischer^†, Johannes Lederer^†
† denotes shared last authorship

ACM Conference on Fairness, Accountability, and Transparency, 2024

05.06.2024

DOICode

Multimodal VAEs have recently gained significant attention as generative models for weakly-supervised learning with multiple heterogeneous modalities. In parallel, VAE-based methods have been explored as probabilistic approaches for clustering tasks. At the intersection of these two research directions, we propose a novel multimodal VAE model in which the latent space is extended to learn data clusters, leveraging shared information across modalities. Our experiments show that our proposed model improves generative performance over existing multimodal VAEs, particularly for unconditional generation. Furthermore, we propose a post-hoc procedure to automatically select the number of true clusters thus mitigating critical limitations of previous clustering frameworks. Notably, our method favorably compares to alternative clustering approaches, in weakly-supervised settings. Finally, we integrate recent advancements in diffusion models into the proposed method to improve generative quality for real-world images.

Emanuele Palumbo, Laura Manduchi, Sonia Laguna, Daphne Chopard, Julia E Vogt

ICLR: The Twelfth International Conference on Learning Representations

17.05.2024

LinkCode

Despite significant progress, evaluation of explainable artificial intelligence remains elusive and challenging. In this paper we propose a fine-grained validation framework that is not overly reliant on any one facet of these sociotechnical systems, and that recognises their inherent modular structure: technical building blocks, user-facing explanatory artefacts and social communication protocols. While we concur that user studies are invaluable in assessing the quality and effectiveness of explanation presentation and delivery strategies from the explainees' perspective in a particular deployment context, the underlying explanation generation mechanisms require a separate, predominantly algorithmic validation strategy that accounts for the technical and human-centred desiderata of their (numerical) outputs. Such a comprehensive sociotechnical utility-based evaluation framework could allow to systematically reason about the properties and downstream influence of different building blocks from which explainable artificial intelligence systems are composed – accounting for a diverse range of their engineering and social aspects – in view of the anticipated use case.

Kacper Sokol, Julia E. Vogt

Extended Abstracts of the 2024 ACM Conference on Human Factors in Computing Systems (CHI)

02.05.2024

LinkDOI

Pulmonary hypertension (PH) in newborns and infants is a complex condition associated with several pulmonary, cardiac, and systemic diseases contributing to morbidity and mortality. Thus, accurate and early detection of PH and the classification of its severity is crucial for appropriate and successful management. Using echocardiography, the primary diagnostic tool in pediatrics, human assessment is both time-consuming and expertise-demanding, raising the need for an automated approach. Little effort has been directed towards automatic assessment of PH using echocardiography, and the few proposed methods only focus on binary PH classification on the adult population. In this work, we present an explainable multi-view video-based deep learning approach to predict and classify the severity of PH for a cohort of 270 newborns using echocardiograms. We use spatio-temporal convolutional architectures for the prediction of PH from each view, and aggregate the predictions of the different views using majority voting. Our results show a mean F1-score of 0.84 for severity prediction and 0.92 for binary detection using 10-fold cross-validation and 0.63 for severity prediction and 0.78 for binary detection on the held-out test set. We complement our predictions with saliency maps and show that the learned model focuses on clinically relevant cardiac structures, motivating its usage in clinical practice. To the best of our knowledge, this is the first work for an automated assessment of PH in newborns using echocardiograms.

Hanna Ragnarsdottir^*, Ece Özkan Elsen^*, Holger Michel^*, Kieran Chin-Cheong, Laura Manduchi, Sven Wellmann^†, Julia E. Vogt^†
* denotes shared first authorship, ^† denotes shared last authorship

International Journal of Computer Vision

06.02.2024

LinkDOI