Skip to main navigation Skip to search Skip to main content

A hybrid ST-ViT–driven multimodal architecture combining spatiotemporal MRI patterns and radiomic features for enhanced prediction of pCR in neoadjuvant breast cancer therapy

  • Satyabrata Pattanayak
  • , Tripty Singh
  • , Rishabh Kumar
  • , Ganesh R. Naik
  • Amrita Vishwa Vidyapeetham
  • Torrens University Australia

Research output: Contribution to journal › Article › peer-review

Abstract

Breast cancer response prediction plays a critical role in treatment planning, especially for identifying patients likely to achieve pathologic complete response (pCR). Traditional approaches rely primarily on baseline clinical variables or static imaging features, limiting their ability to capture the complex biological and temporal dynamics of tumor evolution during therapy. This study presents a spatiotemporal multimodal framework that integrates quantitative clinicopathological variables, radiomics, and longitudinal DCE-MRI to enhance the prediction of pCR. We employ a Spatiotemporal Vision Transformer (ST-ViT) to model tumor evolution across four imaging time points and fuse it with quantitative radiomic and clinical features. The proposed framework captures both spatial heterogeneity and treatment-induced temporal changes, offering a comprehensive representation of tumor biology. Texture-based radiomic analysis reveals meaningful differences between pCR and non-pCR tumors, while enhancement-curve dynamics further highlight early perfusion and washout patterns linked to treatment sensitivity. The integrated spatiotemporal multimodal model demonstrates strong discriminatory power, yielding an AUC of 0.98 during training and 0.96 on the held-out test set. These results highlight the model’s ability to leverage dynamic MRI signatures, radiomic texture descriptors, and clinical features distinguish responders from non-responders effectively. By capturing both spatial and temporal tumor evolution, the framework offers a robust and clinically meaningful tool for early identification of treatment-sensitive phenotypes and supports precision-driven neoadjuvant therapy planning.

Original languageEnglish
Article number93
Number of pages19
JournalBiology Direct
Volume21
Issue number1
DOIs
Publication statusPublished - 24 Apr 2026
Externally publishedYes

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

  1. SDG 3 - Good Health and Well-being
    SDG 3 Good Health and Well-being

Keywords

  • Breast cancer prediction
  • Clinical data fusion
  • DCE-MRI
  • Longitudinal imaging
  • Multimodal learning
  • Radiomics
  • Spatiotemporal vision transformer (ST-ViT)
  • pCR

Fingerprint

Dive into the research topics of 'A hybrid ST-ViT–driven multimodal architecture combining spatiotemporal MRI patterns and radiomic features for enhanced prediction of pCR in neoadjuvant breast cancer therapy'. Together they form a unique fingerprint.

Cite this