Lab Photo

SIRIUS Lab

Surgical & Intelligent Robot & Innovative Unmanned Systems Laboratory

SIRIUS Lab focuses on the intersection of artificial intelligence, robotics, and embodied intelligence. We design learning-enabled robotic algorithms and platforms for robot perception, reasoning, and dexterous manipulations. Our research aims to enable robots to understand dynamic environments, reason with multimodal information, and interact safely and skillfully with the physical world. By combining surgical data science, foundation models, and robot learning, SIRIUS Lab aims to advance the next generation of trustworthy, general-purpose robotic intelligence.
We have openings for postdocs, research assistants and visiting students. Please email us if you're interested!

News

Research ( show selected / show all by date / show all by topic )

Research Highlight

Surg-R1: A Multimodal Foundation Model with Hierarchical Reasoning for Interpretable Decision Support
Surg-R1: A Multimodal Foundation Model with Hierarchical Reasoning for Interpretable Decision Support
Surgical AI / Robot Perception / Robotic Learning
(* / †: indicates equal contribution.)
SurgΣ: A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligence
SurgΣ: A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligence
Z. Zeng*, M. Xu*, J. Jiang*, P. Guo*, Y. Xu, Z. Zhuo, C. H. Low, Y. He, D. Yang, C. Lin, Y. Gu, J. Guo, Y. Ban†, D. Xu†, Q. Dou†, Y. Jin†
arXiv, 2026 [page][paper]
SSP: Safety-guaranteed Surgical Policy via Joint Optimization of Behavioral and Spatial Constraints
SSP: Safety-guaranteed Surgical Policy via Joint Optimization of Behavioral and Spatial Constraints
J. Hu, Z. Guan, L. Song, K. Leelakunwet, H. Wang, W. Xiao, Q. Dou, Y. Ban
arXiv, 2026 [page][paper]
OMP: One-step Meanflow Policy with Directional Alignment
OMP: One-step Meanflow Policy with Directional Alignment
H. Fang, Y. Huang, Y. Zhao, P. Weng, X. Li, Y. Ban
arXiv, 2026 [page][paper]
Generalizable Coarse-to-Fine Robot Manipulation via Language-Aligned 3D Keypoints
Generalizable Coarse-to-Fine Robot Manipulation via Language-Aligned 3D Keypoints
J. Hu, L. Wang, S. Li, Y. Jiang, X. Li, P. Weng, Y. Ban
International Conference on Learning Representations (ICLR), 2026 [paper]
Concept Graph Neural Networks for Video Understanding and Risk Mitigation
Concept Graph Neural Networks for Video Understanding and Risk Mitigation
Y. Ban, J. Eckhoff, T. Ward, D. Hashimoto, O. Meireles, D. Rus, G. Rosman
IEEE Transactions on Medical Imaging (TMI), 2023 [page][code][paper]
TEsoNet: Knowledge Transfer from laparoscopic Sleeve Gastrectomy to the Ivor-Lewis Esophagectomy
TEsoNet: Knowledge Transfer from laparoscopic Sleeve Gastrectomy to the Ivor-Lewis Esophagectomy
J.A. Eckhoff*, Y. Ban*, G. Rosman, D.T. Müller, DA. Hashimoto, D. Rus, C. Bruns, HF Fuchs, O. Meireles
Surgical Endoscopy, 2023 [page][code][paper]
TransCenter: Transformers with Dense Queries for Multiple-Object Tracking
TransCenter: Transformers with Dense Queries for Multiple-Object Tracking
Y. Xu*, Y. Ban*, G. Delorme, C. Gan, D. Rus, X. Alameda-Pineda
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022 [page][code][paper]
SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgery
SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgery
Y. Ban, G. Rosman, J. Eckhoff, T. Ward, D. Hashimoto, T. Kondo, H. Iwaki, O. Meireles, D. Rus
IEEE Robotics and Automation Letters (RA-L) & ICRA, 2022 [page][code][paper]
Artificial Intelligence Prediction of Cholecystectomy Operative Course from Automated Identification of Gallbladder Inflammation
Artificial Intelligence Prediction of Cholecystectomy Operative Course from Automated Identification of Gallbladder Inflammation
T.M. Ward, D.A. Hashimoto, Y. Ban, G. Rosman, O. Meireles
Surgical Endoscopy, 2022 [page][code][paper]
Automated Operative Phase Identification in Peroral Endoscopic Myotomy
Automated Operative Phase Identification in Peroral Endoscopic Myotomy
T.M. Ward, D.A. Hashimoto, Y. Ban, D.W. Rattner, H. Inoue, K.D. Lillemoe, D. Rus, G. Rosman, O. Meireles
Surgical Endoscopy, 2021 [page][code][paper]
Computer Vision in Surgery
Computer Vision in Surgery
T.M. Ward, P. Mascagni, Y. Ban, G. Rosman, N. Padoy, O. Meireles, D.A. Hashimoto
Elsevier Surgery, 2021 [page][code][paper]
Variational Bayesian Inference for Audio-Visual Tracking of Multiple Speakers
Variational Bayesian Inference for Audio-Visual Tracking of Multiple Speakers
Y. Ban, X. Alameda-Pineda, L. Girin, R. Horaud
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019 [page][code][paper]
Tracking Multiple Audio Sources with the Von Mises Distribution and Variational EM
Tracking Multiple Audio Sources with the Von Mises Distribution and Variational EM
Y. Ban, X. Alameda-Pineda, C. Evers, R. Horaud
IEEE Signal Processing Letters (SPL), 2019 [page][code][paper]
A Deep Concept Graph Network for Interaction-Aware Trajectory Prediction
A Deep Concept Graph Network for Interaction-Aware Trajectory Prediction
Y. Ban*, X. Li*, G. Rosman, I. Gilitschenski, O. Meireles, S. Karaman, D. Rus
IEEE International Conference on Robotics and Automation (ICRA), 2022 [page][code][paper]
Holistic Surgical Phase Recognition with Hierarchical Input Dependent State Space Models
Holistic Surgical Phase Recognition with Hierarchical Input Dependent State Space Models
H. Wu, T. Wang, M. Lechner, R. Hasani, JA. Eckhoff, P. Pak, OR. Meireles, G. Rosman, Y. Ban, D. Rus
[page][code][paper]
GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects
GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects
S. Li, H. Zhang, X. Chen, Y. Wang†, Y. Ban†
[page][code][paper]
Time Reversal Symmetry for Efficient Robotic Manipulations in Deep Reinforcement Learning
Time Reversal Symmetry for Efficient Robotic Manipulations in Deep Reinforcement Learning
Y. Jiang, J. Hu, P. Weng†, Y. Ban†
Advances in Neural Information Processing Systems (NeurIPS), 2025 [page][code][paper]
Tracking-Aware Deformation Field Estimation for Non-rigid 3D Reconstruction in Robotic Surgeries
Tracking-Aware Deformation Field Estimation for Non-rigid 3D Reconstruction in Robotic Surgeries
Z. Wang, H. Fang, Y. Xu, Y. Ban
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2025 [page][code][paper]
Diffusion Stabilizer Policy for Automated Surgical Robot Manipulations
Diffusion Stabilizer Policy for Automated Surgical Robot Manipulations
C. Ho, J. Hu, H. Wang, Q. Dou, Y. Ban
IEEE International Conference on Robotics and Automation (ICRA), 2026 [page][code][paper]
Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery
Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery
L. Yin, Y. Ban, J. Eckhoff, O. Meireles, D. Rus, G. Rosman
IEEE International Conference on Robotics and Automation (ICRA), 2025 [page][code][paper]
Systematic Evaluation and Guidelines for Segment Anything Model in Surgical Video Analysis
Systematic Evaluation and Guidelines for Segment Anything Model in Surgical Video Analysis
C. Yuan*, J. Jiang*, K. Yang*, L. Wu, R. Wang, Z. Meng, H. Ping, Z. Xu, H. Wang, Y. Zhou, W. Song, Y. Jin, Q. Dou, Y. Ban
NPJ Digital Surgery, 2025 [page][code][paper]
Understanding and Reducing the Class-Dependent Effects of Data Augmentation with A Two-Player Game Approach
Understanding and Reducing the Class-Dependent Effects of Data Augmentation with A Two-Player Game Approach
Y. Jiang, Y. Ban†, P. Weng†
Transactions on Machine Learning Research (TMLR), 2025 [page][code][paper]
Surgical Scene Segmentation by Transformer with Asymmetric Feature Enhancement
Surgical Scene Segmentation by Transformer with Asymmetric Feature Enhancement
C. Yuan, Y. Ban
IEEE International Symposium on Biomedical Imaging (ISBI), 2025 [page][code][paper]
INViT: A Generalizable Routing Problem Solver with Invariant Nested View Transformer
INViT: A Generalizable Routing Problem Solver with Invariant Nested View Transformer
H. Fang, Z. Song, P. Weng†, Y. Ban†
International Conference on Machine Learning (ICML),2024 [page][code][paper]